Logistic regression is a fundamental task in machine learning and statistics. For the simple case of linear models, Hazan et al. (2014) showed that any logistic regression algorithm that estimates model weights from samples must exhibit exponential dependence on the weight magnitude. As an alternative, we explore a counterintuitive technique called improper learning, whereby one…
Adaptive gradient-based optimizers such as AdaGrad and Adam are among the methods of choice in modern machine learning. These methods maintain second-order statistics of each model parameter, thus doubling the memory footprint of the optimizer. In behemoth-size applications, this memory overhead restricts the size of the model being used as well as the number of…
Bayesian models are applied to probabilistic analysis of phenomena which deal with multiple external stochastic factors and unmeasurable variables. Considering the large amount of available data for the EV driving, recharging and grid services such as solar charging which contains uncertainties and measurement errors, and their hierarchical effect on the battery life, this application of…
Abstract: Chao Gao will discuss the problem of statistical estimation with contaminated data. In the first part of the talk, I will discuss depth-based approaches that achieve minimax rates in various problems. In general, the minimax rate of a given problem with contamination consists of two terms: the statistical complexity without contamination, and the contamination…
Bayesian inference for non-Gaussian state-space models is a ubiquitous problem with applications ranging from geophysical data assimilation to mathematical finance. We will discuss how deterministic couplings between probability distributions enable new solutions to this problem. We first consider filtering in high-dimensional models with nonlinear (potentially chaotic) dynamics and sparse observations in space and time. While…
Statistical learning theory has largely focused on learning and generalization given independent and identically distributed (i.i.d.) samples. Motivated by applications involving time-series data, there has been a growing literature on learning and generalization in settings where data is sampled from an ergodic process. This work has also developed complexity measures, which appropriately extend Rademacher complexity…
Abstract: In many social science experiments, subjects often interact with each other and as a result, one unit's treatment can influence the outcome of another unit. Over the last decade, a significant progress has been made towards causal inference in the presence of such interference between units. In this talk, we will discuss two-stage randomized…
Flexible resources such as batteries and demand-side management technologies are needed to handle future large shares of variable renewable power. Wind and solar power introduce more short-term uncertainty that have to be considered when making investment decisions as it significantly impacts the value of flexible resources. In this work we present a method for using…
Abstract: We consider the problem of efficient sampling from the hard-core and Potts models from statistical physics. On certain families of graphs, phase transitions in the underlying physics model are linked to changes in the performance of some sampling algorithms, including Markov chains. We develop new sampling and counting algorithms that exploit the phase transition…