Composite Difference-Max Programs for Modern Statistical Estimations

Many modern statistical estimation problems are defined by  three major components: a statistical model that postulates the dependence of an output variable on the input features; a loss function measuring the error between the observed output and the model predicted output; and a regularizer that controls the overfitting and/or variable selection in the model.  We study the sampled version of this generic statistical estimation problem where the model parameters are estimated by empirical risk minimization.  In our setup we allow all three component functions to be of the difference-of-convex type and illustrate them with a host of commonly used examples, including those in continuous piecewise affine regression and in deep learning with piecewise affine activation functions.  We describe a non-monotone majorization-minimization (MM) algorithm for solving the unified nonconvex, nondifferentiable optimization problem. Numerical results are presented to demonstrate the effectiveness of the proposed algorithm and the superiority of continuous piecewise affine regression over the standard linear model.  

CUI Ying is currently an assistant professor of the Department of Industrial and Systems Engineering at the University of Minnesota. Her research focuses on the mathematical foundation of data science with emphasis on optimization techniques for operations research, machine learning and statistical estimations. Prior to UMN, she was a postdoc research associate at the University of Southern California. She received her Ph.D from the Department of Mathematics at the National University of Singapore.