Maximum Likelihood Estimation Under Constraints: Singularities and Random Critical Points

We investigate the procedure of semi-parametric maximum likelihood estimation under constraints on summary statistics. Such a procedure results in a discrete probability distribution supported on the data points that maximizes the likelihood among all distributions supported on the data points satisfying the specified constraints (called estimating equations). The resultant distribution is an approximation of the underlying population distribution. The study of such empirical likelihood estimation originates from the seminal work of Owen [1], [2]. We investigate this procedure in the setting of misspecified (or biased) constraints, i.e., when the null hypothesis is not true. We establish that the behavior of the optimal weight distribution under such misspecification differ markedly from their properties under the null, i.e., when the estimating equations are correctly specified (or unbiased). This is manifested by certain “singularities” in the optimal distribution, that are not observed under the null. Furthermore, we establish an anomalous behavior of the log-likelihood based Wilks’ statistic, which, unlike under the null, does not exhibit a chi-squared limit. In the Bayesian setting, we establish the posterior consistency of procedures based on these ideas, where instead of a parametric likelihood, an empirical likelihood is used to define the posterior distribution. In particular, we show that this posterior, as a random probability measure, rapidly converges, with explicit convergence guarantees, to the delta measure at the true parameter value. We also illustrate implications of our results in diverse settings such as degeneracies in exponential random graph models (ERGM) for random networks [3], [4], empirical procedures where the constraints are themselves estimated from data [5], and to approximate Bayesian computation based procedures [6], [7]. A novel feature of our work is to connect the likelihood maximization problem to critical points of random polynomials. This yields the mass of the singular weight in the optimal weight distribution as the leading term in a canonical expansion of a critical point of a random polynomial. Our work unveils the possibility that similar random polynomial based techniques could be effective in analyzing a wide class of problems in related areas.