Nonparametric Maximum Likelihood Estimation: A Practical Guide to Mixture Models

Table of Links

Abstract and 1. Introduction

2. The Compound Decision Paradigm
3. Parametric Priors
4. Nonparametric Prior Estimation
5. Empirical Bayes Methods for Discrete Data
6. Empirical Bayes Methods for Panel Data
7. Conclusion

Appendix A. Tweedie’s Formula

Appendix B. Predictive Distribution Comparison

References

4. Nonparametric Prior Estimation

Parametric models for the prior, G, can be difficult to choose, so it is natural to ask whether some nonparametric procedure can be used to estimate G. An affirmative answer to this question can be traced back to an abstract of Robbins (1950), much more fully elaborated in Kiefer and Wolfowitz (1956). Some further development of the idea of nonparametric maximum likelihood estimation of mixture models was made by Pfanzagl (1988), but practical implementation of these methods was delayed until Laird (1978) showed how the nascent EM algorithm, Dempster et al (1977), could be used to compute it. Heckman and Singer (1984) pioneered the EM approach to explore the sensitivity to various parametric frailty models of duration models in econometrics. Lindsay (1981, 1995) further clarified many aspects of the NPMLE, but computation remained a bottleneck due to slow convergence of the EM algorithm. Fortunately, modern developments in convex optimization have substantially improved computational prospects for the NPMLE.

Identifiability of G in mixture models is thoroughly treated by Teicher (1961, 1967) for a scalar mixing parameter.

When the latent parameter θ is of dimension two interior point methods for computing the NPMLE are still feasible using gridding as illustrated in Gu and Koenker (2017b,a). However, beyond dimension two such methods become unwieldy and alternative first-order methods are probabily required. Recent progress in this direction can be found in Soloff et al (2021), Zhang et al (2022).