Theory of Probability and Mathematical Statistics
Distributional hyperspace-convergence of Argmin-sets in convex M-estimation
Dietmar Ferger
Link
Abstract: In M-estimation we consider the sets of all minimizing points of convex empirical criterion functions. These sets are random closed sets. We derive distributional convergence in the hyperspace of all closed subsets of the real line endowed with the Fell-topology. As a special case single minimizing points converge in distribution in the classical sense. In contrast to the literature so far, unusual rates of convergence and non-normal limits emerge, which go far beyond the square-root asymptotic normality. Moreover, our theory can be applied to the sets of zero-estimators.
Keywords: M-estimation, convex empirical processes, Argmin-sets, random closed sets, Fell-topology
Bibliography: N. Albrecht, Least squares estimation for binary decision trees, PhD thesis, Technische Universität Dresden, 2020.
P. K. Andersen and R. D. Gill, Cox’s regression model for counting processes: a large sample study, Ann. Statist. 10 (1982), no. 4, 1100–1120. MR 673646, DOI 10.1214/aos/1176345976
Peter J. Bickel, Chris A. J. Klaassen, Ya’acov Ritov, and John A. Wellner, Efficient and adaptive estimation for semiparametric models, Springer-Verlag, New York, 1998. Reprint of the 1993 original. MR 1623559
Patrick Billingsley, Convergence of probability measures, John Wiley & Sons, Inc., New York-London-Sydney, 1968. MR 233396
Dietmar Ferger, A continuous mapping theorem for the argmin-set functional with applications to convex stochastic processes, Kybernetika (Prague) 57 (2021), no. 3, 426–445. MR 4299457, DOI 10.14736/kyb-2021-3-0426
Bert Fristedt and Lawrence Gray, A modern approach to probability theory, Probability and its Applications, Birkhäuser Boston, Inc., Boston, MA, 1997. MR 1422917, DOI 10.1007/978-1-4899-2837-5
Peter Gänssler and Winfried Stute, Wahrscheinlichkeitstheorie, Springer-Verlag, Berlin-New York, 1977 (German). MR 501219, DOI 10.1007/978-3-642-66749-7
Shelby J. Haberman, Concavity and estimation, Ann. Statist. 17 (1989), no. 4, 1631–1661. MR 1026303, DOI 10.1214/aos/1176347385
Nils Lid Hjort, Bayes estimators and asymptotic efficiency in parametric counting process models, Scand. J. Statist. 13 (1986), no. 2, 63–85. MR 867355
N.L. Hjort and D. Pollard, Asymptotic for minimizers of convex processes, Preprint, Dept. of Statistics, Yale University, 1993. http://arxiv.org/abs/1107.3806v1.
Peter J. Huber, Robust estimation of a location parameter, Ann. Math. Statist. 35 (1964), 73–101. MR 161415, DOI 10.1214/aoms/1177703732
Peter J. Huber, Robust statistics, Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, Inc., New York, 1981. MR 606374, DOI 10.1002/0471725250
Jana Jurečková, Asymptotic relations of M-estimates and R-estimates in linear regression model, Ann. Statist. 5 (1977), no. 3, 464–472. MR 433698
Jana Jurečková, Estimation in a linear model based on regression rank scores, J. Nonparametr. Statist. 1 (1992), no. 3, 197–203. MR 1241522, DOI 10.1080/10485259208832521
Keith Knight, What are the limiting distributions of quantile estimators?, Statistical data analysis based on the L
1-norm and related methods (Neuchâtel, 2002) Stat. Ind. Technol., Birkhäuser, Basel, 2002, pp. 47–65. MR 2001304
Friedrich Liese and Klaus-J. Miescke, Statistical decision theory, Springer Series in Statistics, Springer, New York, 2008. Estimation, testing, and selection. MR 2421720
Ilya Molchanov, Theory of random sets, Probability Theory and Stochastic Modelling, vol. 87, Springer-Verlag, London, 2017. Second edition of [ MR2132405]. MR 3751326, DOI 10.1007/978-1-4471-7349-6
Hung T. Nguyen, An introduction to random sets, Chapman & Hall/CRC, Boca Raton, FL, 2006. MR 2220761, DOI 10.1201/9781420010619
Constantin P. Niculescu and Lars-Erik Persson, Convex functions and their applications, CMS Books in Mathematics/Ouvrages de Mathématiques de la SMC, vol. 23, Springer, New York, 2006. A contemporary approach. MR 2178902, DOI 10.1007/0-387-31077-0
Wojciech Niemiro, Asymptotics for M-estimators defined by convex minimization, Ann. Statist. 20 (1992), no. 3, 1514–1533. MR 1186263, DOI 10.1214/aos/1176348782
Tommy Norberg, Convergence and existence of random set distributions, Ann. Probab. 12 (1984), no. 3, 726–732. MR 744229
H. Nyquist, The optimal L
p-normnorm estimator in linear regression models, Comm. Statist. A—Theory Methods 12 (1983), no. 21, 2511–2524. MR 715180, DOI 10.1080/03610928308828618
Johann Pfanzagl, Parametric statistical theory, De Gruyter Textbook, Walter de Gruyter & Co., Berlin, 1994. With the assistance of R. Hamböker. MR 1291393, DOI 10.1515/9783110889765
David Pollard, Empirical processes: theory and applications, NSF-CBMS Regional Conference Series in Probability and Statistics, vol. 2, Institute of Mathematical Statistics, Hayward, CA; American Statistical Association, Alexandria, VA, 1990. MR 1089429, DOI 10.1214/cbms/1462061091
David Pollard, Asymptotics for least absolute deviation regression estimators, Econometric Theory 7 (1991), no. 2, 186–199. MR 1128411, DOI 10.1017/S0266466600004394
R. Tyrrell Rockafellar, Convex analysis, Princeton Mathematical Series, No. 28, Princeton University Press, Princeton, NJ, 1970. MR 274683, DOI 10.1515/9781400873173
R. Tyrrell Rockafellar and Roger J.-B. Wets, Variational analysis, Grundlehren der mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 317, Springer-Verlag, Berlin, 1998. MR 1491362, DOI 10.1007/978-3-642-02431-3
Robert J. Serfling, Approximation theorems of mathematical statistics, Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, Inc., New York, 1980. MR 595165, DOI 10.1002/9780470316481
N. V. Smirnov, Limit distributions for the terms of a variational series, Amer. Math. Soc. Translation 1952 (1952), no. 67, 64. MR 47277
