Theory of Probability and Mathematical Statistics
Information in additional observations of a non-parametric experiment that
Tilo Wiklund
Link
Abstract: Using Le Cam’s notion of deficiency between experiments, one can measure the value of obtaining an additional observation, when one has already been given independent and identically distributed observations. We show that for certain types of non-parametric experiments, the value of an additional observation decreases at a rate of 1/√n. This is distinct from the typical rate of 1/n in the case of parametric experiments, and from the non-decreasing value in the case of very large experiments. In particular, the rate of 1/√n holds for the experiment given by observing samples from a density about which we know only that it is bounded from below by some fixed positive constant. For such densities, there exists no estimator that is consistent in total variation distance. We have therefore an example of an experiment which is, in this sense, not estimable, but for which the value of additional observations tends to zero.
Keywords: Statistical information, deficiency, comparison of experiments
Bibliography: Luc Devroye and Gábor Lugosi, Combinatorial methods in density estimation, Springer Series in Statistics, Springer-Verlag, New York, 2001. MR 1843146, DOI 10.1007/978-1-4613-0125-7
Devdatt Dubhashi and Desh Ranjan, Balls and bins: a study in negative dependence, Random Structures Algorithms 13 (1998), no. 2, 99–124. MR 1642566, DOI 10.1002/(SICI)1098-2418(199809)13:2<99::AID-RSA1>3.0.CO;2-M
Jon Helgeland, Additional observations and statistical information in the case of 1-parameter exponential distributions, Z. Wahrsch. Verw. Gebiete 59 (1982), no. 1, 77–100. MR 643790, DOI 10.1007/BF00575527
L. Le Cam, Sufficiency and approximate sufficiency, Ann. Math. Statist. 35 (1964), 1419–1455. MR 207093, DOI 10.1214/aoms/1177700372
Lucien Le Cam, On the information contained in additional observations, Ann. Statist. 2 (1974), 630–649. MR 436400
L. Le Cam, On local and global properties in the theory of asymptotic normality of experiments, Proceedings of the Summer Research Institute on Statistical Inference for Stochastic Processes: Stochastic Processes and Related Topics, 1975, pp. 13–54.
Lucien Le Cam, Asymptotic methods in statistical decision theory, Springer Series in Statistics, Springer-Verlag, New York, 1986. MR 856411, DOI 10.1007/978-1-4612-4946-7
Lucien Le Cam and Grace Lo Yang, Asymptotics in statistics, 2nd ed., Springer Series in Statistics, Springer-Verlag, New York, 2000. Some basic concepts. MR 1784901, DOI 10.1007/978-1-4612-1166-2
Friedrich Liese and Klaus-J. Miescke, Statistical decision theory, Springer Series in Statistics, Springer, New York, 2008. Estimation, testing, and selection. MR 2421720
Friedrich Liese and Igor Vajda, On divergences and informations in statistics and information theory, IEEE Trans. Inform. Theory 52 (2006), no. 10, 4394–4412. MR 2300826, DOI 10.1109/TIT.2006.881731
M. G. Low and H. H. Zhou, A complement to Le Cam’s theorem, Ann. Statist. 35 (2007), no. 3, 1146–1165.
Enno Mammen, The statistical information contained in additional observations, Ann. Statist. 14 (1986), no. 2, 665–678. MR 840521, DOI 10.1214/aos/1176349945
Ester Mariucci, Le Cam theory on the comparison of statistical models, Grad. J. Math. 1 (2016), no. 2, 81–91. MR 3850766
Lutz Mattner, Mean absolute deviations of sample means and minimally concentrated binomials, Ann. Probab. 31 (2003), no. 2, 914–925. MR 1964953, DOI 10.1214/aop/1048516540
Peter McCullagh, What is a statistical model?, Ann. Statist. 30 (2002), no. 5, 1225–1310. With comments and a rejoinder by the author. MR 1936320, DOI 10.1214/aos/1035844977
Michael Nussbaum, Asymptotic equivalence of density estimation and Gaussian white noise, Ann. Statist. 24 (1996), no. 6, 2399–2430. MR 1425959, DOI 10.1214/aos/1032181160
Moshe Shaked and J. George Shanthikumar, Stochastic orders, Springer Series in Statistics, Springer, New York, 2007. MR 2265633, DOI 10.1007/978-0-387-34675-5
A. N. Shiryaev and V. G. Spokoiny, Statistical experiments and decisions, Advanced Series on Statistical Science & Applied Probability, vol. 8, World Scientific Publishing Co., Inc., River Edge, NJ, 2000. Asymptotic theory. MR 1791434, DOI 10.1142/9789812779243
Helmut Strasser, Mathematical theory of statistics, De Gruyter Studies in Mathematics, vol. 7, Walter de Gruyter & Co., Berlin, 1985. Statistical experiments and asymptotic decision theory. MR 812467, DOI 10.1515/9783110850826
Erik N. Torgersen, Measures of information based on comparison with total information and with total ignorance, Ann. Statist. 9 (1981), no. 3, 638–657. MR 615440
Erik Torgersen, Comparison of statistical experiments, Encyclopedia of Mathematics and its Applications, vol. 36, Cambridge University Press, Cambridge, 1991. MR 1104437, DOI 10.1017/CBO9780511666353
Roman Vershynin, High-dimensional probability, Cambridge Series in Statistical and Probabilistic Mathematics, vol. 47, Cambridge University Press, Cambridge, 2018. An introduction with applications in data science; With a foreword by Sara van de Geer. MR 3837109, DOI 10.1017/9781108231596
T. Wiklund, The deficiency introduced by resampling, Math. Methods Statist. 27 (2018), no. 2, 145–161. MR 3827358, DOI 10.3103/S1066530718020047
