Theory of Probability and Mathematical Statistics
A limiting Gaussian process in entropic perspective
Zhiyi Zhang
Link
Abstract: Entropic statistics is a collection of statistical methodologies with a special perspective: the underlying random system is described exclusively based on label-independent parameters, also known as entropies. Statistical models based on entropies are increasingly popular inspired by needs in modern data science. However the general theoretical support for statistical inference based on these models is porous and needs attention. In this article, several basic entropic objects are first introduced on a countable alphabet, including entropic parameters, general entropies, entropic moments, and an entropic moment-generating function. The main objective of this article is to establish the fact that a sample version of the entropic moment-generating function, properly normed, converges to a Gaussian process. The asymptotic Gaussian process may serve as an anchor point in probability for deriving entropy-based statistical inferential tools.
Keywords: Entropies, entropic moment-generating function, characterization of entropies, Rényi entropy, Tsallis entropy, Gaussian process
Bibliography:
C. E. Shannon, A mathematical theory of communication, Bell System Tech. J. 27 (1948), 379–423, 623–656. MR 26286, DOI 10.1002/j.1538-7305.1948.tb01338.x
Alfréd Rényi, On measures of entropy and information, Proc. 4th Berkeley Sympos. Math. Statist. and Prob., Vol. I, Univ. California Press, Berkeley-Los Angeles, Calif., 1960, pp. 547–561. MR 132570
Constantino Tsallis, Possible generalization of Boltzmann-Gibbs statistics, J. Statist. Phys. 52 (1988), no. 1-2, 479–487. MR 968597, DOI 10.1007/BF01016429
E. H. Simpson, Measurement of diversity, Nature 163 (1949), 688.
Zhiyi Zhang, Statistical implications of Turing’s formula, John Wiley & Sons, Inc., Hoboken, NJ, 2017. MR 3642750
M. O. Hill, Diversity and evenness: a unifying notation and its consequences, Ecology 54 (1973), 427–432.
J. M. Emlen, Ecology: an evolutionary approach, Addison-Wesley, Reading, 1973.
Joseph P. S. Kung, Gian-Carlo Rota, and Catherine H. Yan, Combinatorics: the Rota way, Cambridge Mathematical Library, Cambridge University Press, Cambridge, 2009. MR 2483561, DOI 10.1017/CBO9780511803895
Zhiyi Zhang, Several basic elements of entropic statistics, Entropy 25 (2023), no. 7, Paper No. 1060, 19. MR 4630429, DOI 10.3390/e25071060
Zhiyi Zhang, Several basic elements of entropic statistics, Entropy 25 (2023), no. 7, Paper No. 1060, 19. MR 4630429, DOI 10.3390/e25071060
