Theory of Probability and Mathematical Statistics
Higher-order nonlinear shrinkage estimator of large-dimensional precision matrix
Taras Bodnar and Nestor Parolya
Link
Abstract: This paper introduces a new type of nonlinear shrinkage estimators for the precision matrix in high-dimensional settings, where the dimension of the data-generating process exceeds the sample size. The proposed estimators incorporate the Moore–Penrose inverse and the ridge-type inverse of the sample covariance matrix, and they include linear shrinkage estimators as special cases. Recursive formulae of these higher-order nonlinear shrinkage estimators are derived using partial exponential Bell polynomials. Through simulation studies, the new methods are compared with the oracle nonlinear shrinkage estimator of the precision matrix for which no analytical expression is available.
Keywords: Moore–Penrose inverse, ridge-type inverse, nonlinear shrinkage, Bell polynomials, sample covariance matrix, random matrix theory, high-dimensional asymptotics
Bibliography: • Zhidong Bai and Jack W. Silverstein, Spectral analysis of large dimensional random matrices, 2nd ed., Springer Series in Statistics, Springer, New York, 2010. MR 2567175, DOI 10.1007/978-1-4419-0661-8
• Z. D. Bai and Jack W. Silverstein, CLT for linear spectral statistics of large-dimensional sample covariance matrices, Ann. Probab. 32 (2004), no. 1A, 553–605. MR 2040792, DOI 10.1214/aop/1078415845
• E. T. Bell, Partition polynomials, Ann. of Math. (2) 29 (1927/28), no. 1-4, 38–46. MR 1502817, DOI 10.2307/1967979
• E. T. Bell, Exponential polynomials, Ann. of Math. (2) 35 (1934), no. 2, 258–277. MR 1503161, DOI 10.2307/1968431
• Olha Bodnar, Taras Bodnar, and Nestor Parolya, Recent advances in shrinkage-based high-dimensional inference, J. Multivariate Anal. 188 (2022), Paper No. 104826, 13. MR 4353848, DOI 10.1016/j.jmva.2021.104826
• Taras Bodnar, Holger Dette, and Nestor Parolya, Spectral analysis of the Moore-Penrose inverse of a large dimensional sample covariance matrix, J. Multivariate Anal. 148 (2016), 160–172. MR 3493027, DOI 10.1016/j.jmva.2016.03.001
• Taras Bodnar, Holger Dette, and Nestor Parolya, Testing for independence of large dimensional vectors, Ann. Statist. 47 (2019), no. 5, 2977–3008. MR 3988779, DOI 10.1214/18-AOS1771
• Taras Bodnar, Solomiia Dmytriv, Yarema Okhrin, Nestor Parolya, and Wolfgang Schmid, Statistical inference for the expected utility portfolio in high dimensions, IEEE Trans. Signal Process. 69 (2021), 1–14. MR 4213326, DOI 10.1109/TSP.2020.3037369
• Taras Bodnar, Solomiia Dmytriv, Nestor Parolya, and Wolfgang Schmid, Tests for the weights of the global minimum variance portfolio in a high-dimensional setting, IEEE Trans. Signal Process. 67 (2019), no. 17, 4479–4493. MR 3999772, DOI 10.1109/TSP.2019.2929964
• Taras Bodnar, Arjun K. Gupta, and Nestor Parolya, On the strong convergence of the optimal linear shrinkage estimator for large dimensional covariance matrix, J. Multivariate Anal. 132 (2014), 215–228. MR 3266272, DOI 10.1016/j.jmva.2014.08.006
• Taras Bodnar, Arjun K. Gupta, and Nestor Parolya, Direct shrinkage estimation of large dimensional precision matrix, J. Multivariate Anal. 146 (2016), 223–236. MR 3477661, DOI 10.1016/j.jmva.2015.09.010
• Taras Bodnar, Ostap Okhrin, and Nestor Parolya, Optimal shrinkage estimator for high-dimensional mean vector, J. Multivariate Anal. 170 (2019), 63–79. MR 3913028, DOI 10.1016/j.jmva.2018.07.004
• Taras Bodnar and Yarema Okhrin, Properties of the singular, inverse and generalized inverse partitioned Wishart distributions, J. Multivariate Anal. 99 (2008), no. 10, 2389–2405. MR 2463397, DOI 10.1016/j.jmva.2008.02.024
• Taras Bodnar, Yarema Okhrin, and Nestor Parolya, Optimal shrinkage-based portfolio selection in high dimensions, J. Bus. Econom. Statist. 41 (2023), no. 1, 140–156. MR 4522158, DOI 10.1080/07350015.2021.2004897
• T. Bodnar and N. Parolya, Reviving pseudo-inverses: Asymptotic properties of large dimensional Moore–Penrose and ridge-type inverses with applications, 2024, arXiv preprint arXiv:2403.15792.
• Taras Bodnar, Nestor Parolya, and Wolfgang Schmid, Estimation of the global minimum variance portfolio in high dimensions, European J. Oper. Res. 266 (2018), no. 1, 371–390. MR 3737003, DOI 10.1016/j.ejor.2017.09.028
• Taras Bodnar, Nestor Parolya, and Erik Thorsén, Two is better than one: regularized shrinkage of large minimum variance portfolios, J. Mach. Learn. Res. 25 (2024), Paper No. [173], 32. MR 4777415
• Peter J. Brockwell and Richard A. Davis, Introduction to time series and forecasting, 2nd ed., Springer Texts in Statistics, Springer-Verlag, New York, 2002. With 1 CD-ROM (Windows). MR 1894099, DOI 10.1007/b97391
• Peter Bühlmann and Sara van de Geer, Statistics for high-dimensional data, Springer Series in Statistics, Springer, Heidelberg, 2011. Methods, theory and applications. MR 2807761, DOI 10.1007/978-3-642-20192-9
• Didier Chételat and Martin T. Wells, Improved multivariate normal mean estimation with unknown covariance when p is greater than n, Ann. Statist. 40 (2012), no. 6, 3137–3160. MR 3097972, DOI 10.1214/12-AOS1067
• R. Dennis Cook and Liliana Forzani, On the mean and variance of the generalized inverse of a singular Wishart matrix, Electron. J. Stat. 5 (2011), 146–158. MR 2786485, DOI 10.1214/11-EJS602
• Elvira Di Nardo, Giuseppe Guarino, and Domenico Senato, A unifying framework for p-statistics, polykays and their multivariate generalizations, Bernoulli 14 (2008), no. 2, 440–468. MR 2544096, DOI 10.3150/07-BEJ6163
• Bradley Efron and Carl Morris, Families of minimax estimators of the mean of a multivariate normal distribution, Ann. Statist. 4 (1976), no. 1, 11–21. MR 403001
• Yiyong Feng and Daniel P. Palomar, A signal processing perspective on financial engineering, Found. Trends Signal Process. 9 (2015), no. 1-2, ©2016, front matter+231. MR 3540800, DOI 10.1561/2000000072
• Gabriel Frahm and Christoph Memmel, Dominating estimators for minimum-variance portfolios, J. Econometrics 159 (2010), no. 2, 289–302. MR 2733122, DOI 10.1016/j.jeconom.2010.07.007
• V. Golosnoy and Y. Okhrin, Multivariate shrinkage for optimal portfolio weights, Eur. J. Fin. 13 (2007), no. 5, 441–458.
• L. R. Haff, An identity for the Wishart distribution with applications, J. Multivariate Anal. 9 (1979), no. 4, 531–544. MR 556910, DOI 10.1016/0047-259X(79)90056-3
• Shinpei Imori and Dietrich von Rosen, On the mean and dispersion of the Moore-Penrose generalized inverse of a Wishart matrix, Electron. J. Linear Algebra 36 (2020), 124–133. MR 4089045, DOI 10.13001/ela.2020.5091
• K. Krishnamoorthy and A. K. Gupta, Improved minimax estimation of a normal precision matrix, Canad. J. Statist. 17 (1989), no. 1, 91–102 (English, with French summary). MR 1014094, DOI 10.2307/3314766
• Tatsuya Kubokawa and Muni S. Srivastava, Estimation of the precision matrix of a singular Wishart distribution and its application in high-dimensional data, J. Multivariate Anal. 99 (2008), no. 9, 1906–1928. MR 2466543, DOI 10.1016/j.jmva.2008.01.016
• O. Ledoit and M. Wolf, Nonlinear shrinkage of the covariance matrix for portfolio selection: Markowitz meets Goldilocks, The Review of Financial Studies 30 (2017), no. 12, 4349–4388.
• Olivier Ledoit and Michael Wolf, Analytical nonlinear shrinkage of large-dimensional covariance matrices, Ann. Statist. 48 (2020), no. 5, 3043–3065. MR 4152634, DOI 10.1214/19-AOS1921
• Olivier Ledoit and Michael Wolf, Shrinkage estimation of large covariance matrices: keep it simple, statistician?, J. Multivariate Anal. 186 (2021), Paper No. 104796, 24. MR 4308803, DOI 10.1016/j.jmva.2021.104796
• Olivier Ledoit and Michael Wolf, Quadratic shrinkage for large covariance matrices, Bernoulli 28 (2022), no. 3, 1519–1547. MR 4411501, DOI 10.3150/20-bej1315
• V. A. Marčenko and L. A. Pastur, Distribution of eigenvalues for some sets of random matrices, Mathematics of the USSR-Sbornik 1 (1967), no. 4, 457.
• Robb J. Muirhead, Aspects of multivariate statistical theory, Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, Inc., New York, 1982. MR 652932, DOI 10.1002/9780470316559
• Jamal Najim and Jianfeng Yao, Gaussian fluctuations for linear spectral statistics of large random covariance matrices, Ann. Appl. Probab. 26 (2016), no. 3, 1837–1887. MR 3513608, DOI 10.1214/15-AAP1135
• Guangming Pan, Comparison between two types of large sample covariance matrices, Ann. Inst. Henri Poincaré Probab. Stat. 50 (2014), no. 2, 655–677 (English, with English and French summaries). MR 3189088, DOI 10.1214/12-AIHP506
• Axel Ruhe, Perturbation bounds for means of eigenvalues and invariant subspaces, Nordisk Tidskr. Informationsbehandling (BIT) 10 (1970), 343–354. MR 273802, DOI 10.1007/bf01934203
• Jack W. Silverstein, Strong convergence of the empirical distribution of eigenvalues of large-dimensional random matrices, J. Multivariate Anal. 55 (1995), no. 2, 331–339. MR 1370408, DOI 10.1006/jmva.1995.1083
• M. S. Srivastava, Singular Wishart and multivariate beta distributions, Ann. Statist. 31 (2003), no. 5, 1537–1560. MR 2012825, DOI 10.1214/aos/1065705118
• Charles Stein, Inadmissibility of the usual estimator for the mean of a multivariate normal distribution, Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 1954–1955, vol. I, Univ. California Press, Berkeley-Los Angeles, Calif., 1956, pp. 197–206. MR 84922
• J. Tong, R. Hu, J. Xi, Z. Xiao, Q. Guo, and Y. Yu, Linear shrinkage estimation of covariance matrices using low-complexity cross-validation, Signal Processing 148 (2018), 223–233.
• Cheng Wang, Guangming Pan, Tiejun Tong, and Lixing Zhu, Shrinkage estimation of large dimensional precision matrix using random matrix theory, Statist. Sinica 25 (2015), no. 3, 993–1008. MR 3409734
• Ruo-yong Yang and James O. Berger, Estimation of a covariance matrix using the reference prior, Ann. Statist. 22 (1994), no. 3, 1195–1211. MR 1311972, DOI 10.1214/aos/1176325625
• Jianfeng Yao, Shurong Zheng, and Zhidong Bai, Large sample covariance matrices and high-dimensional data analysis, Cambridge Series in Statistical and Probabilistic Mathematics, vol. 39, Cambridge University Press, New York, 2015. MR 3468554, DOI 10.1017/CBO9781107588080
• Shurong Zheng, Zhidong Bai, and Jianfeng Yao, Substitution principle for CLT of linear spectral statistics of high-dimensional sample covariance matrices with applications to hypothesis testing, Ann. Statist. 43 (2015), no. 2, 546–591. MR 3316190, DOI 10.1214/14-AOS1292
