PCA by Shrinkage Estimation: A Comprehensive Mathematical and Statistical Analysis

نویسندگان

1 Payam Noor uinversity

2 Department of Mathematics and Computer Sciences, Iran University of Science and Technology, P.O. Box 16846, Tehran, Iran

doi
10.22054/jdsm.2025.86705.1074
چکیده

Principal Component Analysis (PCA) is a cornerstone technique for dimensionalityreduction and data analysis. However, classic PCA can exhibit instability inhigh-dimensional settings where the number of variables significantly exceeds thenumber of observations. Shrinkage-based PCA addresses this limitation by incorporatingregularization into the covariance matrix estimation process, leading tomore stable and interpretable results. This paper provides a robust mathematicaland statistical foundation for shrinkage-based PCA, compares its performance withclassic PCA, and demonstrates its advantages through theoretical analysis, numericalsimulations, and real-world data experiments. It is important to note that using the idea of a contraction estimator increases the efficiency of the estimator. mean time in this paper, it is shown that the covariance matrix estimator resulting from the contraction estimator is very efficient.It is also worth mentioning that to increase the efficiency of the contraction estimator, the recently discussed interval contraction estimator can be used.keywords: principal component analysis, Shrinkage-based, Estimation, Covariance Structures, Simulation.