فهرست نشریات علمی دسترسی آزاد ایران

Stochastic Gradient Descent algorithms are a cornerstone of large-scale optimization, yet their application to problems with explicit constraints remains a significant challenge. While methods based on relaxed barrier functions offer a promising alternative that avoids computationally expensive projections, they often suffer from sub-optimal convergence. Due to the inherent stochastic noise, their convergence stagnates at a certain level of accuracy, failing to achieve the optimal rate.This paper proposes Tailed-Average SGD, a novel modification designed to recover this optimal rate. Our algorithm integrates the classical Polyak-Ruppert averaging technique, but critically, it applies this averaging only after a predefined initial ”burn-in” period ($K_{burn}$).This ”tailed” approach strategically discards the initial, high-error iterates from the transient phase, which we demonstrate can ”poison” a naive averaging scheme applied from k = 0. We provide a rigorous theoretical analysis proving that TA-SGD achieves the best of both worlds: it preserves the almost sure convergence guarantees of the baseline method, while simultaneously succeeding in noise cancellation to achieve the optimal Mean Squared Error rate of O(1/K).Our numerical experiments on a large-scale quadratic program validate these theoretical claims, demonstrating that TA-SGD successfully overcomes this stagnation. It significantly outperforms both the baseline algorithm (which stagnates) and the naive full-averaging scheme (which converges prohibitively slowly).