Optimizing Deep Q-Networks with fuzzy inference-based adaptive replay buffer management
نویسندگان
1 Department of Computer Engineering, Lorestan University, Khorramabad, Iran
2 Department of Computer Engineering, Technical and Vocational University (TVU), Tehran, Iran.
doi
10.22111/ijfs.2025.9358چکیده
Deep reinforcement learning algorithms, such as Deep Q-Networks (DQN), require careful tuning of replay memory parameters. In standard DQN implementations, these parameters remain fixed, which conflicts with the dynamic nature of the learning process where environmental conditions and reward stability continuously change. This mismatch often results in unstable learning or slow convergence. In this paper, we present a fuzzy logic-based system for adaptively adjusting three key replay memory parameters: memory size, the ratio of recent samples, and priority weight. The proposed fuzzy system evaluates the agent’s state by monitoring reward variations and average training errors, and accordingly updates these parameters to maintain optimal values during training. To assess the effectiveness of the proposed approach, we compared it with conventional DQN and PER-DQN methods across three benchmark reinforcement learning environments: CartPole v1, LunarLander v2, and Taxi v3. Experimental and statistical analyses demonstrate that our method improves average rewards, reduces training time, and enhances learning stability.