فهرست نشریات علمی دسترسی آزاد ایران

Decoding human brain activity evoked by visual stimuli has always been an interesting topic in cognitive neuroscience. In this paper, electroencephalographic (EEG) signals are employed to enhance the image classification accuracy and indirectly define the targets of the images, by fusing EEG-based regressed features and the features extracted from a deep neural network. To this end, we proposed an EEG encoder that included one Long Short-Term Memory (LSTM) layer followed by one convolutional layer to extract the representative features from EEG signals. Then, these EEG-based features are fused with those from a deep neural network (DNN) which is fed directly by the corresponding images. These fused feature vectors and predicted labels from the proposed encoder are employed to train an SVM-based classifier due to its generalization ability. In the test phase, the DNN-based visual feature is projected onto the proper EEG-based feature space via a regressor, and the fused feature vector is obtained and applied to the SVM. Evaluating the proposed model on the ImageNet-EEG dataset, which includes 40 classes of images, shows that the proposed encoder reaches an average accuracy of 99.35% for classifying EEG signals. The proposed human brain-guided system also improved image classification accuracy by over 10% compared to the deep neural network.