Application of data mining techniques for predicting residents’ performance on pre‑board examinations: A case study

نویسندگان

1 Department of Emergency Medicine, Iran University of Medical Sciences, Tehran, Iran

2 Deputy of Specialty and Subspecialty Education and

3 Department of Medical Ethics, Iran University of Medical Sciences, Tehran, Iran

4 Health Laboratories Administration, Birjand University of Medical Sciences, Birjand, Iran

5 Department of Anesthesiology and Pain Medicine, Iran University of Medical Sciences, Tehran, Iran

6

7

doi
چکیده

CONTEXT: Predicting residents’ academic performance is critical for medical educational institutionsto plan strategies for improving their achievement.AIMS: This study aimed to predict the performance of residents on preboard examinations basedon the results of in‑training examinations (ITE) using various educational data mining (DM)techniques.SETTINGS AND DESIGN: This research was a descriptive cross‑sectional pilot study conducted atIran University of Medical Sciences, Iran.PARTICIPANTS AND METHODS: A sample of 841 residents in six specialties participating in theITEs between 2004 and 2014 was selected through convenience sampling. Data were collectedfrom the residency training database using a researcher‑made checklist.STATISTICAL ANALYSIS: The analysis of variance was performed to compare mean scoresbetween specialties, and multiple‑regression was conducted to examine the relationship betweenthe independent variables (ITEs scores in postgraduate 1st year [PGY1] to PG 3rd year [PGY3],sex, and type of specialty training) and the dependent variable (scores of postgraduate 4th yearcalled preboard). Next, three DM algorithms, including multi‑layer perceptron artificial neuralnetwork (MLP‑ANN), support vector machine, and linear regression were utilized to build theprediction models of preboard examination scores. The performance of models was analyzedbased on the root mean square error (RMSE) and mean absolute error (MAE). In the final step,the MLP‑ANN was employed to find the association rules. Data analysis was performed in SPSS22 and RapidMiner 7.1.001.RESULTS: The ITE scores on the PGY‑2 and PGY‑3 and the type of specialty training were thepredictors of scores on the preboard examination (R2 = 0.129, P < 0.01). The algorithm with theoverall best results in terms of measuring error values was MLP‑ANN with the condition of ten‑foldcross‑validation (RMSE = 0.325, MAE = 0.212). Finally, MLP‑ANN was utilized to find the efficientrules.CONCLUSIONS: According to the results of the study, MLP‑ANN was recognized to be useful in theevaluation of student performance on the ITEs. It is suggested that medical, educational databasesbe enhanced to benefit from the potential of DM approach in the identification of residents at risk,allowing instructors to offer constructive advice in a timely manner.