Benchmarking data mining methods in CAT

Abstract

In this study, a ranking test problem of Computer Adaptive Testing (CAT) is benchmarked by employing three popular classifiers: Artificial Neural Network (ANN), Support Vector Machines (SVMs), and Adaptive Network Based Fuzzy Inference System (ANFIS) in terms of ordinal classification performances. As the pilot test, History of Civilization class which offered in Bahcesehir University is selected. Item Response Theory (IRT) is focused for the determination of system inputs which are item responses of students, item difficulties of questions, and question levels. Item difficulties of questions are Gaussian normalized to make ordinal decisions. The distance between predicted and expected class values is employed for accuracy estimation. Comparison study is conducted to the ordinal class prediction correctness and performance analysis which is observed by Receiver Operating Characteristic (ROC) graphs. The results show that ANFIS has better performance and higher accuracy than ANN and SVMs in terms of ordinal question classification when the ordinal decisions are practically made by Gaussian Normal Distribution and ROC graphs are focused to observe any significant difference among the performances of classifiers. © 2012 Springer-Verlag. © 2012 Elsevier B.V., All rights reserved.