A generalization of Shannon's Mutual information for improved feature selection in databases involving possibly rare but well-predictable classes

Abstract

Feature selection is a critical step in many Artificial Intelligence and Pattern Recognition problems. Shannon's Mutual Information is a classical and widely used measure of dependence measure that can serve as a good feature selection algorithm. However, rare yet important regularities can be overlooked by this measure, which can create critical misses (false negatives) especially in the applications of biomedical fields, in which it is assumed that many factors contribute but in small amounts to the target function to learn. We propose a new measure of relevance which accounts for predictability of each signal from the other in the calculations which improves the feature detection capability. Also, this measure, in its formulation, turns out to be a generalization of Shannon's Mutual Information. © 2013 Elsevier B.V., All rights reserved.