Meta-Adversarial Training of Neural Networks for Binary Classification

We propose a novel framework for classification using neural networks via an adversarial training procedure, in which we simultaneously train a main classifier—a neural network that solves the original classification task, i.e classifying instances into two main categories—and two meta-classifiers which act as discriminators and aim to detect false positives and negatives predicted by the original classifier. Our framework operates in two stages: In a first stage, both main and meta classifiers are pre-trained using the cross-entropy loss. The second stage consists of an adversarial training stage in which both main and meta classifiers are placed in a min-max game. Therefore, we switch to our new loss function so that the goal for the main classifier becomes to maximize the probability of failure of the adversarial meta-classifiers. Our training procedure can be explained by the fact that the meta-classifiers are more accurate when the main classifier is weak i.e., instances misclassified by the main classifier are naturally easy to separate and assign to the correct class membership. Opposingly, if the main classifier is robust enough, then the meta-classifiers are supposed to distinguish between instances that are naturally hard to classify, making thus more mistakes. In this work, both main and metaclassifiers are defined by Multi-Layer Perceptrons (MLP) and the entire training system is performed using backpropagation with gradient descent optimization. Experiments demonstrate the potential of our framework in outperforming the traditional learning scheme in improving the classification accuracy.