Online ensemble learning in the presence of concept drift

Abstract

In online learning, each training example is processed separately and then discarded. Environments that require online learning are often non-stationary and their underlying distributions may change over time (concept drift). Even though ensembles of learning machines have been used for handling concept drift, there has been no deep study of why they can be helpful for dealing with drifts and which of their features can contribute for that. The thesis mainly investigates how ensemble diversity affects accuracy in online learning in the presence of concept drift and how to use diversity in order to improve accuracy in changing environments. This is the first diversity study in the presence of concept drift. The main contributions of the thesis are: - An analysis of negative correlation in online learning. - A new concept drift categorisation to allow principled studies of drifts. - A better understanding of when, how and why ensembles of learning machines can help to handle concept drift in online learning. - Knowledge of how to use information learnt from the old concept to aid the learning of the new concept. - A new approach called Diversity for Dealing with Drifts (DDD), which is accurate both in the presence and absence of drifts.