Evaluating learning algorithms a classification perspective Nathalie Japkowicz (University of Ottawa), Mohak Shah (McGill University)

Literaturverz. S. 393 - 402

"The field of machine learning has matured to the point where many sophisticated learning approaches can be applied to practical applications. Thus it is of critical importance that researchers have the proper tools to evaluate learning approaches and understand the underlying issues. This book examines various aspects of the evaluation process with an emphasis on classification algorithms. The authors describe several techniques for classifier performance assessment, error estimation and resampling, obtaining statistical significance as well as selecting appropriate domains for evaluation. They also present a unified evaluation framework and highlight how different components of evaluation are both significantly interrelated and interdependent. The techniques presented in the book are illustrated using R and WEKA facilitating better practical insight as well as implementation. Aimed at researchers in the theory and applications of machine learning, this book offers a solid basis for conducting performance evaluations of algorithms in practical settings"--

"Technological advances, in recent decades, have made it possible to automate many tasks that previously required signi.cant amounts of manual time, performing regular or repetitive activities. Certainly, computing machines have proven to be a great asset in improving on human speed and e.ciency as well as in reducing errors in these essentially mechanical tasks. More impressively, however, the emergence of computing technologies has also enabled the automation of tasks that require signi.cant understanding of intrinsically human domains that can, in no way, be qualified as merely mechanical. While we, humans, have maintained an edge in performing some of these tasks, e.g. recognizing pictures or delineating boundaries in a given picture, we have been less successful at others, e.g., fraud or computer network attack detection, owing to the sheer volume of data involved, and to the presence of nonlinear patterns to be discerned and analyzed simultaneously within these data. Machine Learning and Data Mining, on the other hand, have heralded significant advances, both theoretical and applied, in this direction, thus getting us one step closer to realizing such goals"--4222 $u1. Introduction; 2. Machine learning and statistics overview; 3. Performance measures I; 4. Performance measures II; 5. Error estimation; 6. Statistical significance testing; 7. Data sets and experimental framework; 8. Recent developments; 9. Conclusion; Appendix A: statistical tables; Appendix B: additional information on the data; Appendix C: two case studies.

Japkowicz, Nathalie: Evaluating Learning Algorithms

Japkowicz, Nathalie: Evaluating learning algorithms