Empirical Methods for Artificial Intelligence

Computer science and artificial intelligence in particular have no curriculum inresearch methods, as other sciences do. This book presents empirical methods for studying complexcomputer programs: exploratory tools to help find patterns in data, experiment designs andhypothesis-testing tools to help data speak convincingly, and modeling tools to help explain data.Although many of these techniques are statistical, the book discusses statistics in the context ofthe broader empirical enterprise. The first three chapters introduce empirical questions,exploratory data analysis, and experiment design. The blunt interrogation of statistical hypothesistesting is postponed until chapters 4 and 5, which present classical parametric methods andcomputer-intensive (Monte Carlo) resampling methods, respectively. This is one of few books topresent these new, flexible resampling techniques in an accurate, accessible manner.Much of the bookis devoted to research strategies and tactics, introducing new methods in the context of casestudies. Chapter 6 covers performance assessment, chapter 7 shows how to identify interactions anddependencies among several factors that explain performance, and chapter 8 discusses predictivemodels of programs, including causal models. The final chapter asks what counts as a theory in AI,and how empirical methods -- which deal with specific systems -- can foster generaltheories.Mathematical details are confined to appendixes and no prior knowledge of statistics orprobability theory is assumed. All of the examples can be analyzed by hand or with commerciallyavailable statistics packages.The Common Lisp Analytical Statistics Package (CLASP), developed inthe author's laboratory for Unix and Macintosh computers, is available from The MIT Press.A BradfordBook