Word Frequency Distributions

Word frequency distributions are characterized by very large numbers of rare words, a property which leads to unusual statistical phenomena. Special statistical techniques for their analysis can be found in various technical journals. Baayen's (U. of Nijmegen, The Netherlands) text is intended to make these specialized techniques more accessible to non-specialists. Coverage includes some basic concepts and notation; non-parametric methods for the analysis of word frequency distributions; detailed descriptions of three parametric models (the lognormal model, the Yule-Simon Zipfian model, and the generalized inverse Gauss-Poisson model); mixture distributions; the effect of non-randomness in word use on the accuracy of the non- parametric and parametric models; and examples of applications. Background knowledge in statistics, probability theory, and some knowledge of elementary calculus is required. Each chapter includes exercises, with solutions provided in an appendix. Includes a CD-ROM. This text is for computational linguists, corpus linguists, psycholinguists, and researchers in the field of quantitative statistics. Annotation c. Book News, Inc., Portland, OR (booknews.com)