Abstract

are scanned from a copy of the paper (apologize for the poor quality). It is shown that the distribution of word frequencies for randomly generated texts is very similar to Zipf’s law observed in natural languages such as the English. The facts that the frequency of occurrence of a word is almost an inverse power law function of its rank and the exponent of this inverse power law is very close to 1 are largely due to the transformation from the word’s length to its rank, which stretches an exponential function to a power law function. key words: statistical linguistics, Zipf’s law, power-law distribution, random texts. Zipf observed long time ago [1] that the distribution of word frequencies in English, if the words are aligned according to their ranks, is an inverse power law with the exponent very close to 1. In other words, if the most frequently occurring word appears in the text with the frequency P(1), the next most frequently occurring word has the frequency P(2), and the rank-r word has the frequency P(r), the frequency distribution is P(r) = C rα, (1)

Citations

83	Selected Studies of the Principle of Relative Frequency in Language - Zipf - 1932
60	The Fractal Geometry of Nature (Freeman - Mandelbrot - 1982
56	An informational theory of the statistical structure of language - Mandelbrot - 1953
17	self-similarity and 1/f spectrum in dissipative dynamical systems - Manneville, Intermittency - 1980
6	The Peculiar Distribution of First Digits - RAIMI - 1969
4	Introduction” in The Psycho-biology of Language: An Introduction to Dynamic Philology - Miller - 1965
4	Mutual information functions of natural language texts,” Santa Fe Institute preprint 89-009 (1989). law 7 - Li
1	unpublished notes - Gell-Mann