Calculation of Precise Constants in a Probability Model of Zipf's Law Generation and Asymptotics of Sums of Multinomial Coefficients

Abstract

© 2017 Vladimir Bochkarev and Eduard Lerner.Let ω0,ω1,⋯,ωn be a full set of outcomes (symbols) and let positive pi, i=0,⋯,n, be their probabilities (∑i=0npi=1). Let us treat ω0 as a stop symbol; it can occur in sequences of symbols (we call them words) only once, at the very end. The probability of a word is defined as the product of probabilities of its symbols. We consider the list of all possible words sorted in the nonincreasing order of their probabilities. Let pr be the probability of the rth word in this list. We prove that if at least one of the ratios log pi/log pj, i,j ∈ {1,⋯,n}, is irrational, then the limit limr→∞pr/r-1/γ exists and differs from zero; here γ is the root of the equation ∑i=1n piγ=1. The limit constant can be expressed (rather easily) in terms of the entropy of the distribution (p1γ,⋯,pnγ).