Linguistic Complexity: English vs. Polish, Text vs. Corpus

J. Kwapieńa, S. Drożdża, b and A. Orczyka a Institute of Nuclear Physics, Polish Academy of Sciences, Kraków, Poland b Faculty of Mathematics and Natural Sciences, University of Rzeszów, Rzeszów, Poland

Full Text PDF

We analyze the rank frequency distributions of words in selected English and Polish texts. We show that for the lemmatized (basic) word forms the scale invariant regime breaks after about two decades, while it might be consistent for the whole range of ranks for the inflected word forms. We also find that for a corpus consisting of texts written by different authors the basic scale invariant regime is broken more strongly than in the case of comparable corpus consisting of texts written by the same author. Similarly, for a corpus consisting of texts translated into Polish from other languages the scale invariant regime is broken more strongly than for a comparable corpus of native Polish texts. Moreover, we find that if the words are tagged with their proper part of speech, only verbs show rank frequency distribution that is almost scale invariant.

DOI: 10.12693/APhysPolA.117.716 PACS numbers: 89.75.Da, 89.75.Fb