The term “token” refers to the total number of words in a text, corpus etc, regardless of how often they are repeated. The term “type” refers to the number of distinct words in a text, corpus etc.4 Kas 2015
The term “token” refers to the total number of words in a text, corpus etc, regardless of how often they are repeated. The term “type” refers to the number of distinct words in a text, corpus etc.4 Kas 2015