Is English Kurtotic?

You ever have a random text that sent your brain to work? Here’s mine today:

KD Text

Followed up with examples that lol is bimodal, while loop is positively skewed, and enter is “almost normal”. The lovely K.D. posed this question to me earlier, and I already have procrastinated a lot today, so here’s to more! First, I typed out some fonts in Word to help me figure out how to code the two important parts for this question: width and height. Webdings specifically by request. We have one serif font, one sans serif font, and one monospaced font.

Font Picture

I don’t have a perfect representation of this numerically, but for this analysis, we are going to compare everything to the letter a. Therefore, a is coded as a 1 for width and a 1 for height. For skinnier letters, like f, i, j, and l, they get a .5 for width. The letters m and w received 1.5 for serif and sans serif fonts for being wider than the rest. A mono spaced font like console received all 1s for width. Very strangely, the only skinny letter in Webdings is the letter m for the little man character. For height purposes, all readable fonts are the same, tall letters got a 2 for being tall. They don’t quite actually take up 2 height wise, but for the math here, 2 is a bit easier to work with.

font_size <- read.csv("font_sizes.csv", stringsAsFactors = F)
head(font_size)

##   letter sans_serif mono webdings height
## 1      a        1.0    1        1      1
## 2      b        1.0    1        1      2
## 3      c        1.0    1        1      1
## 4      d        1.0    1        1      2
## 5      e        1.0    1        1      1
## 6      f        0.5    1        1      2

Let’s take the word enter for a second in a sans or serif font (they were the same honestly wide and tall wise). To create a distribution for enter, we are going to do the following:

• e = 1 for the first letter * 1 for width = 1 repeated 1 for height
• n = 1 for e + 1 for width = 2 repeated 1 for height
• t = 1 for e + 1 for n + .5 for width = 2.5 repeated 2 for height
• e = 1 for e + 1 for n + .5 for t + 1 for width = 3.5 repeated 1 for height
• r = 1 for e + 1 for n + .5 for t + 1 for e + .5 for width = 4 repeated 1 for height

enter <- c(1, 2, 2.5, 2.5, 3.5, 4)
hist(enter, breaks = 5)

Again, not a perfect representation, but should do. Then we can calculate skew and kurtosis on the word:

library(moments)
skewness(enter)

## [1] -0.099779

kurtosis(enter)

## [1] 2.050242

So, almost no skew, and some kurtosis, but cut the word some slack with such a small sample size ;). In the Webdings font, this word would be perfectly uniform, as each character takes up the same height and space, since there’s no letter m.

enter2 <- c(1,2,3,4,5)
skewness(enter2)

## [1] 0

kurtosis(enter2)

## [1] 1.7

We will apply this algorithm to the English Lexicon Project with some caveats to only pick words with at least three letters, who are mostly real parts of speech, excluding some NULL values in the ELP data.

elp <- read.csv("elp_small.csv")
elp$word <- gsub("'", "", elp$word) #remove '
elp <- elp[!duplicated(elp$word) , ] #remove duplicates
elp$word <- tolower(elp$word) #lower case
elp$length <- nchar(elp$word) #recalculate length without '

elp <- subset(elp, 
              pos != "NULL" & length >= 3)

nrow(elp)

## [1] 71607

Now, we can create a function to calculate the skew and kurtosis for each word:

• First, create an empty dataframe to save the data in.
• Next, we are going to loop over all the words (this is slow, but whatever).
• For each word:
  • Break apart the letters.
  • mapvalues to convert the letters into numbers based on my scoring system.
  • Make those numbers with as.numeric.
  • Make those cumulative sequences by using the unfortunately named cumsum.
  • All the information about height by using rep to create the “tall” letters in the right places.
  • Calculate skew and kurtosis and save the output.

library(plyr)

final_stuff <- matrix(NA, nrow = length(elp$word), ncol = 7)
colnames(final_stuff) <- c("word", "s_sans", "s_mono", "s_webdings", 
                           "k_sans", "k_mono", "k_webdings")
final_stuff <- as.data.frame(final_stuff)

for (i in 1:length(elp$word)){
  
  letters <- unlist(strsplit(elp$word[i], ""))
  
  numbers_sans <- cumsum(as.numeric(suppressMessages(mapvalues(letters, font_size$letter, font_size$sans_serif))))
  numbers_mono <- cumsum(as.numeric(suppressMessages(mapvalues(letters, font_size$letter, font_size$mono))))
  numbers_webdings <- cumsum(as.numeric(suppressMessages(mapvalues(letters, font_size$letter, font_size$webdings))))
  
  repeating <- as.numeric(mapvalues(letters, font_size$letter, font_size$height))
  
  #height
  distribution_sans <- rep(numbers_sans, repeating)
  distribution_mono <- rep(numbers_mono, repeating)
  
  #all same height, so nothing to double
  distribution_webdings <- numbers_webdings
  
  final_stuff$word[i] <- elp$word[i]
  final_stuff$s_sans[i] <- skewness(distribution_sans)
  final_stuff$s_mono[i] <- skewness(distribution_mono)
  final_stuff$s_webdings[i] <- skewness(distribution_webdings)
  final_stuff$k_sans[i] <- kurtosis(distribution_sans)
  final_stuff$k_mono[i] <- kurtosis(distribution_mono)
  final_stuff$k_webdings[i] <- kurtosis(distribution_webdings)
    
}

write.csv(final_stuff, "final_distribution.csv", row.names = F)

Let’s see what happened! You should take all these results with a small grain of salt, given that the average length of words is 8.33. To better interpret the kurtosis plots, you should know that uniform distributions are ~ 1.76 kurtosis, while normal distributions have a kurtosis of around 3.

kurtosis(c(1,2,4,5,6,7,8))

## [1] 1.762602

kurtosis(c(1,2,3,3,3,3,3,4,5))

## [1] 3.06

##skew
mean(final_stuff$s_sans)

## [1] -0.0005320944

hist(final_stuff$s_sans)

#kurtosis
mean(final_stuff$k_sans)

## [1] 1.913

hist(final_stuff$k_sans)

##skew
mean(final_stuff$s_mono)

## [1] -0.01798692

hist(final_stuff$s_mono)

#kurtosis
mean(final_stuff$k_mono)

## [1] 1.821502

hist(final_stuff$k_mono)

##skew
mean(final_stuff$s_webdings)

## [1] 0.004592325

hist(final_stuff$s_webdings)

##kurtosis
mean(final_stuff$k_webdings)

## [1] 1.754978

hist(final_stuff$k_webdings)