oneminusp.com Computational Finance, Markets, Programming & co

Empirical probabilities Entropy function

Just added this simple but often useful entropy function which can be run on any data set with multiple occurrences of symbols. Say your data is the vector

c(1,1,1,2,2,2,3,3,3)

then

entropy.count

calculates the Shannon entropy using empirical/maximum likelihood probabilities for all unique symbols 1,2,3.

entropy.count <- function(entry) {
	counts <- lapply(split(entry, as.factor(entry)), length)
	counts <- unlist(counts)
	ps <- counts / sum(counts)
	entropy(ps)
}

The code is now also available through the "Papers / Code" tab above.