where P(T|A) is the author language model, and P(A|D) is 1 over the number of authors. We’re going to use EM to estimate the best author language models, and the best author models are those that optimize the probability of the data (the documents), so the likelihood function is something like:

\prod_{D \in Documents} \prod{T \in D} \sum{A \in D} (P(T|A)P(A|D))^{freq(T,D)}

EM will give those estimates for P(T|A) that maximize the likelihood function. I assumed the documents are independent (product over documents), and so are the terms given the documents (product over terms). We’re estimating P(T|A), for every pair (term, author) in a document there are two possibilities: it was either written by the author (=1 occurrence), or not (=0 occurrences; i.e. it was written by one of the other authors in the document). I do this for every occurrence of a term in a document. So the steps are:

E-step:

e_{T,A} = \sum_D ( freq(T, D) * ( P(T|A)P(A|D) / ( \sum_{A’ \in D} P(T|A’)P(A’|D) ) ) )

M-step:

P_{new}(T|A) = e_{T,A} / ( \sum_{T’} e_{T’,A} )

The M-step simply normalizes the probabilities of each author model. I included a little Perl script to show what happens.

Picture taken by Alexander.