Benchmarks

I was inspired by this benchmark. I am not too sure how to do 100% fair comparisons across languages^[1]. There is a small overhead for using PyCall and RCall. I checked that it was small in my experimentation (~ few milliseconds?).

I test only the Gaussian Mixture case since it is the most common type of mixture (remember that this package allows plenty of other mixtures).

In the code, I did not use (too much) fancy programming tricks, the speed only comes from Julia, LogExpFunctions.jl for logsumexp! function and fit_mle for each distribution's coming from Distributions.jl package.

Univariate Gaussian mixture with 2 components

I compare with Sklearn.py^[2], mixtool.R, mixem.py^[3]. I wanted to try mclust, but I did not manage to specify initial conditions

Overall, mixtool.R and mixem.py were constructed in a similar spirit as this package, making them easy to use for me. Sklearn.py is built to match the Sklearn format (all in one). GaussianMixturesModel.jl is built with a similar vibe.

If you have comments to improve these benchmarks, they are welcome.

You can find the benchmark code here.

Conclusion: for Gaussian mixtures, ExpectationMaximization.jl is about 2 to 10 times faster than Python or R implementations and about as fast as the specialized Julia package GaussianMixturesModel.jl.

I did compare with R microbenchmark and Python timeit and they produced very similar timing but in my experience BenchmarkTools.jl is smarter and simpler to use, i.e. it will figure out alone the number of repetition to do in function of the run.

Last, the step-by-step likelihood of Sklearn is not the same as outputted by ExpectationMaximization.jl and mixtool.R (both agree), so I am a bit suspicious.