Closed form gradients of log partition functions

[ismailsenoz]

We need to implement gradients of log partition functions for speed purposes. This is a relatively straightforward task that requires taking derivatives. It needs to be done for the entire list of distributions.

• Bernoulli @HoangMHNguyen
• Beta @bartvanerp
• Univariate Normal @bartvanerp
• Multivariate Normal @bartvanerp @Nimrais
• Wishart @Nimrais
• InverseWishart @Nimrais
• MvNormalWishart @Nimrais
• Contingency
• Dirichlet @wouterwln
• MatrixDirichlet
• Gamma @wouterwln
• Pareto @Sepideh-Adamiat
• Poisson @HoangMHNguyen
• Laplace @abpolym
• Exponential @HoangMHNguyen
• Log Normal @Sepideh-Adamiat
• Rayleigh
• Negative binomial
• Weibull @abpolym
• Von Mises @Nimrais
• Von Mises-Fisher @Nimrais
• Continuous Bernoulli @Nimrais
• Chi Squared
• Normal-Gamma @abpolym
• Geometric @wouterwln
• Erlang
• Multinomial
• Binomial @Sepideh-Adamiat
• Categorical

[ismailsenoz]

PhD students, @wouterwln @Nimrais @HoangMHNguyen @Sepideh-Adamiat @abpolym @bartvanerp, please assign yourselves to three distributions and make the derivations on LaTeX. After @bvdmitri revises and we merge his changes, you must translate your derivations to code. We will provide a template code for it beforehand. If you do not assign yourselves, you will be randomly assigned.

[bartvanerp]

Could you share an overleaf link to a shared document? Then we will have all derivations in one place for later. We could also use the old BIASlab rules document if you want

[bvdmitri]

Guys how do you feel about the timeframe for this issue? Can it be added to the next milestone?

[Nimrais]

@bvdmitri, I believe the only one bummer is the MvNormalWishart. There seem to be some minor suggestions to address in this PR: #149.

So if we will move MvNormalWishart out of the scope I think it's easy to finish it in this milestone.

[bvdmitri]

Thanks for the update @Nimrais ! Lets not block the entire progress because of one distribution. We move it out and address it later. I added the issue to the milestone.