Mismatch between code and paper.

[ziruizhu]

This line
p.data.addcdiv_(-group['lr'], m, h.add(group['eps']).clamp(max=group['rho']))
should be changed to the following to match the paper

                update = m.div(h.add(group['eps'])).clamp(max=group['rho'])
                p.data.add_(update, alpha=-group['lr'])

Upvote & Fund

• We're using Polar.sh so you can upvote and help fund this issue.
• We receive the funding once the issue is completed & confirmed by you.
• Thank you in advance for helping prioritize & fund our backlog.

[dylan-campbell]

I'm also curious about this: the code as written wouldn't clip negative curvature directions, which would lead to the same problem as the Newton optimiser, I would have thought. (The GNB estimator is PSD, so wouldn't need these to be clipped, but this is not the case for Hutchinson.)