QPLayer: efficient differentiation of convex quadratic optimization

[fabinsch]

Together with @Bambade, we worked on this PR to implement QPLayer: efficient differentiation of convex quadratic optimization. The paper is publicly available in HAL (ref 04133055). We leverage primal-dual augmented Lagrangian
techniques for computing derivatives of both feasible and infeasible QPs.

QPLayer enables using a QP as a layer within standard learning architectures. More precisely, QPLayer differentiates over $\theta$ the primal and dual solutions of QP of the form

$$ \begin{align} \min_{x} & ~\frac{1}{2}x^{T}H(\theta)x+g(\theta)^{T}x \\ \text{s.t.} & ~A(\theta) x = b(\theta) \\ & ~l(\theta) \leq C(\theta) x \leq u(\theta) \end{align} $$

where $x \in \mathbb{R}^n$ is the optimization variable.

We provide in the file qplayer_sudoku.py an example that enables training QP layer in two different settings: (i) either we learn only the equality constraint matrix $A$, or (ii) we learn at the same time $A$ and $b$, such that $b$ is structurally in the range space of $A$. The procedure (i) is harder since a priori the fixed right-hand side does not ensure the QP to be feasible. Yet, this learning procedure is more structured, and for some problems can produce better predictions quicker (i.e., in fewer epochs).

In addition to the example to show the usage of the PyTorch implementation for QPLayer, we have added unit tests for the backward function which is implemented in C++ and we have updated the readme and doc.

EDIT: for now we have added a GPLv3 license to this specific contribution (namely the compute_ECJ.hpp and torch/qplayer.py)

[stephane-caron]

The GPLv3 license is viral, so if GPL-covered code is needed to compile the project, the whole project becomes GPL-licensed. To allow dual licensing, there should be some way, for instance a compilation option (similar to EIGEN_MPL2_ONLY in Eigen), to (1) compile without compute_ECJ.hpp, with a BSD-licensed result, or (2) compile with compute_ECJ.hpp, with a GPL-licensed result.

[fabinsch]

The GPLv3 license is viral, so if GPL-covered code is needed to compile the project, the whole project becomes GPL-licensed. To allow dual licensing, there should be some way, for instance a compilation option (similar to EIGEN_MPL2_ONLY in Eigen), to (1) compile without compute_ECJ.hpp, with a BSD-licensed result, or (2) compile with compute_ECJ.hpp, with a GPL-licensed result.

Thanks for the remark. After discussion, we decided not to change the licensing at all. Everything stays under BSD2-clause as before.

[fabinsch]

Now all the tests are passing. I had to switch to ClangCl toolset for windows-latest to avoid some build errors on windows.

[fabinsch]

after discussing with @jcarpent, the ci-linux-osx-wind-conda runs now the following setup successfully for Windows:

• windows-2019 | Release | c++17 | ClangCl

• windows-2019 | Release | c++20 | ClangCl

• windows-2019 | Debug | c++17 | ClangCl

• windows-2019 | Debug | c++20 | ClangCl

• windows-latest | Release | c++17 | v143

• windows-latest | Release | c++20 | ClangCl

• windows-latest | Debug | c++17 | v143

• windows-latest | Debug | c++20 | ClangCl

If that is fine we are good to merge.

[CLAassistant]

All committers have signed the CLA.