Drop Python 3.9 support

[ElliottKasoar]

Resolves #374

As I mention in the issue, we should think carefully about when we want to merge this, but I wanted to check it's also a viable solution to our CI problems, and these changes shouldn't clash too much with ongoing work that may be merged before this.

• Drops Python 3.9 support, remove Python 3.9 from testing matrices, and apply fixes to satisfy ruff
• Adds in missing from __future__ import annotations, which relied on a fix only available for Python 3.10+

Note: if merged after #376, those changes can be reverted.

[alinelena]

personally I think we drop 3.9 we add 3.13...

[ElliottKasoar]

personally I think we drop 3.9 we add 3.13...

It would be nice (and was my original intent) to add 3.13 when we drop 3.9, but it depends in what sense we support 3.13. PyTorch won't fully support it for MacOS/Windows until at least PyTorch 2.6, so we wouldn't be able to fully support it yet in that sense.

I also need to finally decide on a solution for matgl (M3GNet) to be able to move on from PyTorch 2.2, otherwise we definitely can't.

[oerc0122]

personally I think we drop 3.9 we add 3.13...

It would be nice (and was my original intent) to add 3.13 when we drop 3.9, but it depends in what sense we support 3.13. PyTorch won't fully support it for MacOS/Windows until at least PyTorch 2.6, so we wouldn't be able to fully support it yet in that sense.

I also need to finally decide on a solution for matgl (M3GNet) to be able to move on from PyTorch 2.2, otherwise we definitely can't.

Anyone running 3.13 must also run PyTorch 2.6+ so I'm not sure that's a problem as that's with their environment and should be effectively handled by pip and PyTorch's minimum_python. Provided PyTorch 2.6 is compatible with other things, it's not a problem.

[ElliottKasoar]

personally I think we drop 3.9 we add 3.13...

It would be nice (and was my original intent) to add 3.13 when we drop 3.9, but it depends in what sense we support 3.13. PyTorch won't fully support it for MacOS/Windows until at least PyTorch 2.6, so we wouldn't be able to fully support it yet in that sense.
I also need to finally decide on a solution for matgl (M3GNet) to be able to move on from PyTorch 2.2, otherwise we definitely can't.

Anyone running 3.13 must also run PyTorch 2.6+ so I'm not sure that's a problem as that's with their environment and should be effectively handled by pip and PyTorch's minimum_python. Provided PyTorch 2.6 is compatible with other things, it's not a problem.

I think about this slightly different to most dependency resolution issues, in that I'd like our tests to run 3.13 if we support it, whereas for other things that pip/uv resolves, that isn't something we need to worry about.

Since testing on MacOS isn't merged yet (#334), and that has its own Python matrix anyway (I imagine Windows will be similar), this isn't necessarily a problem, but "torch<=2.2,>=2.1" in our pyproject.toml for matgl is blocking.

[oerc0122]

I think about this slightly different to most dependency resolution issues, in that I'd like our tests to run 3.13 if we support it, whereas for other things that pip/uv resolves, that isn't something we need to worry about.

Since testing on MacOS isn't merged yet (#334), and that has its own Python matrix anyway (I imagine Windows will be similar), this isn't necessarily a problem, but "torch<=2.2,>=2.1" in our pyproject.toml for matgl is blocking.

And how does uv's dependency groups handle it? if matgl && python >= 3.13 fall over?

[ElliottKasoar]

I think about this slightly different to most dependency resolution issues, in that I'd like our tests to run 3.13 if we support it, whereas for other things that pip/uv resolves, that isn't something we need to worry about.
Since testing on MacOS isn't merged yet (#334), and that has its own Python matrix anyway (I imagine Windows will be similar), this isn't necessarily a problem, but "torch<=2.2,>=2.1" in our pyproject.toml for matgl is blocking.

And how does uv's dependency groups handle it? if matgl && python >= 3.13 fall over?

I can't test 3.13 very easily yet, as there isn't a PyTorch version compatible with my Mac yet, but there shouldn't be a resolution with matgl at the moment.

Independently from us (I think I included it first, otherwise it wouldn't have been necessary), they set an upper limit on PyTorch: https://github.com/materialsvirtuallab/matgl/blob/main/pyproject.toml#L56, which will never be compatible with 3.13.

The problem is that they rely on dgl, but that now requires individual links for the relevant version of torch/cuda (e.g. https://data.dgl.ai/wheels/torch-2.4/repo.html), and no apparent plans to publish to PyPI anymore: dmlc/dgl#7825, so they're stuck at the last version released to PyPI.

In theory we can now create conflicting groups with uv. I'm looking into it, but I'm not sure there'll be a sensible/useful resolution, given that these are quite major conflicts, so I'm also wondering if it just makes sense to drop automatic installation support for matgl, requiring manual installation instead.

Would there be any objections to that (@alinelena)?

[ElliottKasoar]

It seems like the latest version of MACE is also no longer compatible with 3.9: ACEsuit/mace#699, in a way that pip/uv are unaware of, so I think we probably do need to drop 3.9 now, and hopefully we can add 3.13 as soon as possible.