System.Numerics Vector and Matrix types implementing operator interfaces

[StellarWolfEntertainment]

I have a suggestion regarding the implementation of interfaces in the System.Numerics namespace. Specifically, types like Vector2, Vector3, Vector4, and the Matrix types implement the operator overloads (e.g., operator * for multiplication), but they do not explicitly implement the corresponding IMultiplyOperators<TSelf, TOther, TReturn> interface, which would be a natural fit for these types.

As far as I understand, there is no downside to implementing an interface when the function it declares is already implemented. Therefore, I would like to request that these vector and matrix types explicitly implement these operator interfaces where appropriate. This would provide clearer contracts and better interoperability with generic code that relies on these interfaces.

Please let me know if I'm missing something or if there's a specific reason these interfaces aren't implemented.

[KeterSCP]

See #58162 and #76406

[StellarWolfEntertainment]

Perhaps the initial intent was lost because I am not great at writing, I'm not saying the just because something implements an operator 'x' that takes parameters of types X and Y and returns type Z that it should declare the matching interface, I'm simply saying that 'System.Numerics' defines a number of interfaces with the clear intent that they infer mathematical operations, and Vectors and Matrices while not numbers ARE mathematical constructs with which you can perform a number of mathematical operations, it makes little sense why a System.Numerics defined type wouldn't make use of the interfaces defined alongside it. to hopefully be a bit more clear about this, take for example the IDisposable interface, just because a type has a 'Dispose' method doesn't necessarily mean that the intent of the method is inline with the intent of IDisposable at which point it shouldn't, but if the intents were aligned then it should implement it

[tannergooding]

however that doesn't necessarily mean that a mathematical type shouldn't (like Vector2, 3, or 4) implement these,

I didn't say they shouldn't; I said they wouldn't until higher level interfaces are also defined that allow core/basic operations and versioning of API surface to work as expected.

You don't implement interfaces just to state that members are provided, you implement them to define a contract which often includes defining how certain operations behave. Sometimes such definitions are only useful in broader contexts.

Core functionality, such as being able to create or otherwise instantiate the T are only possible if you have a higher level definition.

public static T FMA<T>(T a, T b, T c) where T : IAdditionOperators<T, T, T>, IMultiplyOperators<T, T, T>

This is notably a great example of an API that shouldn't be defined given such constraints. For starters, FMA stands for FusedMultiplyAdd and that has a strict requirement that the multiply-addition be done as a single fused operation, not two separate operations. You could fix this by defining some MultiplyAdd which is explicitly a two step operation instead. -- This is correspondingly why the vector types define operator *, operator +, FusedMultiplyAdd, and MultiplyAddEstimate (which is explicit in its name and definition such that it can do FMA(a, b, c) or (a * b) + c depending on what the underlying hardware supports).

Correspondingly it's then an API where because of its special behavior (assuming you "fix" the name and whether it is doing a fused operation; or a hardware dependent "estimate" operation) the underlying raw operator interfaces are never sufficient. Instead you need some higher level type such as INumberBase<TSelf> where enough other guarantees exist that exposing some combined operation is sensible.

When it comes to the vector types, there are also other variations such as vector <op> scalar that are only sensible for vector types and due to how constraints and resolution works, wouldn't be viable to define as a type implementing both IMultiplyOperators<Vector2, Vector2, Vector2> and IMultiplyOperators<Vector2, float, Vector2>; instead you rather need some higher level IVector<TSelf, TElement> type that unifies and disambiguates so that they are usable. Such an interface also allows you to do other core/foundational operations such as creating an instance of the vector, indexing the vector, etc.

and Vectors and Matrices while not numbers ARE mathematical constructs with which you can perform a number of mathematical operations

In some senses, yes; while in other sense no. They do not strictly relate to the mathematical concepts and various special rules still apply; rules that are often important to understand.

These additional concepts and special rules are why a higher level IVector<TSelf, TElement> and IMatrix<TSelf, TElement> would be required prior to exposing the foundational operator interfaces on the types; as otherwise we are shipping an incomplete story and potentially backing ourselves into a corner for the design/feature that is actually needed longer term.

[StellarWolfEntertainment]

I am aware that FMA may not have been the best example, but it was the first thing I had thought of while writing, the point was that there are a number of possible types that could benefit from some function or another [admittedly due to the nature of fma it's not really a great option for this] that would need to be all defined separate despite identical internal structure [like (a * b) + c], at which point it really just becomes cumbersome to both write and for those that use tools like intellisense to sort through all of the potential overloads to see 'can I use 'x' with my type' [with generics and ensuring that it meets the requirements, yes, without it then you would have to define a new 'system' and write it for your own types, even though it would have benefited from a generic system], and while I do agree that interfaces are more than just 'it does this', it's a contract promising that it does that, which would allow others to create 'broader' systems for more than just the 'types' they know of 'right now', having to 'modify/expand' it every time something 'new' comes along that is a 'candidate' for that system. and an 'IVector' or 'IMatrix' can't properly exist within C#s current generic framework since both vectors and matrices are defined by both ElementType, AND their size, however this could be done by using 'sub interfaces' with a 'parent' 'IVector<TSelf, TElement>' type and 'sub' IVector2<TSelf, TElement> and other sizes, but here is a better example of a method that is specific about it's requirements, and if a user defines a type that implements all of the 'features' of the interfaces in a way that doesn't align with the intent that's on them

public static T Gaussian<T>(T x, T std, T dev) where T : IFloatingPoint<T>, IRootFunctions<T>, IExponentialFunctions<T>, IPowerFunctions<T>
{
    T two = T.One + T.One;
    T half = T.One / two; // this is a bit weird, but it works.

    return (T.One / (dev * T.Sqrt(T.Tau))) * T.Exp(-half * (T.Pow(x - std, two) / T.Pow(dev, two)));
}

[tannergooding]

the point was that there are a number of possible types that could benefit from some function or another

Right, which is why I continued on with the example and talked about had it been named something different and why the foundational interfaces alone aren't sufficient here. There's more needed when considering correctness, extensibility, etc.

at which point it really just becomes cumbersome to both write and for those that use tools like intellisense to sort through all of the potential overloads to see 'can I use 'x' with my type' [with generics and ensuring that it meets the requirements

This is part of the disconnect. As much as you might not want to do this, it is a fundamental requirement in many scenarios to ensure that the API follows good design practices, can be versioned over time, won't be prone to breaking changes, etc.

You can of course do whatever you want in your own libraries and applications, especially if it remains an internal implementation detail that can be freely changed in the future. The BCL, however, does not have this luxury; we get one opportunity to correctly define and ship an API before it is set in stone forevermore and we can't go and make binary breaks later. Because of this single opportunity to do things "correctly" and ensure that what we ship can stand the test of time and still be just as usable 20 years later, we end up taking a lot more care to ensure the designs are thought out, that they answer the current needs, potential future needs, and that they don't prevent us from doing the right thing later. We don't always get this right, but it is important the time, care, and consideration happens; and the result is that the vast majority of the BCL does stand the test of time.

In this case, there are known use-cases for generic vectors/matrices where eventual IVector or IMatrix like interfaces would be necessary. There are known planned expansions for some kind of struct Vector2<T> where T : IFloatingPointIeee754<T> or similar (such that double, half, and other types can be supported). The foundational interfaces as standalone constraints only work in places where you're guaranteed to only ever be doing a general 1-to-1 over the operation. Such as how they get used in Tensor<T> to indicate that Tensor<T> supports performing + for any where T : IAdditionOperators<T, T, T>. It isn't describing a behavior, just adopting the underlying operation as something that can be performed elementwise. However, it also defines a general ITensor<T> that gives you access to the other required functionality, such as creation, indexing, and beyond and in more practical use-cases a developer will use it as where TTensor : ITensor<T, TElement> and where TElement : INumber<TElement> or a similar higher level constraint that allows doing the appropriate contextual operations for TElement.

Because of all of this, it is a case where we will not be exposing just the raw operator interfaces on these types. We will wait until the rest of the design work can be done and considered to ensure that all the right functionality is in place so that when the interfaces are eventually exposed they check all the boxes and will provide a solid experience for all users, both now and in the future.

but here is a better example of a method that is specific about it's requirements

This is also problematic. What you're trying to do is to define an algorithm that works on the mathematical definition of Gaussian (normal) distribution. But, this purely mathematical algorithm isn't correct in the face of computers, the specialization that many types may need, and the concepts that don't strictly tie into pure math.

For example, the algorithm should not be assuming that (1/y) * x is the same as x/y, because it many cases it won't be. The algorithm shouldn't be trying to manually construct "constant" values using base operations, etc. It's also notably more expensive than the strict mathematical definition which would be closer to:

public static T Gaussian<T>(T x, T mean, T standardDeviation)
    where T : IFloatingPoint<T>, IRootFunctions<T>, IExponentialFunctions<T>
{
    T variance = standardDeviation * standardDeviation;
    T tmp = T.Exp(-(((x - mean) * (x - mean)) / (variance + variance)));
    return tmp / T.Sqrt(T.Tau * standardDeviation);
}

But even with this adjusted definition, it is still "naive" and can break down in various edge cases. It won't correctly handle core concepts that are relevant to typical types (like float or double), it won't be usable in practice for types like decimal because such a type doesn't (and won't ever) expose support for operations like Exp or Sqrt. So the practical is that this really should be constrained to simply where T : IFloatingPointIeee754<T> and then it should handle some of the edge cases around overflow, underflow, negative zero, infinities, nans, etc. This will result in an API that handles the things it should, is robust to future improvements, etc.

---

I'm not sure this is something we'll see eye to eye on. But at the end of the day, it is my responsibility to ensure that changes to the System.Numerics types are done "correctly" and for the best of the entire ecosystem both now and in the future.

This is a case where I strongly believe the only correct way to do it is to wait until the known necessary features and considerations are handled as part of the work; which means ensuring that it fits into the concepts of the future Vector2<T> and related types, that it covers the needs for long term versioning which includes handling concepts like vector op scalar and matrix op vector, that it allows creation of and indexing into the vector/matrix like types, etc.

This is on the general backlog to do, but has to be prioritized alongside other important work like Tensor<T>, continued improvement of generic math as a whole, providing vectorized implementations of core math functions (like the new Vector4.Sin APIs that shipped in .NET 9), continued exposure of additional hardware intrinsic APIs, and other items that are under my general area of ownership. It also has to fit cohesively into the overall design work to ensure the feature is doing what is necessary.

[StellarWolfEntertainment]

Fair enough, It's actually been a bit since I've attempted as large a scale generic concept in C#, I might be a bit rusty regarding edge cases, and I am aware that the BCL really does only get one shot to get it right, but I would point out that if vector and matrix types start with implementing interfaces that would inevitably end up being inherited by the 'higher level' interfaces like IVector then all that replacing them with IVector is doing is adding more to the definition itself not changing the existing definition, so an interface like 'IAdditionOperators' being inherited by IVector then implemented by Vector2 doesn't change the fact that Vector2 implements IAdditionOperators. Having said that I do also understand that sometimes it is necessary to wait until a larger feature is implemented before a smaller one can or should be.

[neon-sunset]

If this limitation impacts your code, you may want to consider implementing it at least partially in F# which can constrain generic arguments on specific operators without having to go through an interface constraint:

// Inferred signature of 'add':
// val inline add:
//   a: 'a (requires static member ( + ) ) ->
//   b: 'b (requires static member ( + ) )
//   -> 'c
let inline add a b = a + b

let lhs = Vector4(1f, 2f, 3f, 4f)
let rhs = Vector4(5f, 6f, 7f, 8f)
let test = add lhs rhs

printfn $"{test}"

[tannergooding]

Noting that while this "works" it runs into many of the general issues that exist with structural typing and the general premise of why there aren't plans to implement just the operator interfaces on any type that provides such operators.

That is, this isn't doing an "if it looks like a duck and quacks like a duck, it must be a duck" test. It is rather doing an "if it looks like a duck, it must be a duck" test instead. Such support can make writing some code simpler, especially if a type doesn't implement all the things required for nominal typing, but it can equally have problems where terms and concepts get overloaded. For example, collection types are typically enumerable, indexable, and expose a Count or Length property; but the same can be said for things like Euclidean vector types and some of those members (like Length) can mean very different things between the two concepts.

Such structural typing puts additional onus on the caller to ensure correctness and that what they're passing in also meets the "quacks like a duck" part of the structural test; otherwise undefined or confusing behavior may occur.

[StellarWolfEntertainment]

I do agree that 'structural typing' is not necessarily a great solution to my request, but large number of things in programming put the onus on the caller to ensure correctness every method and type has 'intent', you can use it in 'y' way, but it was meant to be used in 'x' way, as a kind of weird example, the concept of 'Length' could in theory apply even to scalar values, it's operation is the same as taking the absolute, but it's intent is different, with the Abs we are trying to get a 'non-negative' value, but with Length we want to know that magnitude of the value [how far from 0 it is, and that can't be negative] so while you CAN (and because the length of a number is not something that is a 'named' concept in most programming languages DO) get the magnitude of a number using Abs doesn't mean that, that's what it was meant for.

[StellarWolfEntertainment]

And when it comes to generics, we don't want 'if it looks, walks, and quacks like a duck' because then it just IS a duck, the whole point of generics is that it COULD be a duck, but it doesn't have to be, so long as it looks like one

[tannergooding]

the whole point of generics is that it COULD be a duck, but it doesn't have to be, so long as it looks like one

No. Generics explicitly require constraints (or dynamic type checks) to nominal contracts in order to do anything; you would have some where TDuck : IDuck because there can be many types of ducks and they may not be all be the same. You're accepting the concept of a duck, and know it must be some duck, but not necessarily more specifics about the duck. If you just have an unconstrained T then you cannot do value.Quack(), you would have to do if (value is IDuck duck) { duck.Quack(); } instead.

The same is true for things like where T : INumberBase<T>, you are explicitly accepting "some number" and not just accepting anything that resembles a number. That could be a BigInteger, byte, Complex, decimal, double, Half, short, int, long, nint, Int128, sbyte, float, ushort, uint, ulong, nuint, UInt128, or any number of custom number types defined by external libraries. You are explicitly something that declares its a number without specifically knowing what kind of number. If you want to exclude "complex numbers" and only constrain yourself to what are functionally "extended real numbers" then you instead use INumber<T>. If you only want to constrain yourself to integers then IBinaryInteger<T>, and so on.

Different interfaces provide different constraints and requirements. Some can stand on their own, such as IDisposable or IEquatable<T> because they put enough emphasis on the implementation that they can logically be used. Other interfaces do not quite stand on their own, outside specific scenarios. This includes things like IAdditionOperators because it doesn't require that this do numerical addition, only that the type provide some x + y operator. You need the higher level concept of INumberBase<T> to understand how it relates to IAdditiveIdentity, to concepts like One, Zero, etc. There are still places where such "nominal shapes" can be used and can be useful, such as how they're used for Tensor<T>; but they require much more care and consideration for how that applies to the API in question.

but large number of things in programming put the onus on the caller to ensure correctness every method and type has 'intent', you can use it in 'y' way

Yes and such APIs are often considered more dangerous and error prone. They also tend to have trouble being versioned and maintained over time.

There are times and places this is acceptable, but its not meant to be the primary mechanism and most scenarios should be using the more explicit constraints to ensure that all of the above can be achieved.