Future Rx.NET Packaging #2038
Replies: 15 comments 46 replies
-
#2034 illustrates one possible implementation. It makes the following choices:
To be clear, we've not decided that we will do it this way. This is just a prototype to show how one solution to this problem would look, and to verify that it does indeed fix the 30MB installer problem that we're setting out to solve. (It does.) We don't much like the name |
Beta Was this translation helpful? Give feedback.
-
How widespread is the use of I presume if As for the clean start, there would be a lot of clashes due to the use of extension methods, right? For example, if old Rx and new Rx needed to be interoperated, whose |
Beta Was this translation helpful? Give feedback.
-
I think this is possibly more tolerable than you anticipate. This wouldn't break old versions of people's apps, only new versions when they upgrade. I think it's ok to introduce breaking changes, especially considering the benefits, and advantages over alternative options. Perhaps it's the point of this discussion, but maybe a survey would help? Let your users tell you whether they think this approach is a problem, you might find that most are ok with it. As an example, the last few versions of MediatR have introduced breaking changes. These have been potentially frustrating, but ultimately they are significant improvements and I think users appreciate them. |
Beta Was this translation helpful? Give feedback.
-
@idg10 thanks for creating this discussion, I think we should really exhaust all possibilities here to make sure we get the best decision for the community. I wholeheartedly agree with your option 1). By default we should try to save For completeness, I would like to submit to discussion a hybrid alternative first suggested in #2034 (comment) that I think does go a little bit in the direction of what you would like to achieve ideally. Essentially, consider the possibility of Rx v7 being a multi-TFM package (same as today) with The reason this is somewhat nice is that TFMs are still one of the most used and well understood ways of controlling the dependency graph on NuGet. If you say your app needs to run on .NET 7.0 then you simply can't pull .NET 8.0 dependencies by mistake. This means that your scenario of irremediably breaking something by accidentally upgrading the Rx.NET package won't happen unless you also at the same time update the TFM for your app. For me this is actually a big deal, since I don't think people change TFMs lightly, especially on client-side apps. If they do, dependencies usually need to be properly audited anyway (exactly because dependency graphs can change) and I don't think people should be surprised to find things can break: after all that is the point of versioning dependency trees by TFM. You might say this is not strictly backwards compatible because of platform unification during dependency resolution stage, but to me this is still better than the other two extreme options in terms of combining compatibility with a clean break. This way someone creating a new .NET 8.0 app today will finally have a sane starting point. At the same time someone maintaining an older app can still feel confident upgrading the Rx package to the latest version, since it won't break their dependency tree as long as they keep to their existing TFM. This seems like an acceptable compromise, as it would give even people on existing TFMs a chance to start upgrading and more easily explore the possibility of bumping TFMs by testing and identifying any problems, while contacting any legacy project maintainers or forking problematic dependencies throughout the coming year. |
Beta Was this translation helpful? Give feedback.
-
Separation of concernsFrom my point of view this would be best approached with multiple Core and Platform specific packages, there could even be multiple core packages to split the core up into specific functional areas such as Schedulers. Branch Divide and ConquerPerhaps the best way to start from a new base is to create a new Legacy branch within this Repository and make that the on-going patch and support base for the current state of affairs. My Reactive HistoryI have been using Reactive coding since 2013 and have used it in various areas; ASP.Net, WPF and a Windows Service were the first implementations used in conjunction with SignalR working with David Fowler to ensure that SignalR delivered the features I needed to use within my company's projects at the time. The FutureGoing forward I would like to see Interfaces (the design surface) extracted from the functionality to allow for an extendable and replaceable functionality framework. The ability to create functions that are functionally different but still operate within the Reactive mechanism is important for flexibility. I myself have come across issues within the Reactive codebase that when trying to add a reasonably small change to the built in functionality lead to having to fork the entire codebase as the 'Onion Coding Style' used just lead to another layer of dependency until eventually I realised that there was no way of making the alteration as an extension of the core without entirely rebuilding it. ReactiveUIWithin ReactiveUI we have core functionality exposed under a series of Interfaces however the framework loading mechanism is built around enabling replaceable functionality, this allows easier support for various UI Frameworks and extensibility / alteration of functionality to meet the requirements of the end user. We have a Core package then a series of UI framework support packages to either alter or extend the core functionality. Avalonia is an example of this flexibility where they have used ReactiveUI as a base and then made extensions fitting to their UI Framework. Desired Core PackagesFollowing a similar package pattern to Bonsai
Specific Microsoft UI Implementations If RelevantThis may be better handled by a UI framework such as ReactiveUI, but then no specific UI implementation should take place in
NOTE these are my suggestions based upon my experiences of using Rx over the years. Over the past few years there was little advancement in the DotNet Rx world and its lead people to think it's not the right way for them. |
Beta Was this translation helpful? Give feedback.
-
Stop me if this has already been considered, elsewhere... A more traditional approach to breaking changes would be to mark certain APIs as Considering the running example, This is of course still a breaking change, but perhaps a more acceptable one. This also requires tolerating the core issue of packaging bloat for a while longer, instead of actually solving it, but the trade-off is to alleviate the pain of the breaking change for most consumers. Does this make sense? Are there any other categories of changes in play here that wouldn't be compatible with this example? |
Beta Was this translation helpful? Give feedback.
-
All of these cures seem infinitely worse than the original problem of "Wow there's too many DLLs". Please don't make a mess of the Rx namespaces again, we have already been through this, Rx is already confusing enough, we know from the first time around that having "too many DLLs" caused immense user confusion and (for some reason) accusations of "bloat" because "there were too many references" - now we are proposing making it even more confusing, in order to solve a problem that is largely Annoying in nature Are we really worried about Disk Space here? In 2023? And that's worth nuking our entire ecosystem over? I really really really want people to reconsider the pain this will cause users of Rx, especially those who are using it via intermediate libraries. I know pulling in WPF sucks when you don't need it, but any kind of proposal like this will be so much worse. |
Beta Was this translation helpful? Give feedback.
-
Just as a matter of clarity, and since I didn't see this being discussed here: is there a possibility to improve the dependency resolution that ends up bringing the WPF/Winforms dlls, in such a way that doesn't happen to non-WPF/Winforms applications? |
Beta Was this translation helpful? Give feedback.
-
I want to add one other option we've thought about, and which might "work" but which we're reluctant to use because we believe it'd be completely unsupported. It would be technically possible for us to build a NuGet package for The benefit of this is that because from a NuGet packaging perspective, there's no apparent framework reference to the desktop frameworks, you don't get all the WPF/WinForms DLLs incorporated in your self-contained deployment just because you happened to add a reference to Rx.NET, but if you are in fact using those frameworks, it all works like you'd want. (And you get reasonably informative errors. If you try to use the relevant Rx.NET types in an app where you've not explicitly opted into using the relevant frameworks, you get a compiler error telling you that you're trying to use a, say, Windows Forms type without having a suitable assembly reference. And if you try to rely on the feature at runtime, you get a The massive downside is that this is a bit of a nasty hack and, as far as I know, completely unsupported. Our experience with the problems of trying to use UWP in an unsupported way (i.e., using it in conjunction with the modern .NET project system) is that hacking the build to make it do things that aren't supported causes grief. For that reason, I'd be quite reluctant to go down this path. |
Beta Was this translation helpful? Give feedback.
-
I get the reluctancy to break backwards compatibility, but sometimes you need to nuke an egg to make a pizza. WPF, and other UI frameworks are "on top of" the .net runtime and BCL, same goes for ASP, and Entity Framework. I would like to see a splitting, where there is a base that has zero dependencies outside the runtime, then anything that hooks into anything else, you make a new nuget. So its something like this for "tie-in" nugets: *.AspNetCore.Extensions |
Beta Was this translation helpful? Give feedback.
-
A disruptive change is worth it if it solves the current problem and meets future plans, it also helps to ship new features and bug fixes faster. |
Beta Was this translation helpful? Give feedback.
-
From my point of view as someone who works in building libraries at my current company I really would like the team to consider that a major update is a major update and that breaking changes are acceptable. In my oppinion it really isn't a great flow when we don't get thing done properly just cause of "we want to make it backwards compatible". However I undrestand the reasons behind it. Just stating my oppinion that I would be fine if we simply have to do steps "A, B, C" if I update. Angular dose it in such a way that they have a guide when updating to a major release where you have a list of checkmarks. I think the approach is pretty good, I never had a issue so far approching things this way. |
Beta Was this translation helpful? Give feedback.
-
As I've worked to write up the proposed approach and build a prototype implementation of it, I have made a slightly surprising discovering: #205 was the original fix for the plug-in conflict problem, but there was a regression in Rx 5.0! I've been building various test harnesses to verify that whatever packaging changes we ultimately make do not cause regressions for any of the various problems that earlier packaging changes were trying to fix, which is how I discovered that Rx 5.0 did in fact cause exactly that kind of regression. Here's the sequence that will illustrate the problem:
This came as something of a surprise because the plug-in conflict issue was apparently a big deal at the time. I was baffled when I realised this regression occurred over 3 years ago but apparently nobody noticed this time. I suspect this is because there are some things that are different this time. There's a straightforward workaround if a maintainer of an old plug-in hits this issue when upgrading to Rx 5.0: don't upgrade! They can stay on Rx 4.4.1, which does not have this problem. Most of the new work in Rx 5.0 was in support of newer frameworks, so legacy .NET Framework plug-ins almost certainly don't need to upgrade. Also, it only occurs when a plug-in uses a version of .NET that is older than 4.7.2. That shipped about 6 years ago, so in practice nobody's going to be building new plug-ins that have this issue. (So if you really need a later version of Rx you just need to make sure your plug-in targets The reason Rx 5.0 caused this regression is partly because in the Great Unification of Rx 4.0, the fix applied for Rx 3.1 (weird assembly version numbers) was abandoned. However, more by luck than judgement Rx 4.0 happened not to be susceptible to this bug. It only recurred when Rx 5.0 changed the TFMs. Specifically:
Here's the critical difference: with Rx 4.4.1, if you were targetting ANY version of .NET Framework, there was no way you could possibly load the But in Rx 5.0, that's not true. If you write a plug-in that targets I think the only way to avoid this would be for I think the answer to that might be to continue to offer |
Beta Was this translation helpful? Give feedback.
-
We now have two significant PRs. They are currently in draft because no final decisions have been made. We want to put them out for public review for now:
We expect everyone to hate the names of the new packages, so if you have any good ideas, please let us know. But bear in mind that some of the good names are already taken by the existing façade packages, and we have good reasons not to want to make those packages serve two unrelated purposes. A set of packages built in the proposed way is available from the feed in the Azure DevOps project for Rx: https://dev.azure.com/dotnet/Rx.NET/_artifacts/feed/RxNet (which should be accessible on https://pkgs.dev.azure.com/dotnet/Rx.NET/_packaging/RxNet/nuget/v3/index.json for NuGet) The packages all have version We aren't planning to push even preview builds of these out to NuGet until we've got consensus on the approach. |
Beta Was this translation helpful? Give feedback.
-
Anyone following this discussion might want to know that we are now planning to move forward. Please see #2177 for details. |
Beta Was this translation helpful? Give feedback.
-
We think that it might be necessary to change how Rx.NET is packaged in NuGet in a future version. We would much prefer not to do this, but the alternative appears to be a choice between leaving a major problem unfixed, and breaking backwards compatibility.
The Major Problem we Want to Fix
One of our main goals for the next version of Rx is to fix the problem where taking a dependency on
System.Reactive
can implicitly give you a dependency on all of WPF and Windows Forms, which can in turn cause deployments to become tens of megabytes larger than they need to be.See AvaloniaUI/Avalonia#9549 for an example of this kind of problem.
This is an unfortunate consequence of a decision made in Rx v4. The "great unification" in that release meant that you needed only a single NuGet reference to
System.Reactive
to get all of Rx. This was much simpler than previous releases. (More subtly it also happened to fix some problems that used to occur in plug-in systems.) At the time it was introduced, this did not cause the problem just described for the simple reason that WPF and Windows Forms were only available in .NET Framework. If you targetted .NET Framework you got a version of Rx.NET that included WPF and Windows Forms support. And if you targetted .NET Core, you got one that didn't.Where it all went wrong was when .NET Core 3.1 added WPF and Windows Forms support. The difficulty arose because not all .NET Core 3.1 systems have these frameworks. What was Rx to do? The solution it adopted was to look at your application's target framework moniker. If you had a windows-specific TFM targetting a sufficiently recent version (e.g.
net5.0-windows10.0.19401
) then it would give you the version of Rx with WPF and Windows Forms but otherwise it would not.But this makes no sense for UI applications using other UI frameworks (e.g. Avalonia). Those have no need for Rx's WPF and Windows Forms features.
It becomes particularly disastrous if you create a self-contained deployment, because this will now include all of the WPF and Windows Forms library component. This tends to make your application about 30MB bigger than it needed to be.
What needs to happen
It needs to be possible for applications targetting Windows-specific TFMs to be able to use Rx without ending up with a dependency on WPF and Windows Forms. Dependencies on Rx's WPF and/or Windows Forms functionality should be an opt-in feature.
Implications for packaging
There are essentially two options:
System.Reactive
so that it no longer automatically includes WPF and Windows Forms featuresSystem.Reactive
and introduce a new package to be used as the new "main" Rx packageIn either case, WPF and Windows Forms features would move out into new NuGet packages. (With 2, there is the option to have the deprecated
System.Reactive
comprise type forwarders to the new version, but it doesn't necessarily have to work that way.)What we'd like to do but can't
If we thought it was possible, we'd prefer option 1 above: to remove all UI-framework-specific features from
System.Reactive
. The API surface area that you get today (Rx v6) if you targetnet6.0
has absolutely no UI-specific functionality. Ideally we'd like that to be the case no matter what your target framework is.If we were able to rewrite history, we'd make it so that Rx v4 had worked this way, and that the WPF and Windows Forms (and UWP) features were always out in separate components. But obviously we can't do that. And unfortunately, that's why we don't believe we can do this in Rx 7.0 without breaking backwards compatibility in a serious way.
To understand why, consider an application
MyApp
that has two dependencies (which might be indirect, non-obvious dependencies) of this form:UiLibA
usesSystem.Reactive
v5.0. This is a UI library and it and depends onDispatcherScheduler
NonUiLibB
was also usingSystem.Reactive
v5.0 and it does not use any UI-framework-specific Rx featuresNow suppose
MyApp
needs to upgrade to a newer version ofNonUiLibB
. What if this newNonUiLibB
now depends on Rx 7.0, and uses a V7.0-specific feature. (E.g., imagine for the sake of argument that we add aRateLimit
operator to deal with the fact that almost nobody likesThrottle
)So we now have this situation:
UiLibA
will requireDispatcherScheduler
to be available throughSystem.Reactive
NonUiLibB
will requireObservable.RateLimit
to be available throughSystem.Reactive
So what will happen if we have modifed
System.Reactive
so that it no longer automatically includes WPF and Windows Forms features? Well eitherMyApp
will loadSystem.Reactive
v7, which won't haveDispatcherScheduler
(because that's a WPF feature), soUiLibA
is going to get aMissingMethodException
. Or the app will load an olderSystem.Reactive
in which caseObservable.RateLimit
won't be present, soNonUiLibB
will get aMissingMethodException
.You can't fix this by also adding a reference to, say,
System.Reactive.Wpf
(or whatever we call the package that containsDispatcherScheduler
) becauseUiLibA
won't be looking forDispatcherScheduler
in that library. It will expect it to be inSystem.Reactive
.This is a bad situation to put
MyApp
into. It can't work around this (except by sticking with the old version ofNonUiLibB
, or by dropping eitherUiLibA
orNonUiLibB
). The particularly insidious thing about this is that the problem is caused entirely in code not owned byMyApp
, so if Rx v7 went in this direction, applications would start finding themselves in this situation and not be able to do anything about it.You might think we can solve this by adding type forwarders in
System.Reactive
so that libraries such asUiLibA
that are looking forDispatcherScheduler
in that library will be told where it really is. The reason this doesn't work is that doing so requiresSystem.Reactive
to have a dependency on whatever library now contains the realDispatcherScheduler
. That means that a reference toSystem.Reactive
would implicitly mean you also get a reference to the WPF and Windows Forms libraries. So we're back at the problem we were trying to solve.What we think we will have to do (reluctantly)
We think the only way forward is to deprecate
System.Reactive
, and to introduce a new NuGet package that replaces it as the 'entry point' for Rx.We hate the idea of doing this. Rx.NET has already been through several confusing iterations of its packaging solution. If we knew of a way to solve AvaloniaUI/Avalonia#9549 while retaining
System.Reactive
as the main Rx package, without breaking existing applications that depend on Rx, we would prefer that. But so far, every proposal we know of to do this causes problems that might put existing applications into a state where they are stuck on old versions of libraries.UPDATE 2023/11/21: A possible alternative
Thanks to a question from @heronbpv, we did some further investigation and may have come up with an alternative. The full explanation is at #2038 (reply in thread) but in summary: it looks like the use of
DisableTransitiveFrameworkReferences
can work around this problem today (even in Rx 6.0.0). If that proves to be viable for affected parties, then that somewhat reduces the urgency around fixing this.If this does work out, we might be able to move more slowly towards the state in which UI-framework-specific functionality is fully separated out. We would likely still create separate NuGet packages for each UI framework, but leave the existing types in place in
System.Reactive
, marking them as[Obsolete]
. And then, several years from now, we could finally remove them completely.Should we fork Rx?
It has been suggested (e.g. see #2034 (comment) and the discussion following that comment) that there is an opportunity for a "clean break" here. There are at least two (incompatible) views on what that might mean, including:
System.Reactive
—remove the UI-framework-specific parts, and just declare that v7+ doesn't have these, leaving it for existing applications to try to fix up the consequences of thisAs I understand it, a motivation for this is to enable more innovation. I don't currently have a clear idea of what changes we might make in this new scheme that we currently cannot, although one possible area would be to separate out the parts of Rx that are not well suited to trimming. (There's some code relating to
IQbservable
where we couldn't find entirely satisfactory ways of annotating it for trimmability, and you tend to fall off a cliff in terms of binary size if you start using that.)I currently think that if we were going to do this, we'd need to build up a shopping list of the big changes we think we'd want to make to take advantage of this "clean break" because once we've pulled that trigger (if we do it) the opportunity to make significant changes is now in the past.
What do you think?
We've opened this discussion because we expect people to have opinions on this, and we hope that people might be able to design solutions to this that haven't occurred to us—perhaps there is a better way that we've just not seen yet. Please let us know what you think!
Beta Was this translation helpful? Give feedback.
All reactions