Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix msbuild property order, to make IsTestingPlatformApplication true, and make it global kill switch #4307

Open
nohwnd opened this issue Dec 10, 2024 · 1 comment

Comments

@nohwnd
Copy link
Member

nohwnd commented Dec 10, 2024

Describe the bug

IsTestingPlatformApplication is re-assigned to false, making it difficult to use. And not acting as global opt-in.

Image

Steps To Reproduce

The IsTestingPlatformApplication MSBuild property seems to be broken. It works for Mstest because all the props that IsTestingPlatformApplication splits into are being set again explicitly by mstest target.

This property not working is problematic for additional frameworks that wish to opt in (like NUnit) which then have to know too much about the internals. and cannot simply add single property to opt into the new testing platform. Instead the have to re-define every switch we provide.

The cause is these props being set in .props, so they are included and evaluated before the user has a chance to set them.

Image

So IsTestingPlatformApplication will be set to false, and all the following props that use it as condition, e.g. this
are not included.

Image

We don't see this as problematic in mstest because we do all the same work again but conditioning it to EnableMSTestRunner, and xunit and nunit do the same, because we fixed them that way.

Image

We cannot move the IsTestingPlatformApplication property to targets because it would be evaluated too late:

Adapter.props
MSBuild.props
csproj
MSBuild.targets
Adapter.targets --> So too late because MSBuild.targets was already evaluated

This is because of package deps. Adapter -> MSBuild so Adapter is resolved first/last

so that means that we cannot use the IsTestingPlatformApplication property as a condition for property groups. And so we cannot use it to define the default values for the dependent props.

so there are 2 options.

  1. require the testing frameworks to specify the properties in targets, and set all of them to true or false, this puts the burden of knowing all the knobs on the framework

  2. ask testing frameworks to move their property to targets, and only require to set IsTestingPlatformApplication to true, which would enable all features for them, and then they can opt out from e.g. GenerateTestingPlatformEntryPoint by setting it to false.

And we would need to remove the property groups conditioned from our targets, they do nothing and are confusing. and change these conditions

Image

that way the IsTestingPlatformApplication will remain the single global knob, and we can add more options that the frameworks can opt out from, but that they will have enabled by default

option 3) is like option 2, but we set all the values to true in msbuild.props, and add the istestingplatform to all conditions that use the opt-out knobs. so IsTestingPlatformApplication remains the global kill switch. This has "disadvantage of always seeing property re-assignment in the binlog, when run is not enabling TP or some feature

Expected behavior

IsTestingPlatformApplication is a single switch to opt into testing platform, and it will remain true when enabled.

@nohwnd
Copy link
Member Author

nohwnd commented Dec 10, 2024

Option 2 is the best fix here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant