[Apple] TvOS-arm64 failures on Serialization ExporterTypesTests

[matouskozak]

Build Information

Build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=721722
Build error leg or test failing: System.Runtime.Serialization.Xml.XsdDataContractExporterTests.ExporterTypesTests.TypesTest
Affected CI: tvos_arm64_release_allsubsets_mono (runtime-extra-platforms)
Range of commits: 42ade70...5a0eb6e
Stack trace example:

[05:51:23.5637080] 2024-06-28 02:51:23.561 System.Runtime.Serialization.Xml.Tests[45703:97828148] 1) 	[FAIL] System.Runtime.Serialization.Xml.XsdDataContractExporterTests.ExporterTypesTests.TypesTest   Test name: System.Runtime.Serialization.Xml.XsdDataContractExporterTests.ExporterTypesTests.TypesTest
[05:51:23.5637570] 2024-06-28 02:51:23.561 System.Runtime.Serialization.Xml.Tests[45703:97828148]    Assembly:  [System.Runtime.Serialization.Xml.Tests, Version=9.0.0.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51]
[05:51:23.5637710]    Exception messages: Assert.Contains() Failure: Sub-string not found
[05:51:23.5637770] String:    "<xs:complexType name="ExporterTypesTests."···
[05:51:23.5637870] 2024-06-28 02:51:23.561 System.Runtime.Serialization.Xml.Tests[45703:97828148] Not found: "<xs:complexType name="ExporterTypesTests."···   Exception stack traces:    at System.Runtime.Serialization.Xml.XsdDataContractExporterTests.SchemaUtils.OrderedContains(String expected, String& actual)
[05:51:23.5638070] 2024-06-28 02:51:23.561 System.Runtime.Serialization.Xml.Tests[45703:97828148]    at System.Runtime.Serialization.Xml.XsdDataContractExporterTests.ExporterTypesTests.TypesTest()
[05:51:23.5638170]    at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Method(Object obj, IntPtr* args)
[05:51:23.5638230]    at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
[05:51:23.5639960] 2024-06-28 02:51:23.561 System.Runtime.Serialization.Xml.Tests[45703:97828148]    Execution time: 0.1320086 **** Output start ****<?xml version="1.0" encoding="utf-16"?><xs:schema xmlns:tns="http://schemas.datacontract.org/2004/07/SerializableTypes.XsdDataContractExporterTests" elementFormDefault="qualified" targetNamespace="http://schemas.datacontract.org/2004/07/SerializableTypes.XsdDataContractExporterTests" xmlns:xs="http://www.w3.org/2001/XMLSchema">  <xs:complexType name="DataContractTypes.Person1">    <xs:sequence>      <xs:element minOccurs="0" name="age" type="xs:int" />      <xs:element minOccurs="0" name="name" nillable="true" type="xs:string" />      <xs:element minOccurs="0" name="salary" type="xs:float" />    </xs:sequence>  </xs:complexType>  <xs:element name="DataContractTypes.Person1" nillable="true" type="tns:DataContractTypes.Person1" /></xs:schema><?xml version="1.0" encoding="utf-16"?><xs:schema xmlns:tns="http://schemas.datacontract.org/2004/07/System.Runtime.Serialization.Xml.XsdDataContractExporterTests" elementFormDefault="qualified" targetNamespace="http://schemas.datacontract.org/2004/07/System.Runtime.Serialization.Xml.XsdDataContractExporterTests" xmlns:xs="http://www.w3.org/2001/XMLSchema">  <xs:import namespace="http://schemas.microsoft.com/2003/10/Serialization/Arrays" />  <xs:complexType name="ExporterTypesTests.Group">    <xs:sequence>      <xs:element name="People" nillable="true" type="tns:ArrayOfExporterTypesTests.Person" />    </xs:sequence>  </xs:complexType>  <xs:element name="ExporterTypesTests.Group" nillable="true" type="tns:ExporterTypesTests.Group" />  <xs:complexType name="ArrayOfExporterTypesTests.Person">    <xs:sequence>      <xs:element minOccurs="0" maxOccurs="unbounded" name="ExporterTypesTests.Person" nillable="true" type="tns:ExporterTypesTests.Person" />    </xs:sequence>  </xs:complexType>  <xs:element name="ArrayOfExporterTypesTests.Person" nillable="true" type="tns:ArrayOfExporterTypesTests.Person" />  <xs:complexType name="ExporterTypesTests.Person">    <xs:sequence>      <xs:element name="name" nillable="true" type="xs:string" />    </xs:sequence>  </xs:complexType>  <xs:element name="ExporterTypesTests.Person" nillable="true" type="tns:ExporterTypesTests.Person" />  <xs:complexType name="ExporterTypesTests.Employee">    <xs:complexContent mixed="false">      <xs:extension base="tns:ExporterTypesTests.Person">        <xs:sequence>          <xs:element name="empId" type="xs:int" />        </xs:sequence>      </xs:extension>    </xs:complexContent>  </xs:complexType>  <xs:element name="ExporterTypesTests.Employee" nillable="true" type="tns:ExporterTypesTests.Employee" />  <xs:complexType name="ExporterTypesTests.Engineer">    <xs:complexContent mixed="false">      <xs:extension base="tns:ExporterTypesTests.Employee">        <xs:sequence />      </xs:extension>    </xs:complexContent>  </xs:complexType>  <xs:element name="ExporterTypesTests.Engineer" nillable="true" type="tns:ExporterTypesTests.Engineer" />  <xs:complexType name="ExporterTypesTests.Admin">    <xs:complexContent mixed="false">      <xs:extension base="tns:ExporterTypesTests.Employee">        <xs:sequence />      </xs:extension>    </xs:complexContent>  </xs:complexType>  <xs:element name="ExporterTypesTests.Admin" nillable="true" type="tns:ExporterTypesTests.Admin" />  <xs:complexType name="ExporterTypesTests.Architect">    <xs:complexContent mixed="false">      <xs:extension base="tns:ExporterTypesTests.Employee">        <xs:sequence />      </xs:extension>    </xs:complexContent>  </xs:complexType>  <xs:element name="ExporterTypesTests.Architect" nillable="true" type="tns:ExporterTypesTests.Architect" />  <xs:complexType name="ExporterTypesTests.NoDataContract">    <xs:sequence />  </xs:complexType>  <xs:element name="ExporterTypesTests.NoDataContract" nillable="true" type="tns:ExporterTypesTests.NoDataContract" />  <xs:complexType name="ExporterTypesTests.DataContractWithValidMember">    <xs:sequence>      <xs:element minOccurs="0" name="member" nillable="true" type="tns:ExporterTypesTests.NoDataContract" />    </xs:sequence>  </xs:complexType>  <xs:element name="ExporterTy
[05:51:23.5641760] 2024-06-28 02:51:23.562 System.Runtime.Serialization.Xml.Tests[45703:97828148] pesTests.DataContractWithValidMember" nillable="true" type="tns:ExporterTypesTests.DataContractWithValidMember" />  <xs:complexType name="ExporterTypesTests.PersonInfo">    <xs:sequence>      <xs:element minOccurs="0" name="PersonArrayList" nillable="true" xmlns:q1="http://schemas.microsoft.com/2003/10/Serialization/Arrays" type="q1:ArrayOfanyType" />      <xs:element minOccurs="0" name="Persons" nillable="true" type="tns:ArrayOfExporterTypesTests.Person" />      <xs:element minOccurs="0" name="innerPersonInfo" nillable="true" type="tns:ExporterTypesTests.InnerPersonCollection" />    </xs:sequence>  </xs:complexType>  <xs:element name="ExporterTypesTests.PersonInfo" nillable="true" type="tns:ExporterTypesTests.PersonInfo" />  <xs:complexType name="ExporterTypesTests.InnerPersonCollection">    <xs:sequence>      <xs:element minOccurs="0" name="Friends" nillable="true" type="tns:ArrayOfExporterTypesTests.Person" />      <xs:element minOccurs="0" name="PotentialExpenditures" nillable="true" xmlns:q2="http://schemas.microsoft.com/2003/10/Serialization/Arrays" type="q2:ArrayOfdouble" />      <xs:element minOccurs="0" name="PotentialSalaries" nillable="true" xmlns:q3="http://schemas.microsoft.com/2003/10/Serialization/Arrays" type="q3:ArrayOfdouble" />    </xs:sequence>  </xs:complexType>  <xs:element name="ExporterTypesTests.InnerPersonCollection" nillable="true" type="tns:ExporterTypesTests.InnerPersonCollection" /></xs:schema><?xml version="1.0" encoding="utf-16"?><xs:schema xmlns:tns="http://schemas.microsoft.com/2003/10/Serialization/Arrays" elementFormDefault="qualified" targetNamespace="http://schemas.microsoft.com/2003/10/Serialization/Arrays" xmlns:xs="http://www.w3.org/2001/XMLSchema">  <xs:complexType name="ArrayOfanyType">    <xs:sequence>      <xs:element minOccurs="0" maxOccurs="unbounded" name="anyType" nillable="true" type="xs:anyType" />    </xs:sequence>  </xs:complexType>  <xs:element name="ArrayOfanyType" nillable="true" type="tns:ArrayOfanyType" />  <xs:complexType name="ArrayOfdouble">    <xs:sequence>      <xs:element minOccurs="0" maxOccurs="unbounded" name="double" type="xs:double" />    </xs:sequence>  </xs:complexType>  <xs:element name="ArrayOfdouble" nillable="true" type="tns:ArrayOfdouble" /></xs:schema><?xml version="1.0" encoding="utf-16"?><xs:schema targetNamespace="http://www.w3.org/2001/XMLSchema" xmlns:xs="http://www.w3.org/2001/XMLSchema">  <xs:element name="schema">    <xs:complexType />  </xs:element></xs:schema>----------------- 5, 36, 18 **** Output end ****

Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "Not found: \"<xs:complexType name=\"ExporterTypesTests.\"",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=721722
Error message validated: [Not found: "<xs:complexType name="ExporterTypesTests."]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/28/2024 3:30:00 PM UTC

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

[dotnet-policy-service]

Tagging subscribers to 'os-tvos': @vitek-karas, @kotlarmilos, @ivanpovazan, @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

[matouskozak]

Could the failures be related to #103740 @sbomer ?

[sbomer]

It doesn't look related to me because I don't see that test calling Type.GetType or any methods annotated with DynamicallyAccessedMembersAttribute, but it's possible I'm missing something. Is it only failing when trimming?

[matouskozak]

It doesn't look related to me because I don't see that test calling Type.GetType or any methods annotated with DynamicallyAccessedMembersAttribute, but it's possible I'm missing something. Is it only failing when trimming?

Thank you for the insight. Your PR was the closest from the failing tests I could find, do you know if something from this range could be more related 42ade70...5a0eb6e ?

It is failing with disabled trimming (disabled in the suspected range of commits #104028). Do you think that disabling trimming could cause the failure?

Edit. it seems possible, there is an open PR to re-enable trimming (#104097) on Apple mobile and the tests doesn't fail on the PR.

[sbomer]

I don't understand how disabling trimming would cause the failure since usually trimming will if anything result in failures due to missing types, but it's possible. I see that this test is calling XsdDataContractExporter.Export, which is annotated with RequiresDynamicCode and RequiresUnreferencedCode - so trimming can't make any guarantees about the behavior of the test in general.

It's also possibly related to #103594 that set DynamicCodeSupport=true, (but wasn't supposed to apply to AOT scenarios). #104097 is setting it back to false in AOT test targets.

[vitek-karas]

Moved to 9 since this is a recent regression, so we should at least understand what's going on.

[matouskozak]

Resolved in #104097