Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The active test run was aborted. Reason: Test host process crashed #451

Open
3 tasks
fanyang-mono opened this issue Jul 25, 2023 · 8 comments
Open
3 tasks

Comments

@fanyang-mono
Copy link
Member

fanyang-mono commented Jul 25, 2023

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=351450

Build leg reported

chrome-DebuggerTests.EvaluateOnCallFrameTests.WorkItemExecution

Pull Request

dotnet/runtime#89217

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The active test run was aborted. Reason: Test host process crashed",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

  • Feature changes/additions
  • Bug fixes
  • Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Report

Build Definition Step Name Console log Pull Request
903012 dotnet/roslyn Run Integration Tests Log
900073 dotnet/roslyn Run Integration Tests Log
899709 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76517
899492 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#75581
899447 dotnet/roslyn Run Integration Tests Log
899394 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76509
899197 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76509
899158 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76504
899075 dotnet/roslyn Run Integration Tests Log
898930 dotnet/roslyn Run Integration Tests Log
898763 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76490
898150 dotnet/roslyn Run Integration Tests Log
898015 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76479
897448 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76437
897339 dotnet/roslyn Run Integration Tests Log
897311 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76459
897138 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76466
895268 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76427
895169 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76424
895154 dotnet/roslyn Run Integration Tests Log
895032 dotnet/roslyn Run Integration Tests Log
894592 dotnet/roslyn Run Integration Tests Log
894433 dotnet/roslyn Run Integration Tests Log
894400 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76411
894342 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76413
894314 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76386
894260 dotnet/roslyn Run Integration Tests Log
894249 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76408
894156 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76351
894113 dotnet/roslyn Run Integration Tests Log
893996 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76358
893920 dotnet/roslyn Run Integration Tests Log
893668 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76235
893200 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76387
893046 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76384
892987 dotnet/roslyn Run Integration Tests Log
892776 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76357
892595 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76375
891923 dotnet/roslyn Run Integration Tests Log
891892 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76352
891503 dotnet/roslyn Run Integration Tests Log
891036 dotnet/roslyn Run Integration Tests Log
890631 dotnet/roslyn Run Integration Tests Log
890433 dotnet/roslyn Run Integration Tests Log
2597946 dotnet-interactive Test / Blame Log
889144 dotnet/roslyn Run Integration Tests Log
888914 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76321
888839 dotnet/roslyn Run Integration Tests Log
888829 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76318
888714 dotnet/roslyn Run Integration Tests Log
888602 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76309
887757 dotnet/roslyn Run Integration Tests Log
887746 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76291
887749 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76276
887732 dotnet/roslyn Run Integration Tests Log
887658 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76286
887405 dotnet/roslyn Run Integration Tests Log
887402 dotnet/roslyn Run Integration Tests Log
886534 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76269
886460 dotnet/roslyn Run Integration Tests Log
885670 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76254
885664 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76253
885460 dotnet/roslyn Run Integration Tests Log
885199 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76185
884735 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76233
884729 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#69804
884674 dotnet/roslyn Run Integration Tests Log
884480 dotnet/roslyn Run Integration Tests Log
884436 dotnet/roslyn Run Integration Tests Log
884349 dotnet/roslyn Run Integration Tests Log
884247 dotnet/roslyn Run Integration Tests Log
884156 dotnet/roslyn Run Integration Tests Log
884153 dotnet/roslyn Run Integration Tests Log
884066 dotnet/roslyn Run Integration Tests Log
883939 dotnet/roslyn Run Integration Tests Log
883917 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76149
883921 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76150
883855 dotnet/roslyn Run Integration Tests Log
883730 dotnet/roslyn Run Integration Tests Log
883251 dotnet/roslyn Run Integration Tests Log dotnet/roslyn#76219
Build Definition Test Pull Request
904735 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45019
904727 dotnet/sdk Microsoft.NET.Build.Tests.dll.7.WorkItemExecution dotnet/sdk#45665
904710 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45664
904684 dotnet/sdk Microsoft.NET.Build.Tests.dll.11.WorkItemExecution dotnet/sdk#42416
904641 dotnet/sdk Microsoft.NET.Publish.Tests.dll.11.WorkItemExecution dotnet/sdk#45618
904496 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45663
904254 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45618
904201 dotnet/sdk dotnet-watch.Tests.dll.3.WorkItemExecution dotnet/sdk#45620
903341 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45614
904167 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45636
904101 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45629
904076 dotnet/sdk Microsoft.NET.Publish.Tests.dll.8.WorkItemExecution dotnet/sdk#45651
903944 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#42416
903935 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45019
903860 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution dotnet/sdk#45650
903781 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45647
903749 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45629
903748 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45630
903523 dotnet/sdk Microsoft.NET.Publish.Tests.dll.8.WorkItemExecution dotnet/sdk#45629
903407 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.15.WorkItemExecution dotnet/sdk#45642
903382 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45637
903395 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.15.WorkItemExecution dotnet/sdk#45641
903389 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45633
903294 dotnet/sdk Microsoft.NET.Publish.Tests.dll.11.WorkItemExecution dotnet/sdk#45639
903240 dotnet/sdk Microsoft.NET.Build.Tests.dll.10.WorkItemExecution dotnet/sdk#45637
902850 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45160
903020 dotnet/aspnetcore Microsoft.AspNetCore.Mvc.Core.Test--net10.0.WorkItemExecution dotnet/aspnetcore#59543
902880 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45630
902686 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.15.WorkItemExecution dotnet/sdk#45630
902680 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.15.WorkItemExecution dotnet/sdk#45627
902648 dotnet/sdk Microsoft.NET.Publish.Tests.dll.11.WorkItemExecution dotnet/sdk#42416
902506 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45622
902436 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45623
902478 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45567
902439 dotnet/sdk Microsoft.NET.Publish.Tests.dll.8.WorkItemExecution dotnet/sdk#45625
902381 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45614
902416 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45622
899898 dotnet/sdk Microsoft.NET.Publish.Tests.dll.10.WorkItemExecution dotnet/sdk#45567
902340 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45552
902284 dotnet/sdk Microsoft.NET.Sdk.BlazorWebAssembly.Tests.dll.4.WorkItemExecution dotnet/sdk#45419
902134 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45618
901820 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45614
901984 dotnet/sdk Microsoft.NET.Publish.Tests.dll.10.WorkItemExecution dotnet/sdk#45354
900775 dotnet/sdk Microsoft.NET.Publish.Tests.dll.8.WorkItemExecution dotnet/sdk#45522
901888 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45615
901732 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45612
901455 dotnet/sdk Microsoft.NET.Sdk.BlazorWebAssembly.Tests.dll.4.WorkItemExecution dotnet/sdk#45160
901419 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45019
901401 dotnet/sdk Microsoft.NET.Sdk.BlazorWebAssembly.Tests.dll.4.WorkItemExecution dotnet/sdk#45605
901349 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.19.WorkItemExecution dotnet/sdk#45589
901347 dotnet/aspnetcore Microsoft.AspNetCore.Mvc.FunctionalTests--net10.0.WorkItemExecution
901021 dotnet/sdk Microsoft.NET.Publish.Tests.dll.7.WorkItemExecution dotnet/sdk#45598
901290 dotnet/aspnetcore Templates.Mvc.Tests--net8.0.WorkItemExecution
901153 dotnet/sdk Microsoft.DotNet.Watch.UnitTests.DotNetWatcherTests.Microsoft.DotNet.Watch.UnitTests.DotNetWatcherTests.RunsWithDotnetLaunchProfileEnvVariableWhenExplicitlySpecified dotnet/sdk#45369
901150 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45603
901199 dotnet/aspnetcore Microsoft.AspNetCore.Mvc.FunctionalTests--net10.0.WorkItemExecution dotnet/aspnetcore#59592
901208 dotnet/sdk Microsoft.NET.Publish.Tests.dll.7.WorkItemExecution dotnet/sdk#45605
901221 dotnet/sdk Microsoft.NET.Publish.Tests.dll.7.WorkItemExecution dotnet/sdk#45609
901149 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45071
2605950 dotnet-aspnetcore Microsoft.AspNetCore.Mvc.FunctionalTests--net10.0.WorkItemExecution
900813 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution dotnet/sdk#45594
898008 dotnet/sdk Microsoft.NET.Publish.Tests.dll.9.WorkItemExecution dotnet/sdk#45494
901050 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45570
901010 dotnet/sdk dotnet-watch.Tests.dll.3.WorkItemExecution dotnet/sdk#45593
900953 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45592
900940 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45584
900095 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45312
900702 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45591
900604 dotnet/aspnetcore Microsoft.AspNetCore.App.Analyzers.Test--net10.0.WorkItemExecution dotnet/aspnetcore#59585
900595 dotnet/sdk Microsoft.NET.Build.Tests.dll.24.WorkItemExecution dotnet/sdk#45589
900261 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution dotnet/sdk#45574
900415 dotnet/aspnetcore Templates.Mvc.Tests--net8.0.WorkItemExecution
900310 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45580
900341 dotnet/sdk Microsoft.NET.Publish.Tests.dll.11.WorkItemExecution dotnet/sdk#45522
900303 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45578
900262 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45573
900187 dotnet/sdk Microsoft.NET.Publish.Tests.dll.10.WorkItemExecution dotnet/sdk#44461
900139 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.11.WorkItemExecution dotnet/sdk#45572
900006 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.15.WorkItemExecution dotnet/sdk#45563
899970 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45525
899860 dotnet/sdk Microsoft.NET.Sdk.BlazorWebAssembly.Tests.dll.1.WorkItemExecution dotnet/sdk#45566
899815 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45565
899786 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.15.WorkItemExecution dotnet/sdk#45563
899811 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45552
899513 dotnet/aspnetcore Templates.Mvc.Tests--net8.0.WorkItemExecution
899521 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45554
897207 dotnet/sdk Microsoft.NET.Publish.Tests.dll.5.WorkItemExecution dotnet/sdk#45485
898646 dotnet/sdk Microsoft.NET.Publish.Tests.dll.10.WorkItemExecution dotnet/sdk#45534
899202 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45463
899189 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45552
2604328 dotnet-roslyn Microsoft.CodeAnalysis.CSharp.CodeStyle.UnitTests_10.WorkItemExecution
899042 dotnet/sdk Microsoft.NET.Sdk.BlazorWebAssembly.Tests.dll.4.WorkItemExecution dotnet/sdk#45476
898974 dotnet/sdk Microsoft.NET.Sdk.BlazorWebAssembly.Tests.dll.4.WorkItemExecution dotnet/sdk#45549
899039 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.15.WorkItemExecution dotnet/sdk#45354
898895 dotnet/sdk Microsoft.NET.Build.Tests.dll.23.WorkItemExecution dotnet/sdk#45353
898844 dotnet/sdk dotnet-watch.Tests.dll.1.WorkItemExecution dotnet/sdk#45545
898783 dotnet/roslyn Microsoft.CodeAnalysis.VisualBasic.Features.UnitTests_Microsoft.CodeAnalysis.VisualBasic.ExpressionEvaluator.ResultProvider.UnitTests_Microsoft.CodeAn..._86.WorkItemExecution dotnet/roslyn#76493
898560 dotnet/sdk dotnet-watch.Tests.dll.2.WorkItemExecution dotnet/sdk#45397
898169 dotnet/roslyn Microsoft.CodeAnalysis.CSharp.EndToEnd.UnitTests_0.WorkItemExecution dotnet/roslyn#75581
898075 dotnet/sdk Microsoft.NET.Sdk.Razor.Tests.dll.10.WorkItemExecution dotnet/sdk#45518
Displaying 100 of 272 results

Summary

24-Hour Hit Count 7-Day Hit Count 1-Month Count
6 31 352

Known issue validation

Build: 🔎
Result validation: ⚠️ Validation could not be done without an Azure DevOps build URL on the issue. Please add it to the "Build: 🔎" line.
Validation performed at: 2/8/2024 9:24:30 PM UTC

@andriipatsula
Copy link
Member

Hello @fanyang-mono, could you please update the "ErrorMessage" : "" by following the step by step documentation on how to create a known issue

@fanyang-mono
Copy link
Member Author

Updated.

@pavelsavara
Copy link
Member

pavelsavara commented Nov 7, 2023

This is duplicate of dotnet/runtime#89409, except here it's not only wasm debugger

@dougbu
Copy link
Member

dougbu commented Nov 23, 2023

@pavelsavara and @fanyang-mono, should this issue be closed in favour of dotnet/runtime#94560❔ if not, the error message isn't sufficiently precise to understand what (if anything) we (DNCEng) could do about it.

@pavelsavara
Copy link
Member

@pavelsavara and @fanyang-mono, should this issue be closed in favour of

You mean dotnet/runtime#89409

I think we should keep this one, because this one has some hits which are not just wasm debugger related.
Here we see other repo failures, on roslyn and aspnetcore CI.

if not, the error message isn't sufficiently precise to understand what (if anything) we (DNCEng) could do about it.

I have not investigated it myself yet. But got questions which could help to narrow it down

  • are there crash dump files on the machine ?
  • are there some hints on what's going on in the log files before crash ?
  • is this OOM or stack overflow in disguise ?
  • did we run out of disk space or number of inodes ?
  • hit some limit in docker container ?
  • is this always the same OS, version, image ?
  • could this be hardware issue ?

@fanyang-mono
Copy link
Member Author

This issue was originally created to track issues happened to wasm debugger tests (belonging to dotnet/runtime repo). However, the error message was generic enough to include issues from other repos, such as dotnet/roslyn and dotnet/aspnetcore. We need insights from owners from dotnet/roslyn and dotnet/aspnetcore to confirm that they are aware of these issue. And if they are okay to close this issue as well. We could close then.

@mitchdenny
Copy link
Member

We just got an instance of this in the dotnet/aspire repo. It wasn't a WASM related failure for us. Just looks like the test process crashed and took a few tests with it.

@nagilson
Copy link
Member

nagilson commented Dec 3, 2024

@marcpopMSFT was working on getting the test infrastructure to generate crash dumps when it times out to help with investigation. I think it was generating the dumps but not yet correctly uploading them to the artifacts.

We are waiting to hear from the infra team on an issue with this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants