The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
2964251	dotnet-sdk	🔒 Start Network Isolation	Log
2964226	dotnet-sdk	Run AoT Tests	Log	#60658
2964227	dotnet-sdk	Run dotnet-format on dotnet/aspnetcore repo folder	Log	#60659
1403548	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#127603
2964225	dotnet-sdk	Build	Log	#60659
1403713	dotnet/arcade	Monitor Helix Jobs	Log	dotnet/arcade#16740
2964194	dotnet-sdk	SDL Binary Analysis Tools Pre-Checks + AntiMalware MDE Linux Scanner (Binary) (1ES PT)	Log	#60658
2964193	dotnet-sdk	Run TestBuild Tests	Log	#60657
2964196	dotnet-sdk	Run Tests in Helix	Log	#60655
2964195	dotnet-sdk	Run Tests in Helix	Log	#60654
1403413	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#127464
1403385	dotnet/runtime	Zip Build Assets	Log	dotnet/runtime#127269
1403523	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127555
1403370	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#127560
1403408	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127238
1403447	dotnet/maui	Download PackageArtifacts	Log	dotnet/maui#35223
1403482	dotnet/runtime	Send to Helix	Log
1403217	dotnet/maui	Controls Shell	Log	dotnet/maui#31789
1403472	dotnet/maui	Controls WebView	Log	dotnet/maui#31159
2964136	dotnet-sdk	Run Tests in Helix	Log	#60654
2964137	dotnet-sdk	Run Tests in Helix	Log	#60655
1403371	dotnet/maui	Controls Shell	Log	dotnet/maui#35234
2964127	dotnet-roslyn	Test	Log
1403377	dotnet/maui	Controls CollectionView	Log	dotnet/maui#35225
1399748	dotnet/maui	Controls WebView	Log	dotnet/maui#34573
1402120	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#54143
2964098	dotnet-runtime	Send to Helix	Log	#60646
1403279	dotnet/runtime	Publish	Log	dotnet/runtime#127599
1403306	dotnet/runtime	Publish Build Assets	Log
1403171	dotnet/maui	Controls Shell	Log
1403265	dotnet/runtime	Publish Build Assets	Log
1402968	dotnet/runtime	Send tests to Helix (Unix)	Log
2963977	dotnet-aspnetcore	Publish Build Assets	Log
2964055	dotnet-aspnetcore	🛡 Guardian: TSAUpload	Log	#60637
2964078	dotnet-performance	Send job to Helix (Windows)	Log
1402970	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#53847
2964050	dotnet-aspnetcore	Run build.cmd helix target	Log	#60636
2963991	dotnet-dotnet	Build	Log
1403257	dotnet/maui	Provision Android SDK - Common Packages	Log	dotnet/maui#31159
1403268	dotnet/maui	Controls CollectionView	Log
1403200	dotnet/runtime	Run Native AOT Test App Tests	Log
1403254	dotnet/maui	Restore .NET Tools	Log	dotnet/maui#35215
1403164	dotnet/runtime	Zip Build Assets	Log	dotnet/runtime#127597
1403256	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#54154
1403079	dotnet/runtime	Send to Helix	Log
1403204	dotnet/maui	Run Integration Tests - Build windows	Log	dotnet/maui#31789
1403185	dotnet/roslyn	Test	Log
1403181	dotnet/maui	Provision Android SDK - Common Packages	Log	dotnet/maui#34957
2963955	dotnet-roslyn	Test	Log
1403154	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#127595
1403076	dotnet/runtime	Send to Helix	Log
1403149	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#127096
1403139	dotnet/runtime	Send tests to Helix (Unix)	Log
1403188	dotnet/roslyn	Build with analyzers	Log	dotnet/roslyn#83491
1402834	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#127587
1402922	dotnet/maui	Controls Shell	Log
1402396	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#54149
1402969	dotnet/maui	Controls WebView	Log
1403101	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#125795
1403114	dotnet/roslyn	Test	Log
1402971	dotnet/runtime	Send tests to Helix (Unix)	Log
1403122	dotnet/runtime	Send job to Helix (Unix)	Log
1402980	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#126277
1403104	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#127591
2963885	dotnet-aspnetcore	Run build.cmd helix target	Log	#60605
2963887	dotnet-sdk	Run TestBuild Tests	Log	#60627
2963888	dotnet-sdk	Run AoT Tests	Log	#60628
2963877	dotnet-aspnetcore	Run build.cmd helix target	Log	#60622
1403089	dotnet/runtime	Send to Helix	Log	dotnet/runtime#126782
1402774	dotnet/maui	Controls CollectionView	Log
1403081	dotnet/sdk	🟣 Run AoT Tests	Log	dotnet/sdk#54052
2963899	dotnet-docker-tools	🟪 Clean public/dotnet/* (pruneEol > 15d)	Log
1402924	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#127548
2963879	dotnet-roslyn	Test	Log
2963876	dotnet-sdk	Run Tests in Helix	Log	#60619
2963875	dotnet-sdk	Run Tests in Helix	Log	#60620
1402862	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#127589
1402853	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#127588
1402794	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#54116
1402840	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#127538
1403056	dotnet/roslyn	Test	Log
1402846	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#127574
2963859	dotnet-roslyn	Test	Log
1402381	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#127557
1403099	dotnet/roslyn	Test	Log
1402808	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#127585
1402942	dotnet/runtime	Publish Build Assets	Log	dotnet/runtime#127583
1403106	dotnet/aspnetcore	Run build.cmd helix target	Log
1403066	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127419
1402805	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#127545
1402754	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#127561
1402871	dotnet/roslyn	Test	Log
1403061	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127590
1402761	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#126108
1402825	dotnet/aspnetcore	Run build.cmd helix target	Log	dotnet/aspnetcore#66532
1402973	dotnet/roslyn	Test	Log
1402855	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#127588
2963830	dotnet-dotnet-monitor	Build	Log
2963681	dotnet-maui	Controls WebView	Log
1402547	dotnet/maui	Controls CollectionView	Log
Displaying 100 of 3019 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
282	831	3019

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez