Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]: UseDotNet task installs x86 build on Windows arm64 #20300

Open
4 of 7 tasks
azchohfi opened this issue Aug 14, 2024 · 6 comments · May be fixed by #20695
Open
4 of 7 tasks

[BUG]: UseDotNet task installs x86 build on Windows arm64 #20300

azchohfi opened this issue Aug 14, 2024 · 6 comments · May be fixed by #20695
Labels
Area: ABTT Akvelon Build Tasks Team area of work bug

Comments

@azchohfi
Copy link
Contributor

New issue checklist

Task name

UseDotNet

Task version

2

Issue Description

My host is an Arm64 machine, but it trying to install the x86 version of the dotnet sdk.

Environment type (Please select at least one enviroment where you face this issue)

  • Self-Hosted
  • Microsoft Hosted
  • VMSS Pool
  • Container

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

No response

Operation system

Windows 2022 ARM64

Relevant log output

Tool to install: .NET Core sdk version 8.0.x.
Found version 8.0.400 in channel 8.0 for user specified version spec: 8.0.x
Version 8.0.400 was not found in cache.
Getting URL to download .NET Core sdk version: 8.0.400.
Detecting OS platform to find correct download package for the OS.
C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoLogo -Sta -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command "& 'C:\a\_work\_tasks\UseDotNet_b0ce7256-7898-45d3-9cb5-176b752bfea6\2.243.1\externals\get-os-platform.ps1'"
Primary:win-x86
Detected platform (Primary): win-x86
Downloading: https://download.visualstudio.microsoft.com/download/pr/4bc6c939-8cb1-45c4-afde-a1b0715eec7d/c3835988860504c59187049089ca2466/dotnet-sdk-8.0.400-win-x86.zip
Extracting downloaded package C:\a\_work\_temp\1b29a698-0d9b-40f8-837e-492b74a90b1f.
Extracting archive
C:\Windows\system32\chcp.com 65001
Active code page: 65001

C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -NoLogo -Sta -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command "$ErrorActionPreference = 'Stop' ; try { Add-Type -AssemblyName System.IO.Compression.FileSystem } catch { } ; [System.IO.Compression.ZipFile]::ExtractToDirectory('C:\a\_work\_temp\1b29a698-0d9b-40f8-837e-492b74a90b1f', 'C:\a\_work\_temp\99cb')"

Successfully installed .NET Core sdk version 8.0.400.
Creating global tool path and pre-pending to PATH.

Full task logs with system.debug enabled

 [REPLACE THIS WITH YOUR INFORMATION] 

Repro steps

- task: UseDotNet@2
  displayName: Setup .NET 8
  inputs:
    version: 8.0.x
@azchohfi
Copy link
Contributor Author

This seems to be missing arm64:

{ ($_ -eq "amd64") -or ($_ -eq "x64") } { return "x64" }
{ $_ -eq "x86" } { return "x86" }

But weirdly it went the x86 code path.

@v-schhabra v-schhabra added Area: ABTT Akvelon Build Tasks Team area of work and removed Area: Release labels Aug 16, 2024
@aleksandrlevochkin
Copy link
Contributor

Hi @azchohfi thank you for reporting this issue. As I see, the script determines the architecture by checking the env variable PROCESSOR_ARCHITECTURE. It seems like this variable does not necessarily return the actual architecture of the processor but of the process with this env variable.

Is there any chance that you're using the x86 version of the self-hosted agent?

@DHowett
Copy link
Member

DHowett commented Sep 23, 2024

Is there any chance that you're using the x86 version of the self-hosted agent?

Yes. This is going to be true of all Microsoft-internal 1ES-managed pools, as well.
 

Windows support runs the x86 Azure DevOps agent in emulation mode, so requires a Cobalt or later OS release.
from eng.ms

Fixing this will require the task to be more intelligent than it currently is.

@DHowett
Copy link
Member

DHowett commented Sep 23, 2024

For example, it is possible to determine the actual architecture using WMI.

$Arch = if ((Get-WmiObject -Class Win32_Processor).Architecture -eq 12) { "arm64" } else { "x64" }

@DHowett
Copy link
Member

DHowett commented Sep 24, 2024

And final FYI:

dotnet install-scripts has long since been updated to account for this

function Get-Machine-Architecture() {
    Say-Invocation $MyInvocation

    # On PS x86, PROCESSOR_ARCHITECTURE reports x86 even on x64 systems.
    # To get the correct architecture, we need to use PROCESSOR_ARCHITEW6432.
    # PS x64 doesn't define this, so we fall back to PROCESSOR_ARCHITECTURE.
    # Possible values: amd64, x64, x86, arm64, arm
    if( $ENV:PROCESSOR_ARCHITEW6432 -ne $null ) {
        return $ENV:PROCESSOR_ARCHITEW6432
    }

    try {        
        if( ((Get-CimInstance -ClassName CIM_OperatingSystem).OSArchitecture) -like "ARM*") {
            if( [Environment]::Is64BitOperatingSystem )
            {
                return "arm64"
            }  
            return "arm"
        }
    }
    catch {
        # Machine doesn't support Get-CimInstance
    }

    return $ENV:PROCESSOR_ARCHITECTURE
}

function Get-CLIArchitecture-From-Architecture([string]$Architecture) {
    Say-Invocation $MyInvocation

    if ($Architecture -eq "<auto>") {
        $Architecture = Get-Machine-Architecture
    }

    switch ($Architecture.ToLowerInvariant()) {
        { ($_ -eq "amd64") -or ($_ -eq "x64") } { return "x64" }
        { $_ -eq "x86" } { return "x86" }
        { $_ -eq "arm" } { return "arm" }
        { $_ -eq "arm64" } { return "arm64" }
        default { throw "Architecture '$Architecture' not supported. If you think this is a bug, report it at https://github.com/dotnet/install-scripts/issues" }
    }
}

from https://github.com/dotnet/install-scripts/blob/033622cafb10f813db17a4ed511f2bbade9a77ee/src/dotnet-install.ps1#L266

crutkas pushed a commit to microsoft/PowerToys that referenced this issue Sep 25, 2024
This pull request rewrites the entire Azure DevOps build system.

The guiding principles behind this rewrite are:

- No pipeline definitions should contain steps (or tasks) directly.
- All jobs should be in template files.
- Any set of steps that is reused across multiple jobs must be in
  template files.
- All artifact names can be customized (via a property called
  `artifactStem` on all templates that produce or consume artifacts).
- No compilation happens outside of the "Build" phase, to consolidate
  the production and indexing of PDBs.
- All step and job templates are named with `step` or `job` _first_,
  which disambiguates them in the templates directory.
- Most jobs can be run on different `pool`s, so that we can put
  expensive jobs on expensive build agents and cheap jobs on cheap
  build agents. Some jobs handle pool selection on their own, however.

Our original build pipelines used the `VSBuild` task _all over the
place._ This resulted in PowerToys being built in myriad ways, different
for every pipeline. There was an attempt at standardization early on,
where `ci.yml` consumed jobs and steps templates... but when
`release.yml` was added, all of that went out the window.

It's the same story as Terminal (microsoft/terminal#15808).

The new pipelines are consistent and focus on a small, well-defined set
of jobs:

- `job-build-project`
    - This is the big one!
    - Takes a list of build configurations and platforms.
    - Produces an artifact named `build-PLATFORM-CONFIG` for the entire
      matrix of possibilities.
    - Builds all of the installers.
    - Optionally signs the output (all of the output).
    - Admittedly has a lot going on.
- `job-test-project`
    - Takes **one** build config and **one** platform.
    - Consumes `build-PLATFORM-CONFIG`
    - Selects its own pools (hardcoded) because it knows about
      architectures and must choose the right agent arch.
    - Runs tests (directly on the build agent).
- `job-publish-symbols-using-symbolrequestprod-api`
    - Consumes `**/*.pdb` from all prior build phases.
    - Uploads all PDBs in one artifact to Azure DevOps
    - Uses Microsoft's internal symbol publication REST API to submit
      stripped symbols to MSDL for public consumption.

Finally, this pull request has some additional benefits:

- Symbols are published to the private and public feeds at the same
  time, in the same step. They should be available in the public symbol
  server for public folks to debug against!
- We have all the underpinnings necessary to run tests on ARM64 build
  agents.
    - Right now, `ScreenResolutionUtility` is broken
    - I had to introduce a custom version of `UseDotNet` which would
      install the right architecture (🤦); see microsoft/azure-pipelines-tasks#20300.
- All dotnet and nuget versioning is consolidated into a small set of
  step templates.
- This will provide a great place for us to handle versioning changes
  later, since all versioning happens in one place.
@MichelZ
Copy link

MichelZ commented Dec 20, 2024

@DergachevE you seem to be code owner for this. Any chance this can get reviewed? #20695

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: ABTT Akvelon Build Tasks Team area of work bug
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants