dotnet-local & net8: local dotnet first in PATH (#8222)

The .NET 8 `dotnet` command appears to require that `dotnet` be inThe .NET 8 `dotnet` command appears to require that `dotnet` be in
`$PATH`, lest it invoke a *different* `dotnet`.  This can result in
weird error messages:

	% ~/Developer/src/xamarin/xamarin-android/dotnet-local.sh build *.csproj 
	…
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : You must install or update .NET to run this application. 
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error :  
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : App: …/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/bincore/csc.dll 
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : Architecture: x64 
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : Framework: 'Microsoft.NETCore.App', version '8.0.0-rc.1.23371.3' (x64) 
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : .NET location: /usr/local/share/dotnet/ 
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error :  
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : The following frameworks were found: 
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error :   6.0.16 at [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] 
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error :   7.0.3 at [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] 
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error :   7.0.4 at [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] 
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error :   7.0.5 at [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] 
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error :   7.0.9 at [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] 
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error :  
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : Learn about framework resolution: 
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : https://aka.ms/dotnet/app-launch-failed 
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error :  
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : To install missing framework, download: 
	…/dotnet/sdk/8.0.100-rc.1.23373.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : https://aka.ms/dotnet-core-applaunch?framework=Microsoft.NETCore.App&framework_version=8.0.0-rc.1.23371.3&arch=x64&rid=osx.13-x64 
	…

To focus on the weirdness: `dotnet-local.sh` uses the .NET 8 that we
provision into `bin/$(Configuration)/dotnet`, which is .NET 8, while
the error message states that it can only find .NET 6 and 7 from
`/usr/local/share/dotnet`, and *isn't* looking at `bin/*/dotnet`!

.NET 8 Roslyn apparently requires that the "executing" `dotnet` be in
`$PATH`:

	% PATH=…/xamarin-android/bin/Debug/dotnet:$PATH \
	    …/xamarin-android/dotnet-local.sh build *.csproj
	# no errors

Update `dotnet-local.sh` and `dotnet-local.cmd` so that `bin/*/dotnet`
is prepended to `$PATH`.  Additionally, for readability in
`dotnet-loca.sh`, explicitly export all `DOTNET*` environment
variables so that they don't all need to be on one line.

Update build scripts to use `-nodeReuse:false` instead of `-nr:false`
for readability.

…and some notes about `dotnet-local.cmd`: [`CMD.EXE`][0] environment
variable handling is "baroque".

By default, evironment variables added within a `.cmd` script are
exported to the caller.  This is similar to Unixy platforms
["source"][1]ing a script instead of executing a script.  To avoid
this behavior, the script should use [SETLOCAL][2].  As we want
`dotnet-local.cmd` to update `%PATH%`, add use of `SETLOCAL` so that
we don't update `%PATH%` on every `dotnet-local.cmd` invocation.

Environment variables are evalated *when a statement is read*,
***not*** when that statement is *executed*.  This can make for some
"weird" behavior:

	IF EXIST "%ROOT%\bin\Release\dotnet\dotnet.exe" (
	    SET XA_DOTNET_ROOT=%ROOT%\bin\Release\dotnet
	    echo XA_DOTNET_ROOT=%XA_DOTNET_ROOT%
	)

The above fragment will print `XA_DOTNET_ROOT=` because
`%XA_DOTNET_ROOT%` is evaluated when the entire `IF EXIST …` statment
is read, *not* when the `echo` is executed.  This has similar
"wierdness" if we instead attempt to update `PATH`:

	IF EXIST "%ROOT%\bin\Release\dotnet\dotnet.exe" (
	    SET PATH=%ROOT%\bin\Release\dotnet;%PATH%
	)

The above invariably fails with weirdness such as:

	\NSIS\ was unexpected at this time.

or

	\Microsoft was unexpected at this time.

The workaround is one of two options:

 1. Consider use "delayed environment variable expansion"; see the
    `SET /?` output [^0].

 2. Don't Do That™: don't `SET` an environment variable to include
    values which are also `SET` in the same statement, and `(…)`
    is evaluated as a single statement.

We opt to go with (2), drastically simplifying `dotnet-local.cmd` so
that the `COMMAND` within `IF EXIST filename COMMAND` *only* sets
environment variables which do not reference other environment
variables within the same `COMMAND`

[0]: https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/cmd
[1]: https://linuxize.com/post/bash-source-command/
[2]: https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/setlocal

[^0]: From `SET /?`:

	Delayed environment variable expansion is useful for getting around
	the limitations of the current expansion which happens when a line
	of text is read, not when it is executed.  The following example
	demonstrates the problem with immediate variable expansion:

	    set VAR=before
	    if "%VAR%" == "before" (
	        set VAR=after
	        if "%VAR%" == "after" @echo If you see this, it worked
	    )

	would never display the message, since the %VAR% in BOTH IF statements
	is substituted when the first IF statement is read, since it logically
	includes the body of the IF, which is a compound statement.  So the
	IF inside the compound statement is really comparing "before" with
	"after" which will never be equal.  Similarly, the following example
	will not work as expected:

	    set LIST=
	    for %i in (*) do set LIST=%LIST% %i
	    echo %LIST%

	in that it will NOT build up a list of files in the current directory,
	but instead will just set the LIST variable to the last file found.
	Again, this is because the %LIST% is expanded just once when the
	FOR statement is read, and at that time the LIST variable is empty.
	So the actual FOR loop we are executing is:

	    for %i in (*) do set LIST= %i

	which just keeps setting LIST to the last file found.

	Delayed environment variable expansion allows you to use a different
	character (the exclamation mark) to expand environment variables at
	execution time. …

Author: dellis1972

Diff for: Documentation/building/windows/instructions.md

@@ -20,21 +20,18 @@ MSBuild version 15 or later is required.
 
  5. In a [Developer Command Prompt][developer-prompt], prepare the project:
 
-        dotnet msbuild Xamarin.Android.sln -t:Prepare
+        dotnet msbuild Xamarin.Android.sln -t:Prepare -nodeReuse:false
+        dotnet msbuild external\Java.Interop\Java.Interop.sln -t:Prepare -nodeReuse:false
 
     This will ensure that the build dependencies are installed, perform
     `git submodule update`, download NuGet dependencies, and other
     "preparatory" and pre-build tasks that need to be performed.
 
  6. Build the project:
 
-        dotnet-local.cmd build Xamarin.Android.sln -m:1
+        dotnet-local.cmd build Xamarin.Android.sln  -nodeReuse:false
 
- 7. In order to use the in-tree Xamarin.Android, build xabuild:
-
-         msbuild tools/xabuild/xabuild.csproj /restore
-
- 8. (For Microsoft team members only - Optional) In a [Developer Command
+ 7. (For Microsoft team members only - Optional) In a [Developer Command
     Prompt][developer-prompt], build external proprietary git
     dependencies:
 

Diff for: build-tools/automation/azure-pipelines.yaml

@@ -112,7 +112,7 @@ stages:
         restoreNUnitConsole: false
         updateMono: false
         xaprepareScenario: EmulatorTestDependencies
-        
+
     - task: DownloadPipelineArtifact@2
       inputs:
         artifactName: $(TestAssembliesArtifactName)
@@ -140,7 +140,7 @@ stages:
         arguments: -t:PrepareJavaInterop -c $(XA.Build.Configuration) --no-restore
         displayName: prepare java.interop $(XA.Build.Configuration)
         continueOnError: false
-      
+
     - template: yaml-templates/start-stop-emulator.yaml
 
     - template: yaml-templates/apk-instrumentation.yaml
@@ -201,7 +201,7 @@ stages:
         extraBuildArgs: -p:TestsFlavor=AotLlvm -p:EnableLLVM=true -p:AndroidEnableProfiledAot=false
         artifactSource: bin/Test$(XA.Build.Configuration)/$(DotNetTargetFramework)-android/Mono.Android.NET_Tests-Signed.aab
         artifactFolder: $(DotNetTargetFramework)-AotLlvm
-        
+
     - template: yaml-templates/apk-instrumentation.yaml
       parameters:
         configuration: $(XA.Build.Configuration)
@@ -220,13 +220,19 @@ stages:
         artifactSource: bin/Test$(XA.Build.Configuration)/$(DotNetTargetFramework)-android/Xamarin.Android.JcwGen_Tests-Signed.apk
         artifactFolder: $(DotNetTargetFramework)-FastDev_Assemblies_Dexes
         extraBuildArgs: /p:AndroidFastDeploymentType=Assemblies:Dexes
-        
+
     - template: yaml-templates/run-nunit-tests.yaml
       parameters:
         testRunTitle: Xamarin.Android.Tools.Aidl-Tests - macOS
         testAssembly: $(System.DefaultWorkingDirectory)/bin/Test$(XA.Build.Configuration)/$(DotNetStableTargetFramework)/Xamarin.Android.Tools.Aidl-Tests.dll
         testResultsFile: TestResult-Aidl-Tests-macOS-$(XA.Build.Configuration).xml
 
+    - task: ShellScript@2
+      displayName: Test dotnet-local.sh
+      inputs:
+        scriptPath: dotnet-local.sh
+        args: build samples/HelloWorld/HelloWorld/HelloWorld.DotNet.csproj
+
     - ${{ if ne(parameters.macTestAgentsUseCleanImages, true) }}:
       - template: yaml-templates/start-stop-emulator.yaml
         parameters:
@@ -276,6 +282,12 @@ stages:
         dotNetTestExtraArgs: --filter "TestCategory = SmokeTests"
         testResultsFile: TestResult-NETSmokeMSBuildTests-Linux-$(XA.Build.Configuration).xml
 
+    - task: ShellScript@2
+      displayName: Test dotnet-local.sh
+      inputs:
+        scriptPath: dotnet-local.sh
+        args: build samples/HelloWorld/HelloWorld/HelloWorld.DotNet.csproj
+
     - template: yaml-templates/upload-results.yaml
       parameters:
         configuration: $(XA.Build.Configuration)

Diff for: build-tools/automation/yaml-templates/build-windows.yaml

@@ -98,6 +98,12 @@ stages:
         testResultsFile: TestResult-SmokeMSBuildTests-WinDotnetBuild-$(XA.Build.Configuration).xml
         dotNetTestExtraArgs: --filter "TestCategory = SmokeTests"
 
+    - task: BatchScript@1
+      displayName: Test dotnet-local.cmd
+      inputs:
+        filename: dotnet-local.cmd
+        arguments: build samples\HelloWorld\HelloWorld\HelloWorld.DotNet.csproj
+
     - template: upload-results.yaml
       parameters:
         artifactName: ${{ parameters.buildResultArtifactName }}

Diff for: build-tools/scripts/PrepareWindows.targets

@@ -24,5 +24,6 @@
     />
     <Exec Command="dotnet $(_XAPrepareExe) $(_XAPrepareStandardArgs) -a" WorkingDirectory="$(_TopDir)" />
     <MSBuild Projects="$(_TopDir)\Xamarin.Android.BootstrapTasks.sln" Targets="Restore;Build" />
+    <CallTarget Targets="PrepareJavaInterop" />
   </Target>
 </Project>

Diff for: build.cmd

@@ -8,23 +8,21 @@ IF ERRORLEVEL 1 CALL:FAILED_CASE
 IF ERRORLEVEL 1 CALL :DEFAULT_CASE
 
 :Prepare_CASE
-    dotnet build Xamarin.Android.sln -t:Prepare -nr:false
+    dotnet msbuild Xamarin.Android.sln -t:Prepare -nodeReuse:false
     GOTO END_CASE
 :PrepareExternal_CASE
-    dotnet build Xamarin.Android.sln -t:PrepareExternal -nr:false
+    dotnet build Xamarin.Android.sln -t:PrepareExternal -nodeReuse:false
     GOTO END_CASE
 :Build_CASE
-    dotnet-local.cmd build Xamarin.Android.sln  -nr:false
-    dotnet-local.cmd build tools/xabuild/xabuild.csproj -nr:false
+    dotnet-local.cmd build Xamarin.Android.sln  -nodeReuse:false
     GOTO END_CASE
 :Pack_CASE
-    dotnet-local.cmd build  Xamarin.Android.sln -t:PackDotNet -nr:false
+    dotnet-local.cmd build  Xamarin.Android.sln -t:PackDotNet -nodeReuse:false
     GOTO END_CASE
 :DEFAULT_CASE
-    dotnet build Xamarin.Android.sln -t:Prepare -nr:false
-    dotnet-local.cmd build Xamarin.Android.sln -nr:false
-    dotnet-local.cmd build tools/xabuild/xabuild.csproj -nr:false
-    dotnet-local.cmd build Xamarin.Android.sln -t:PackDotNet -nr:false
+    dotnet msbuild Xamarin.Android.sln -t:Prepare -nodeReuse:false
+    dotnet-local.cmd build Xamarin.Android.sln -nodeReuse:false
+    dotnet-local.cmd build Xamarin.Android.sln -t:PackDotNet -nodeReuse:false
     GOTO END_CASE
 :FAILED_CASE
     echo "Failed to find an instance of Visual Studio. Please check it is correctly installed."

Diff for: dotnet-local.cmd

@@ -1,13 +1,22 @@
 @echo off
+SETLOCAL
+
 SET ROOT=%~dp0
+
 IF EXIST "%ROOT%\bin\Release\dotnet\dotnet.exe" (
-    SET DOTNETSDK_WORKLOAD_MANIFEST_ROOTS=%ROOT%\bin\Release\lib\sdk-manifests
-    SET DOTNETSDK_WORKLOAD_PACK_ROOTS=%ROOT%\bin\Release\lib
-    call "%ROOT%\bin\Release\dotnet\dotnet.exe" %*
+    SET XA_CONFIG=Release
 ) ELSE IF EXIST "%ROOT%\bin\Debug\dotnet\dotnet.exe" (
-    SET DOTNETSDK_WORKLOAD_MANIFEST_ROOTS=%ROOT%\bin\Debug\lib\sdk-manifests
-    SET DOTNETSDK_WORKLOAD_PACK_ROOTS=%ROOT%\bin\Debug\lib
-    call "%ROOT%\bin\Debug\dotnet\dotnet.exe" %*
+    SET XA_CONFIG=Debug
 ) ELSE (
     echo "You need to run 'msbuild Xamarin.Android.sln /t:Prepare' first."
-)
+    goto :exit
+)
+
+SET XA_DOTNET_ROOT=%ROOT%\bin\%XA_CONFIG%\dotnet
+SET PATH=%XA_DOTNET_ROOT%;%PATH%
+SET DOTNETSDK_WORKLOAD_MANIFEST_ROOTS=%ROOT%\bin\%XA_CONFIG%\lib\sdk-manifests
+SET DOTNETSDK_WORKLOAD_PACK_ROOTS=%ROOT%\bin\%XA_CONFIG%\lib
+
+call "%XA_DOTNET_ROOT%\dotnet.exe" %*
+
+:exit

Diff for: dotnet-local.sh

@@ -1,10 +1,15 @@
 #!/bin/bash
 ROOT="$(dirname "${BASH_SOURCE}")"
 FULLROOT="$(cd "${ROOT}"; pwd)"
-if [[ -x "${ROOT}/bin/Release/dotnet/dotnet" ]]; then
-    DOTNETSDK_WORKLOAD_MANIFEST_ROOTS=${FULLROOT}/bin/Release/lib/sdk-manifests DOTNETSDK_WORKLOAD_PACK_ROOTS=${FULLROOT}/bin/Release/lib exec ${ROOT}/bin/Release/dotnet/dotnet "$@"
-elif [[ -x "${ROOT}/bin/Debug/dotnet/dotnet" ]]; then
-    DOTNETSDK_WORKLOAD_MANIFEST_ROOTS=${FULLROOT}/bin/Debug/lib/sdk-manifests DOTNETSDK_WORKLOAD_PACK_ROOTS=${FULLROOT}/bin/Debug/lib exec ${ROOT}/bin/Debug/dotnet/dotnet "$@"
-else
-    echo "You need to run 'make prepare' first."
-fi
+for config in Release Debug ; do
+    if [[ ! -x "${ROOT}/bin/${config}/dotnet/dotnet" ]] ; then
+        continue
+    fi
+    XA_DOTNET_ROOT="${FULLROOT}/bin/${config}/dotnet"
+    export PATH="${XA_DOTNET_ROOT}:${PATH}"
+    export DOTNETSDK_WORKLOAD_MANIFEST_ROOTS="${FULLROOT}/bin/${config}/lib/sdk-manifests"
+    export DOTNETSDK_WORKLOAD_PACK_ROOTS="${FULLROOT}/bin/${config}/lib"
+    exec "${ROOT}/bin/${config}/dotnet/dotnet" "$@"
+done
+
+echo "You need to run 'make prepare' first."

Diff for: src/Xamarin.Android.Build.Tasks/Tests/Xamarin.ProjectTools/Common/DotNetCLI.cs

@@ -152,7 +152,7 @@ List<string> GetDefaultCommandLineArgs (string verb, string target = null, strin
 				$"/flp1:LogFile=\"{BuildLogFile}\";Encoding=UTF-8;Verbosity={Verbosity}",
 				$"/bl:\"{Path.Combine (testDir, $"{(string.IsNullOrEmpty (target) ? "msbuild" : target)}.binlog")}\"",
 				"-m:1",
-				"-nr:false",
+				"-nodeReuse:false",
 				"/p:_DisableParallelAot=true",
 			};
 			if (!string.IsNullOrEmpty (target)) {