WIP: Create a unified build for (swift-)foundation-tests

[ahoppen]

We can’t add the tests to the general unified build because they use testable imports and building a package with testable imports also re-builds all of its dependencies with testable imports (swiftlang/swift-package-manager#8344) and is thus incompatible with the other package’s unified build. But we should still be able to share build products between corelibs-foundation tests and swift-foundation-tests.

[ahoppen]

@swift-ci Please smoke test Linux

[ahoppen]

@swift-ci Please smoke test Linux

[ahoppen]

@swift-ci Please test Linux

[ahoppen]

@swift-ci Please smoke test Linux

[ahoppen]

@swift-ci Please smoke test Linux

[ahoppen]

OK, this seems to be trickier than I initially thought because: ' swift-corelibs-foundation' requires the 'plutil executable
product to be built for its tests to pass. When doing the unified build, we need to pass '--test-product to specify the targets that should be tested. The package's test product doesn't contain the 'plutil' executable product and thus 'plutil' is not getting built. Performing an independent build of plutil' using a separate 'swift build' invocation performs a build with testability disabled, which conflicts with the swift test build that has testability enabled, causing all targets (including dependencies) to get rebuilt. As far as I can tell, there is no way to build a non-test product with testability enabled.

Possible solutions are (in no particular order):

1. Avoid using testable imports in swift-foundation and swift-corelibs-foundation, and instead use the package access level or @_spi(Testing) imports. Then we would perform all builds with testability disabled and wouldn’t have the conflict when building plutil. This would have the added advantage that swift-corelibs-foundation and swift-foundation could join the unified build of swift-syntax etc. and thus avoid rebuilding swift-syntax at all (saving ~30 minutes of CI time in total).
2. Build and run tests in debug instead of release configuration. Debug builds always have testability enabled, so we shouldn’t have the conflict anymore. This shouldn’t have any effect on the produced toolchain because the version of swift-foundation and swift-corelibs-foundation being installed into the toolchain aren’t produced by the swiftfoundationtest and foundationtest build script products anyway. While this would still cause a re-build of swift-syntax in CI, since we wouldn’t be re-using its unified build, it would be significantly faster. I did not measure but I would expect any improvements in build time to outweigh slow downs in test execution because tests currently finish in ~1 minute while the build takes >15 minutes.
3. Add an option to SwiftPM to perform a build that has testability enabled. That way we could build plutil with testability enabled and wouldn’t need to clean the build directory for it.
4. Adopt SE-0455 in swift-(corelibs-)foundation and only build those targets with testability enabled that actually need it. Similar to (1), this means that swift-foundation could join the entire unified build. A catch seems to be that the PR implementing the proposal hasn’t been merged yet. Also, if swift-(corelibs-)foundation need to support building with older versions of Swift, they would need a [email protected] manifest that controls these flags
5. Re-design TestProcess .test_plutil to not assume that plutil is present in the build directory. I don’t know much about this test but as far as I can tell, it is not really testing behavior of plutil itself, so this might be possible to re-design the test to use some other executable to test the Process type, which seems to be the test’s purpose.

[finagolfin]

I implemented 2. in #78390, some data on how much CI time it saved there.