Skip to content

8257679: Improved unix compatibility layer in Windows build (winenv) #1597

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from
Closed

8257679: Improved unix compatibility layer in Windows build (winenv) #1597

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
855751f
8257679: Improved unix compatibility layer in Windows build (winenv)
magicus Dec 3, 2020
c00d5a9
Missed my last bugfix for fixpath.sh...
magicus Dec 3, 2020
20f2619
Address remarks in code review
magicus Dec 4, 2020
5b7a749
Make fixpath import ignore bad paths while importing path lists
magicus Dec 4, 2020
b8a2412
Fix syntax error in fixpath.sh
magicus Dec 4, 2020
41cee7d
Fix for .exe on WSL in fixpath.sh
magicus Dec 4, 2020
ef81e1f
Extract only the actual contents added to the PATH by VS SetEnv.cmd.
magicus Dec 5, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions .github/workflows/submit.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -938,15 +938,18 @@ jobs:
run: >
$env:Path = "$HOME\cygwin\cygwin64\bin;$HOME\cygwin\cygwin64\bin;$env:Path" ;
$env:Path = $env:Path -split ";" -match "C:\\Windows|PowerShell|cygwin" -join ";" ;
$env:BOOT_JDK = cygpath "$HOME/bootjdk/$env:BOOT_JDK_VERSION" ;
$env:JT_HOME = cygpath "$HOME/jtreg" ;
$env:GTEST = cygpath "$env:GITHUB_WORKSPACE/gtest" ;
& bash configure
--with-conf-name=windows-x64
--with-msvc-toolset-version=14.27
${{ matrix.flags }}
--with-version-opt="$env:GITHUB_ACTOR-$env:GITHUB_SHA"
--with-version-build=0
--with-boot-jdk="$HOME/bootjdk/$env:BOOT_JDK_VERSION"
--with-jtreg="$HOME/jtreg"
--with-gtest="$env:GITHUB_WORKSPACE/gtest"
--with-boot-jdk="$env:BOOT_JDK"
--with-jtreg="$env:JT_HOME"
--with-gtest="$env:GTEST"
--with-default-make-target="product-bundles test-bundles"
--enable-jtreg-failure-handler
working-directory: jdk
Expand Down
2 changes: 1 addition & 1 deletion make/CreateJmods.gmk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,7 +169,7 @@ ifeq ($(MODULE), java.base)
endif
endif
else # not java.base
ifeq ($(call isTargetOs, windows), true)
ifeq ($(call isTargetOs, windows)+$(CREATING_BUILDJDK), true+false)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks strange. Why would CREATING_BUILDJDK be relevant here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we are creating a buildjdk (which really means we are cross-compiling), we must not meddle with the Windows runtime dll:s, since they are for the target CPU architecture. To be fully correct, we should really turn the Windows runtime dll:s into a "normal"/BUILD_ split version, like with SYSROOT etc.

But since this is just used locally for building, I'm assuming the build system has these runtime DLLs installed already. There seems to be no point in wasting time preparing a buildjdk with files that are just bundled for installation.

Maybe there's a better way to express this, but the only thing I could get a hold of for the fact that we are creating a build-JDK (that is, not targeting the normal OPENJDK_TARGET_CPU) was the CREATING_BUILDJDK variable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reverted this change, and added a comment when we copy the libs.

# Only java.base needs to include the MSVC*_DLLs. Make sure no other module
# tries to include them (typically imported ones).
ifneq ($(MSVCR_DLL), )
Expand Down
4 changes: 2 additions & 2 deletions make/GenerateLinkOptData.gmk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ endif

# Save the stderr output of the command and print it along with stdout in case
# something goes wrong.
$(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) $(CLASSLIST_JAR)
$(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXECUTABLE_SUFFIX) $(CLASSLIST_JAR)
$(call MakeDir, $(LINK_OPT_DIR))
$(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $@))
$(call LogInfo, Generating $(patsubst $(OUTPUTDIR)/%, %, $(JLI_TRACE_FILE)))
Expand Down Expand Up @@ -94,7 +94,7 @@ $(CLASSLIST_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) $(CLASSLIST_JAR)
# dependencies, make will correctly rebuild both jli trace and classlist
# incrementally using the single recipe above.
$(CLASSLIST_FILE): $(JLI_TRACE_FILE)
$(JLI_TRACE_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXE_SUFFIX) $(CLASSLIST_JAR)
$(JLI_TRACE_FILE): $(INTERIM_IMAGE_DIR)/bin/java$(EXECUTABLE_SUFFIX) $(CLASSLIST_JAR)

TARGETS += $(CLASSLIST_FILE) $(JLI_TRACE_FILE)

Expand Down
4 changes: 2 additions & 2 deletions make/InterimImage.gmk
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#
# Copyright (c) 2016, 2018, Oracle and/or its affiliates. All rights reserved.
# Copyright (c) 2016, 2020, Oracle and/or its affiliates. All rights reserved.
# DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
#
# This code is free software; you can redistribute it and/or modify it
Expand Down Expand Up @@ -32,7 +32,7 @@ include Modules.gmk
################################################################################

# Use this file inside the image as target for make rule
JIMAGE_TARGET_FILE := bin/java$(EXE_SUFFIX)
JIMAGE_TARGET_FILE := bin/java$(EXECUTABLE_SUFFIX)

INTERIM_MODULES_LIST := $(call CommaList, $(INTERIM_IMAGE_MODULES))

Expand Down
17 changes: 6 additions & 11 deletions make/RunTests.gmk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -60,19 +60,14 @@ define SetTestOpt
endif
endef

# Setup _NT_SYMBOL_PATH on Windows
# Setup _NT_SYMBOL_PATH on Windows, which points to our pdb files.
ifeq ($(call isTargetOs, windows), true)
ifndef _NT_SYMBOL_PATH
# Can't use PathList here as it adds quotes around the value.
_NT_SYMBOL_PATH := \
$(subst $(SPACE),;,$(strip \
$(foreach p, $(sort $(dir $(wildcard \
$(addprefix $(SYMBOLS_IMAGE_DIR)/bin/, *.pdb */*.pdb)))), \
$(call FixPath, $p) \
) \
))
export _NT_SYMBOL_PATH
$(call LogDebug, Rewriting _NT_SYMBOL_PATH to $(_NT_SYMBOL_PATH))
SYMBOL_PATH := $(call PathList, $(sort $(patsubst %/, %, $(dir $(wildcard \
$(addprefix $(SYMBOLS_IMAGE_DIR)/bin/, *.pdb */*.pdb))))))
export _NT_SYMBOL_PATH := $(subst \\,\, $(call FixPath, \
$(subst $(DQUOTE),, $(SYMBOL_PATH))))
$(call LogDebug, Setting _NT_SYMBOL_PATH to $(_NT_SYMBOL_PATH))
endif
endif

Expand Down
7 changes: 2 additions & 5 deletions make/RunTestsPrebuilt.gmk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -189,15 +189,13 @@ ifeq ($(OPENJDK_TARGET_CPU), x86_64)
endif

ifeq ($(OPENJDK_TARGET_OS), windows)
ifeq ($(wildcard $(TEST_IMAGE_DIR)/bin/fixpath.exe), )
ifeq ($(wildcard $(TEST_IMAGE_DIR)/bin/fixpath.sh), )
$(info Error: fixpath is missing from test image '$(TEST_IMAGE_DIR)')
$(error Cannot continue.)
endif
FIXPATH := $(TEST_IMAGE_DIR)/bin/fixpath.exe -c
PATH_SEP:=;
FIXPATH := $(BASH) $(TEST_IMAGE_DIR)/bin/fixpath.sh exec
else
FIXPATH :=
PATH_SEP:=:
endif

# Check number of cores and memory in MB
Expand Down Expand Up @@ -280,7 +278,6 @@ $(call CreateNewSpec, $(NEW_SPEC), \
BASH := $(BASH), \
JIB_JAR := $(JIB_JAR), \
FIXPATH := $(FIXPATH), \
PATH_SEP := $(PATH_SEP), \
OPENJDK_TARGET_OS := $(OPENJDK_TARGET_OS), \
OPENJDK_TARGET_OS_TYPE := $(OPENJDK_TARGET_OS_TYPE), \
OPENJDK_TARGET_OS_ENV := $(OPENJDK_TARGET_OS_ENV), \
Expand Down
6 changes: 1 addition & 5 deletions make/RunTestsPrebuiltSpec.gmk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,16 +116,13 @@ JAVAC_CMD := $(BOOT_JDK)/bin/javac
JAR_CMD := $(BOOT_JDK)/bin/jar
JLINK_CMD := $(JDK_OUTPUTDIR)/bin/jlink
JMOD_CMD := $(JDK_OUTPUTDIR)/bin/jmod
JARSIGNER_CMD := $(BOOT_JDK)/bin/jarsigner

JAVA := $(FIXPATH) $(JAVA_CMD) $(JAVA_FLAGS_BIG) $(JAVA_FLAGS)
JAVA_SMALL := $(FIXPATH) $(JAVA_CMD) $(JAVA_FLAGS_SMALL) $(JAVA_FLAGS)
JAVA_DETACH := $(FIXPATH) $(FIXPATH_DETACH_FLAG) $(JAVA_CMD) $(JAVA_FLAGS_BIG) $(JAVA_FLAGS)
JAVAC := $(FIXPATH) $(JAVAC_CMD)
JAR := $(FIXPATH) $(JAR_CMD)
JLINK := $(FIXPATH) $(JLINK_CMD)
JMOD := $(FIXPATH) $(JMOD_CMD)
JARSIGNER := $(FIXPATH) $(JARSIGNER_CMD)

BUILD_JAVA := $(JDK_IMAGE_DIR)/bin/JAVA
################################################################################
Expand Down Expand Up @@ -166,7 +163,6 @@ TAIL := tail
TEE := tee
TR := tr
TOUCH := touch
UNIQ := uniq
WC := wc
XARGS := xargs
ZIPEXE := zip
Expand All @@ -177,7 +173,7 @@ HG := hg
ULIMIT := ulimit

ifeq ($(OPENJDK_BUILD_OS), windows)
CYGPATH := cygpath
PATHTOOL := cygpath
endif

################################################################################
Expand Down
4 changes: 2 additions & 2 deletions make/TestImage.gmk
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,9 @@ include MakeBase.gmk
############################################################################

ifeq ($(call isTargetOs, windows), true)
FIXPATH_COPY := $(TEST_IMAGE_DIR)/bin/fixpath.exe
FIXPATH_COPY := $(TEST_IMAGE_DIR)/bin/fixpath.sh

$(FIXPATH_COPY): $(firstword $(FIXPATH))
$(FIXPATH_COPY): $(TOPDIR)/make/scripts/fixpath.sh
$(call install-file)

FIXPATH_WORKSPACE_ROOT := $(call FixPath, $(WORKSPACE_ROOT))
Expand Down
49 changes: 19 additions & 30 deletions make/autoconf/basic.m4
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,34 +70,28 @@ AC_DEFUN_ONCE([BASIC_SETUP_PATHS],
fi

if test "x$OPENJDK_TARGET_OS" = "xwindows"; then
PATH_SEP=";"
EXE_SUFFIX=".exe"
BASIC_CHECK_PATHS_WINDOWS
else
PATH_SEP=":"
EXE_SUFFIX=""
BASIC_SETUP_PATHS_WINDOWS
fi
AC_SUBST(PATH_SEP)
AC_SUBST(EXE_SUFFIX)

# We get the top-level directory from the supporting wrappers.
BASIC_WINDOWS_VERIFY_DIR($TOPDIR, source)
UTIL_FIXUP_PATH(TOPDIR)
AC_MSG_CHECKING([for top-level directory])
AC_MSG_RESULT([$TOPDIR])
AC_SUBST(TOPDIR)
AC_SUBST(CONFIGURE_START_DIR)

# We can only call UTIL_FIXUP_PATH after BASIC_CHECK_PATHS_WINDOWS.
UTIL_FIXUP_PATH(TOPDIR)
UTIL_FIXUP_PATH(CONFIGURE_START_DIR)

if test "x$CUSTOM_ROOT" != x; then
BASIC_WINDOWS_VERIFY_DIR($CUSTOM_ROOT, custom root)
UTIL_FIXUP_PATH(CUSTOM_ROOT)
WORKSPACE_ROOT="${CUSTOM_ROOT}"
else
WORKSPACE_ROOT="${TOPDIR}"
fi
AC_SUBST(WORKSPACE_ROOT)

UTIL_FIXUP_PATH(CONFIGURE_START_DIR)
AC_SUBST(CONFIGURE_START_DIR)

# Locate the directory of this script.
AUTOCONF_DIR=$TOPDIR/make/autoconf

Expand Down Expand Up @@ -234,9 +228,12 @@ AC_DEFUN_ONCE([BASIC_SETUP_DEVKIT],
# if no Xcode installed, xcodebuild exits with 1
# if Xcode is installed, even if xcode-select is misconfigured, then it exits with 0
if test "x$DEVKIT_ROOT" != x || /usr/bin/xcodebuild -version >/dev/null 2>&1; then
# We need to use xcodebuild in the toolchain dir provided by the user, this will
# fall back on the stub binary in /usr/bin/xcodebuild
AC_PATH_PROG([XCODEBUILD], [xcodebuild], [/usr/bin/xcodebuild], [$TOOLCHAIN_PATH])
# We need to use xcodebuild in the toolchain dir provided by the user
UTIL_LOOKUP_PROGS(XCODEBUILD, xcodebuild, $TOOLCHAIN_PATH)
if test x$XCODEBUILD = x; then
# fall back on the stub binary in /usr/bin/xcodebuild
XCODEBUILD=/usr/bin/xcodebuild
fi
else
# this should result in SYSROOT being empty, unless --with-sysroot is provided
# when only the command line tools are installed there are no SDKs, so headers
Expand Down Expand Up @@ -306,6 +303,7 @@ AC_DEFUN_ONCE([BASIC_SETUP_DEVKIT],
AC_MSG_RESULT([$SYSROOT])
AC_MSG_CHECKING([for toolchain path])
AC_MSG_RESULT([$TOOLCHAIN_PATH])
AC_SUBST(TOOLCHAIN_PATH)
AC_MSG_CHECKING([for extra path])
AC_MSG_RESULT([$EXTRA_PATH])
])
Expand Down Expand Up @@ -380,6 +378,7 @@ AC_DEFUN_ONCE([BASIC_SETUP_OUTPUT_DIR],
AC_MSG_CHECKING([what configuration name to use])
AC_MSG_RESULT([$CONF_NAME])

BASIC_WINDOWS_VERIFY_DIR($OUTPUTDIR, output)
UTIL_FIXUP_PATH(OUTPUTDIR)

CONFIGURESUPPORT_OUTPUTDIR="$OUTPUTDIR/configure-support"
Expand Down Expand Up @@ -416,26 +415,16 @@ AC_DEFUN([BASIC_CHECK_DIR_ON_LOCAL_DISK],
# df -l lists only local disks; if the given directory is not found then
# a non-zero exit code is given
if test "x$DF" = x; then
if test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.msys"; then
# msys does not have df; use Windows "net use" instead.
IS_NETWORK_DISK=`net use | grep \`pwd -W | cut -d ":" -f 1 | tr a-z A-Z\`:`
if test "x$IS_NETWORK_DISK" = x; then
$2
else
$3
fi
else
# No df here, say it's local
$2
fi
# No df here, say it's local
$2
else
# JDK-8189619
# df on AIX does not understand -l. On modern AIXes it understands "-T local" which
# is the same. On older AIXes we just continue to live with a "not local build" warning.
if test "x$OPENJDK_TARGET_OS" = xaix; then
DF_LOCAL_ONLY_OPTION='-T local'
elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.wsl"; then
# In WSL, we can only build on a drvfs file system (that is, a mounted real Windows drive)
elif test "x$OPENJDK_BUILD_OS_ENV" = "xwindows.wsl1"; then
# In WSL1, we can only build on a drvfs file system (that is, a mounted real Windows drive)
DF_LOCAL_ONLY_OPTION='-t drvfs'
else
DF_LOCAL_ONLY_OPTION='-l'
Expand Down
Loading