updated perf data and fixed perf counter, faster building without binary copy. Added v65 notes.

Modified build to avoid copying the cef binary locally. This would allow for remotely building these containers much faster.
Updated build times for full auto build on server with latest scripts.
Write perf data to a log file as well.
Added v65 sample.
Fixed per timing to properly update the global variable.

Author: mitchcapper

Dockerfile_cef_binary

@@ -1,4 +1,7 @@
-FROM cef_compiled
+FROM cef as builder
+
+FROM vs
+COPY --from=builder c:/code/binaries c:/code/binaries
 
 SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'SilentlyContinue';"]
 

Dockerfile_cef_compiled

@@ -21,7 +21,7 @@
 ## Summary
 Automated chrome cef building and cefsharp building dockerfiles and scripts.
 
-While the processes of building CEF and CEFSHARP are not hard they require a very exacting environment and build steps can take a _long_ time so are annoying to repeat.  The goal if this repo is a collection of scripts to automate everything to make it easy for anyone to do.  We are using Docker to run everything in a container as it makes it much easier to reproduce and won't pollute your dev environment with all the pre-reqs.  You can easily tweak the exact versions you want to build, and the build flags.  From creating a VM on your cloud provider of choice (or your own machine) it is about 20 minutes of setup, starting a build script, and just waiting a few hours for it to spit out the compiled binaries. 
+While the processes of building CEF and CEFSHARP are not hard they require a very exacting environment and build steps can take a _long_ time so are annoying to repeat.  The goal if this repo is a collection of scripts to automate everything to make it easy for anyone to do.  We are using Docker to run everything in a container as it makes it much easier to reproduce and won't pollute your dev environment with all the pre-reqs.  You can easily tweak the exact versions you want to build, and the build flags.  From creating a VM on your cloud provider of choice (or your own machine) it is about 20 minutes of setup, starting a build script, and just waiting a few hours for it to spit out the compiled binaries.  It has been tested with 63 and 65.
 
 
 ## Thanks
@@ -34,7 +34,7 @@ Beware if using the exact same version string as an official CEF Build as it wil
 In part we use the latest version of several installers/build tools if they changed so might the success of these dockerfiles.  It does not build the debug versions of CEF or CEFSharp.   This could be added as an option pretty easily (but would probably at-least double build times). For some reason I had issues getting the automated build script for CEF to work doing the calls by hand is pretty basic however.
 
 ## Requirements
-The following requirements are for chrome 63 and the current vs_2017 installer, they may change over time.  Compiling is largely CPU bound but linking is largely IO bound.
+The following requirements are for chrome 63(and more or less 65) and the current vs_2017 installer, they may change over time.  Compiling is largely CPU bound but linking is largely IO bound.
 
 - At least 32GB of ram dedicated to this would recommend 40GB total with page file to make sure you don't run out.  You can have any amount of that 32/40GB as a page file, just beware the less actual ram the much slower linking will be.
 - At least 250GB of space.
@@ -68,13 +68,12 @@ Create a new resource, search for the prebuilt image noted above.  You do not ne
 I suggest auto-shutdown to make sure you don't leave it running.
 
 ### Estimated time requirements
-With the Azure F32 v2 host above the total estimated build time is about 3.5 hours (~ $10 on azure). Machines are nice 600MB/sec read/write to the local disk.  The time could be cut close to in half if you used a F64 v2 VM, but your cost will remain the same (as its twice the price for twice the power).  Note it can vary somewhat dramatically for the not cef build steps based on the luck of the draw (but the cef build is most of the build time).  It seems local IO depending on what physical host it is spun up on can cause 30-50% performance fluxes.  Most of the build steps make efficient use of the machine however: The git cloning is not very efficient. It is 30 minutes of the cef build time below. It doesn't quite max out network or IO. The linking stage is also not super efficient see the DUAL_BUILD flag below to help with that. Linking will take 20+ minutes per platform (40 total unless run concurrently).  Here are the individual build/commit times:
-- pull source image: 6 minutes
-- vs: 11.6 minutes
-- cef: 2.8 hours
-- cef binary copy: 6 seconds
-- cef_compiled: 65 seconds
-- cefsharp: 12 minutes
+With the Azure F32 v2 host above the total estimated build time is about 2.1 hours (~$6 on azure). Machines are nice 600MB/sec read/write to the local disk.  The time could be cut close to in half if you used a F64 v2 VM, but your cost will remain the same (as its twice the price for twice the power).  Note it can vary somewhat dramatically for the not cef build steps based on the luck of the draw (but the cef build is most of the build time).  It seems local IO depending on what physical host it is spun up on can cause 30-50% performance fluxes.  Most of the build steps make efficient use of the machine however: The git cloning is not very efficient. It is 30 minutes of the cef build time below. It doesn't quite max out network or IO. The linking stage is also not super efficient see the DUAL_BUILD flag below to help with that. Linking will take 20+ minutes per platform (40 total unless run concurrently).  Here are the individual build/commit times:
+- pull source image: 5 minutes
+- vs: 11 minutes
+- cef: 1.8 hours
+- cef-binary: 3 minutes
+- cefsharp: 3.7 minutes
 
 
 
@@ -101,7 +100,8 @@ Note the DUAL_BUILD may speed up builds by running x86 and x64 builds concurrent
 
 
 ## Docker for Windows Caveats
-- It is slow and Docker for windows is flaky at *best*. Keep in mind this is running latest 16299 w/ 32 gigs of ram. Sometimes it will miss a step that should be cached and redo it. It seems less flaky running docker in process isolation mode (--isolation=process) instead of hyper-v mode.  This is a legal compile time limitation however that windows clients cannot use this mode.  Granted building your own docker windows binary is also not that hard.  NOTE: If you are not using process isolation mode you WILL need to do -m 64 (or similar) on every docker build step to up the default memory limit.
+- Most of these issues become an issue on the longer docker runs and disk speed. On an Azure VM it rarely fails out.  There has been some redundancy added to build scripts to address these issues.
+- It is slow and Docker for windows can be flaky. Keep in mind this is running latest 16299 w/ 32 gigs of ram. Sometimes it will miss a step that should be cached and redo it. It seems less flaky running docker in process isolation mode (--isolation=process) instead of hyper-v mode.  This is a legal compile time limitation however that windows clients cannot use this mode.  Granted building your own docker windows binary is also not that hard.  NOTE: If you are not using process isolation mode you WILL need to do -m 64 (or similar) on every docker build step to up the default memory limit.
 - Make sure to disable file indexing on the drive used for docker data and DISABLE any anti-virus / windows defender it will hugely slow you down.  The build script will try to notify you if it notices defender is doing real time monitoring.
 - Space is massively hogged and it is super slow with large numbers of small files and large files are rewritten 3 extra times when a container is committed.  To avoid this we will remove repos/etc used during a build step before it finishes to speed things up. 
 - windowsfilter behind the scenes horridness: First you work in the vhd file, so all changes made while it is building or you are running it happen in the VHD.	Second after commit / build step finishes the container will exit. Docker will not return until it fully commits this build step (but the container will NOT show running). Docker starts the diff with the VHD and copies all the files for that layer to docker_data\tmp\random_id\.  Oddly it actually seems to create one random tmp random id folders with duplicate data from the VM, then it reads each file in this tmp folder writing it to another docker-data\tmp\random_id\ folder.  It slowly deletes from one of them once it finishes writing the second.   Then it makes another copy to the docker_data\windowsfilter\final_id permanent folder then removes the temp folder and the original VHD.   I am not sure why all the copying.  This can take A LONG time (hours on a 7200 rpm drive), the only way to know if this is going on is watch your storage. If docker is writing then its doing it.  Use procmon.exe if it is reading from a VHD writing to a tmp folder then its step 1.  If it is reading from one tmp folder and writing to another tmp folder that is step 2.  If it is reading from a tmp folder and writing to a windowsfilter sub folder then it is on the final step 3.  

README.md

@@ -13,6 +13,8 @@ if (Test-Path ./versions.ps1 -PathType Leaf){
 }
 Set-StrictMode -version latest;
 $ErrorActionPreference = "Stop";
+
+$global:PERF_FILE = Join-Path $WorkingDir "perf.log";
 if ((Get-MpPreference).DisableRealtimeMonitoring -eq $false){
 	Write-Host Warning, windows defender is enabled it will slow things down. -Foreground Red 
 }
@@ -42,31 +44,20 @@ if (! $NoMemoryWarn){
 
 echo packages_cefsharp.zip | out .dockerignore
 echo binaries.zip | out .dockerignore -Append
-if (! (Test-Path ./binaries.zip -PathType Leaf) -or $NoSkip){
-	TimerNow("Starting");
-    RunProc -proc "docker" -opts "pull $VAR_BASE_DOCKER_FILE";
-    TimerNow("Pull base file");
-	RunProc -proc "docker" -opts "build --build-arg BASE_DOCKER_FILE=`"$VAR_BASE_DOCKER_FILE`" -f Dockerfile_vs -t vs ."
-	TimerNow("VSBuild");
-	RunProc -proc "docker" -opts "build --build-arg DUAL_BUILD=`"$VAR_DUAL_BUILD`" --build-arg GN_DEFINES=`"$VAR_GN_DEFINES`" --build-arg GYP_DEFINES=`"$VAR_GYP_DEFINES`" --build-arg CHROME_BRANCH=`"$VAR_CHROME_BRANCH`" -f Dockerfile_cef -t cef ."
-	TimerNow("CEF Build");
-	docker rm cef;
-    Start-Sleep -s 3; #sometimes we are too fast, file in use error
-	RunProc -proc "docker" -opts "run --name cef cef cmd /C echo CopyVer"
-	RunProc -proc "docker" -opts "cp cef:/binaries.zip ."
-	TimerNow("Copy CEF Binary Local (1GB+)");
-	echo packages_cefsharp.zip  | out .dockerignore
-	RunProc -proc "docker" -opts "build -f Dockerfile_cef_compiled -t cef_compiled ."
-	TimerNow("Add CEF package to clean cef_compiled image");
-	echo binaries.zip | out .dockerignore -Append
-}else{
-		Write-Host binaries.zip already exists, skipping initial vs, cef, and cef_compiled build. Use flag NoSkip to avoid this. -ForegroundColor Red
-}
+
+TimerNow("Starting");
+RunProc -proc "docker" -opts "pull $VAR_BASE_DOCKER_FILE";
+TimerNow("Pull base file");
+RunProc -proc "docker" -opts "build --build-arg BASE_DOCKER_FILE=`"$VAR_BASE_DOCKER_FILE`" -f Dockerfile_vs -t vs ."
+TimerNow("VSBuild");
+RunProc -proc "docker" -opts "build --build-arg DUAL_BUILD=`"$VAR_DUAL_BUILD`" --build-arg GN_DEFINES=`"$VAR_GN_DEFINES`" --build-arg GYP_DEFINES=`"$VAR_GYP_DEFINES`" --build-arg CHROME_BRANCH=`"$VAR_CHROME_BRANCH`" -f Dockerfile_cef -t cef ."
+TimerNow("CEF Build");
 RunProc -proc "docker" -opts "build -f Dockerfile_cef_binary -t cef_binary ."
 TimerNow("CEF Binary compile");
 RunProc -proc "docker" -opts "build --build-arg CEFSHARP_BRANCH=`"$VAR_CEFSHARP_BRANCH`" --build-arg CEFSHARP_VERSION=`"$VAR_CEFSHARP_VERSION`" --build-arg CEF_VERSION_STR=`"$VAR_CEF_VERSION_STR`" --build-arg CHROME_BRANCH=`"$VAR_CHROME_BRANCH`" -f Dockerfile_cefsharp -t cefsharp ."
 TimerNow("CEFSharp compile");
 docker rm cefsharp;
 Start-Sleep -s 3; #sometimes we are too fast, file in use error
 RunProc -proc "docker" -opts "run --name cefsharp cefsharp cmd /C echo CopyVer"
-RunProc -proc "docker" -opts "cp cefsharp:/packages_cefsharp.zip ."
+RunProc -proc "docker" -opts "cp cefsharp:/packages_cefsharp.zip ."
+TimerNow("CEFSharp copy files locally");

build.ps1

@@ -64,6 +64,7 @@ if (@(dir -Filter "cef_binary_3.*_windows32.zip" "c:/code/chromium/src/cef/binar
 if (@(dir -Filter "cef_binary_3.*_windows64.zip" c:/code/chromium/src/cef/binary_distrib/).Count -ne 1){
 	throw "Not able to find win64 file as expected";
 }
-Compress-Archive -Path c:/code/chromium/src/cef/binary_distrib/*.zip -CompressionLevel Fastest -DestinationPath C:/binaries;
+mkdir c:/code/binaries;
+copy-item c:/code/chromium/src/cef/binary_distrib/*.zip -destination  C:/code/binaries;
 Set-Location -Path /;
 Remove-Item -Recurse -Force c:/code/chromium;

cef_build.ps1

@@ -59,10 +59,15 @@ function out {
 Function confirm($str){
 	return $PSCmdlet.ShouldContinue($str, "");
 }
-$last_time = Get-Date;
+$global:last_time = Get-Date;
+$global:PERF_FILE="";
 Function TimerNow($name){
 	$now = Get-Date;
-	$diff = ($now -  $last_time).TotalSeconds.ToString("0.0");
-	Write-Host $name took $diff  -ForegroundColor Green;
-	$last_time = $now;
+	$diff = ($now -  $global:last_time).TotalSeconds.ToString("0.0");
+	$str = "$(Get-Date) $name took $diff";
+	Write-Host $str  -ForegroundColor Green;
+	if ($PERF_FILE -ne ""){
+		$str | out -FilePath $PERF_FILE -Append;
+	}
+	$global:last_time = $now;
 }

functions.ps1

@@ -5,4 +5,10 @@ $VAR_BASE_DOCKER_FILE="microsoft/dotnet-framework:4.7.1-windowsservercore"; #mic
 $VAR_DUAL_BUILD="0"; #set to 1 to build x86 and x64 together, mainly to speed up linking which is single threaded, note may need excess ram.
 $VAR_GN_DEFINES="";
 $VAR_GYP_DEFINES=""; #use_jumbo_build=true see http://magpcss.org/ceforum/viewtopic.php?p=37293 about enabling this if you are doing proprietary_codecs as well, note this does not seem to actually cause a problem however in the builds we tested.
-$VAR_CEF_VERSION_STR="auto"; #can set to "3.3239.1723" or similar if you have multiple binaries that Docker_cefsharp might find
+$VAR_CEF_VERSION_STR="auto"; #can set to "3.3239.1723" or similar if you have multiple binaries that Docker_cefsharp might find
+
+if ($false){ #Sample 65 overrides
+	$VAR_CHROME_BRANCH="3325";
+	$VAR_CEFSHARP_VERSION="65.0.90";
+	$VAR_CEFSHARP_BRANCH="master";
+}