From 3df144f098e4f4e6f58a793ee8bba8cdf7f8d4a0 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Thu, 19 Dec 2024 19:25:54 +0000 Subject: [PATCH 01/25] docs(development.md): moves optional command addendums after LLVM details - Makes the "in-tree" and "out-of-tree" variants adjacent to each other --- docs/development.md | 33 +++++++++++++++++---------------- 1 file changed, 17 insertions(+), 16 deletions(-) diff --git a/docs/development.md b/docs/development.md index dbe11e70645e..d7d1df9f9079 100644 --- a/docs/development.md +++ b/docs/development.md @@ -150,22 +150,6 @@ cmake -GNinja -Bbuild \ externals/llvm-project/llvm ``` -##### Options to enable MLIR debugging - -* Enabling `--debug` and `--debug-only` flags (see [MLIR docs](https://mlir.llvm.org/getting_started/Debugging/)) for the `torch-mlir-opt` tool -```shell - -DCMAKE_BUILD_TYPE=RelWithDebInfo \ # or =Debug - -DLLVM_ENABLE_ASSERTIONS=ON \ -``` - -##### Options to run end-to-end tests - -Running the end-to-end execution tests locally requires enabling the native PyTorch extension features and the JIT IR importer, which depends on the -former and defaults to `ON` if not changed: -```shell - -DTORCH_MLIR_ENABLE_PYTORCH_EXTENSIONS=ON \ - -DTORCH_MLIR_ENABLE_JIT_IR_IMPORTER=ON \ -``` ##### ...with LLVM "out-of-tree" @@ -184,6 +168,23 @@ The same QoL CMake flags can be used to enable clang, ccache, and lld. Be sure t Be aware that the installed version of LLVM needs in general to match the committed version in `externals/llvm-project`. Using a different version may or may not work. +##### Options to enable MLIR debugging + +* Enabling `--debug` and `--debug-only` flags (see [MLIR docs](https://mlir.llvm.org/getting_started/Debugging/)) for the `torch-mlir-opt` tool +```shell + -DCMAKE_BUILD_TYPE=RelWithDebInfo \ # or =Debug + -DLLVM_ENABLE_ASSERTIONS=ON \ +``` + +##### Options to run end-to-end tests + +Running the end-to-end execution tests locally requires enabling the native PyTorch extension features and the JIT IR importer, which depends on the +former and defaults to `ON` if not changed: +```shell + -DTORCH_MLIR_ENABLE_PYTORCH_EXTENSIONS=ON \ + -DTORCH_MLIR_ENABLE_JIT_IR_IMPORTER=ON \ +``` + #### Initiate Build After either cmake run (in-tree/out-of-tree), use one of the following commands to build the project: From 9da0ec0b5eaa81044b3dcb7a695e6aae49682372 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Thu, 19 Dec 2024 22:57:26 +0000 Subject: [PATCH 02/25] docs(development.md): tweaks headers to communicate mutual exclusivity of "in-tree" vs "out-of-tree" --- docs/development.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/docs/development.md b/docs/development.md index d7d1df9f9079..d2d2009e0bcb 100644 --- a/docs/development.md +++ b/docs/development.md @@ -95,13 +95,15 @@ sudo apt install clang ccache lld #### Configure for Building... +##### ...with LLVM... + Two setups are possible to build: in-tree and out-of-tree. The in-tree setup is the most straightforward, as it will build LLVM dependencies as well. -##### ...with LLVM "in-tree" using... +###### ..."in-tree" using... The following commands generate configuration files to build the project *in-tree*, that is, using llvm/llvm-project as the main build. This will build LLVM as well as torch-mlir and its subprojects. -###### ...Base + Optimization Options +####### ...Base + Optimization Options This will build `libtorch` / `PyTorch` wheels from source and requires [the enablement mentioned earlier](#optional-enable-quicker-builds). If you encounter issues when you run this, try the [simplified build command](#base-options) instead. ```shell @@ -134,7 +136,7 @@ cmake -GNinja -Bbuild \ -DLIBTORCH_VARIANT=shared ``` -###### ...Base Options +####### ...Base Options If you're running into issues with the above build command, consider using the following: @@ -150,8 +152,7 @@ cmake -GNinja -Bbuild \ externals/llvm-project/llvm ``` - -##### ...with LLVM "out-of-tree" +###### ..."out-of-tree" If you have built llvm-project separately in the directory `$LLVM_INSTALL_DIR`, you can also build the project *out-of-tree* using the following command as template: ```shell From 93abcadc00f1f6cc8a717f97f6b76e341665e4f9 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Wed, 18 Dec 2024 17:59:24 +0000 Subject: [PATCH 03/25] docs(development.md): adds missing DLLVM_ENABLE_ASSERTIONS under "Simplified Build" section --- docs/development.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/development.md b/docs/development.md index d2d2009e0bcb..a8c0f65a71e2 100644 --- a/docs/development.md +++ b/docs/development.md @@ -143,6 +143,7 @@ If you're running into issues with the above build command, consider using the f ```shell cmake -GNinja -Bbuild \ -DCMAKE_BUILD_TYPE=Release \ + -DLLVM_ENABLE_ASSERTIONS=ON \ -DPython3_FIND_VIRTUALENV=ONLY \ -DLLVM_ENABLE_PROJECTS=mlir \ -DLLVM_EXTERNAL_PROJECTS="torch-mlir" \ @@ -158,6 +159,7 @@ If you have built llvm-project separately in the directory `$LLVM_INSTALL_DIR`, ```shell cmake -GNinja -Bbuild \ -DCMAKE_BUILD_TYPE=Release \ + -DLLVM_ENABLE_ASSERTIONS=ON \ -DPython3_FIND_VIRTUALENV=ONLY \ -DMLIR_DIR="$LLVM_INSTALL_DIR/lib/cmake/mlir/" \ -DLLVM_DIR="$LLVM_INSTALL_DIR/lib/cmake/llvm/" \ From e04bdc0ffbdbaf723fe221c129ab0009c6da6900 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Thu, 19 Dec 2024 15:45:11 +0000 Subject: [PATCH 04/25] docs(development.md): rearranges cmake configuration options - emphasizes the common options between "in-tree extended", "in-of-tree base", and "out-of-tree" --- docs/development.md | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/docs/development.md b/docs/development.md index a8c0f65a71e2..7ed8b73f652b 100644 --- a/docs/development.md +++ b/docs/development.md @@ -108,15 +108,16 @@ This will build `libtorch` / `PyTorch` wheels from source and requires [the enab ```shell cmake -GNinja -Bbuild \ - externals/llvm-project/llvm \ -DCMAKE_BUILD_TYPE=Release \ -DLLVM_ENABLE_ASSERTIONS=ON \ -DPython3_FIND_VIRTUALENV=ONLY \ + -DMLIR_ENABLE_BINDINGS_PYTHON=ON \ + -DLLVM_TARGETS_TO_BUILD=host \ + `# For building LLVM "in-tree"` \ + externals/llvm-project/llvm \ -DLLVM_ENABLE_PROJECTS=mlir \ -DLLVM_EXTERNAL_PROJECTS="torch-mlir" \ -DLLVM_EXTERNAL_TORCH_MLIR_SOURCE_DIR="$PWD" \ - -DMLIR_ENABLE_BINDINGS_PYTHON=ON \ - -DLLVM_TARGETS_TO_BUILD=host \ `# use clang`\ -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \ `# use ccache to cache build results` \ @@ -145,12 +146,13 @@ cmake -GNinja -Bbuild \ -DCMAKE_BUILD_TYPE=Release \ -DLLVM_ENABLE_ASSERTIONS=ON \ -DPython3_FIND_VIRTUALENV=ONLY \ - -DLLVM_ENABLE_PROJECTS=mlir \ - -DLLVM_EXTERNAL_PROJECTS="torch-mlir" \ - -DLLVM_EXTERNAL_TORCH_MLIR_SOURCE_DIR="$PWD" \ -DMLIR_ENABLE_BINDINGS_PYTHON=ON \ -DLLVM_TARGETS_TO_BUILD=host \ - externals/llvm-project/llvm + `# For building LLVM "in-tree"` \ + externals/llvm-project/llvm \ + -DLLVM_ENABLE_PROJECTS=mlir \ + -DLLVM_EXTERNAL_PROJECTS="torch-mlir" \ + -DLLVM_EXTERNAL_TORCH_MLIR_SOURCE_DIR="$PWD" ``` ###### ..."out-of-tree" @@ -161,10 +163,11 @@ cmake -GNinja -Bbuild \ -DCMAKE_BUILD_TYPE=Release \ -DLLVM_ENABLE_ASSERTIONS=ON \ -DPython3_FIND_VIRTUALENV=ONLY \ - -DMLIR_DIR="$LLVM_INSTALL_DIR/lib/cmake/mlir/" \ - -DLLVM_DIR="$LLVM_INSTALL_DIR/lib/cmake/llvm/" \ -DMLIR_ENABLE_BINDINGS_PYTHON=ON \ -DLLVM_TARGETS_TO_BUILD=host \ + `# For building LLVM "out-of-tree"` \ + -DMLIR_DIR="$LLVM_INSTALL_DIR/lib/cmake/mlir/" \ + -DLLVM_DIR="$LLVM_INSTALL_DIR/lib/cmake/llvm/" . ``` The same QoL CMake flags can be used to enable clang, ccache, and lld. Be sure to have built LLVM with `-DLLVM_ENABLE_PROJECTS=mlir`. From 5002543bc33247f82d26dff0d4d970cd475e58ef Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Thu, 19 Dec 2024 18:59:01 +0000 Subject: [PATCH 05/25] docs(development.md): clarifies rationale for configuring "in-tree" build using base options --- docs/development.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/development.md b/docs/development.md index 7ed8b73f652b..83d98b09837e 100644 --- a/docs/development.md +++ b/docs/development.md @@ -139,7 +139,7 @@ cmake -GNinja -Bbuild \ ####### ...Base Options -If you're running into issues with the above build command, consider using the following: +If you don't anticipate needing to frequently rebuild LLVM "in-tree", run: ```shell cmake -GNinja -Bbuild \ From f020a57df399c8f0b33e24f6e564d7520ce266f8 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Thu, 19 Dec 2024 20:35:38 +0000 Subject: [PATCH 06/25] docs(development.md): moves "base options" to be first "in-tree" command --- docs/development.md | 36 ++++++++++++++++++------------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/docs/development.md b/docs/development.md index 83d98b09837e..35fa1d2b81c4 100644 --- a/docs/development.md +++ b/docs/development.md @@ -103,6 +103,24 @@ Two setups are possible to build: in-tree and out-of-tree. The in-tree setup is The following commands generate configuration files to build the project *in-tree*, that is, using llvm/llvm-project as the main build. This will build LLVM as well as torch-mlir and its subprojects. +####### ...Base Options + +If you don't anticipate needing to frequently rebuild LLVM "in-tree", run: + +```shell +cmake -GNinja -Bbuild \ + -DCMAKE_BUILD_TYPE=Release \ + -DLLVM_ENABLE_ASSERTIONS=ON \ + -DPython3_FIND_VIRTUALENV=ONLY \ + -DMLIR_ENABLE_BINDINGS_PYTHON=ON \ + -DLLVM_TARGETS_TO_BUILD=host \ + `# For building LLVM "in-tree"` \ + externals/llvm-project/llvm \ + -DLLVM_ENABLE_PROJECTS=mlir \ + -DLLVM_EXTERNAL_PROJECTS="torch-mlir" \ + -DLLVM_EXTERNAL_TORCH_MLIR_SOURCE_DIR="$PWD" +``` + ####### ...Base + Optimization Options This will build `libtorch` / `PyTorch` wheels from source and requires [the enablement mentioned earlier](#optional-enable-quicker-builds). If you encounter issues when you run this, try the [simplified build command](#base-options) instead. @@ -137,24 +155,6 @@ cmake -GNinja -Bbuild \ -DLIBTORCH_VARIANT=shared ``` -####### ...Base Options - -If you don't anticipate needing to frequently rebuild LLVM "in-tree", run: - -```shell -cmake -GNinja -Bbuild \ - -DCMAKE_BUILD_TYPE=Release \ - -DLLVM_ENABLE_ASSERTIONS=ON \ - -DPython3_FIND_VIRTUALENV=ONLY \ - -DMLIR_ENABLE_BINDINGS_PYTHON=ON \ - -DLLVM_TARGETS_TO_BUILD=host \ - `# For building LLVM "in-tree"` \ - externals/llvm-project/llvm \ - -DLLVM_ENABLE_PROJECTS=mlir \ - -DLLVM_EXTERNAL_PROJECTS="torch-mlir" \ - -DLLVM_EXTERNAL_TORCH_MLIR_SOURCE_DIR="$PWD" -``` - ###### ..."out-of-tree" If you have built llvm-project separately in the directory `$LLVM_INSTALL_DIR`, you can also build the project *out-of-tree* using the following command as template: From a24d47b73512a4d61a514cf301c60429eda3a425 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Fri, 20 Dec 2024 19:28:31 +0000 Subject: [PATCH 07/25] docs(development.md): clarifies "configure" headers - tweaks existing headers to communicate command construction rather than immediate execution - adds new header to indicate when the command should actually be executed --- docs/development.md | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/docs/development.md b/docs/development.md index 35fa1d2b81c4..2d403d47bcaf 100644 --- a/docs/development.md +++ b/docs/development.md @@ -93,9 +93,9 @@ sudo apt install clang ccache lld 1. Set up Developer PowerShell [for Visual Studio](https://learn.microsoft.com/en-us/visualstudio/ide/reference/command-prompt-powershell?view=vs-2022#start-in-visual-studio) 1. Ensure that the compiler and linker binaries are in the `PATH` variable. -#### Configure for Building... +#### Configure for Building -##### ...with LLVM... +##### Choose command LLVM built... Two setups are possible to build: in-tree and out-of-tree. The in-tree setup is the most straightforward, as it will build LLVM dependencies as well. @@ -105,7 +105,7 @@ The following commands generate configuration files to build the project *in-tre ####### ...Base Options -If you don't anticipate needing to frequently rebuild LLVM "in-tree", run: +If you don't anticipate needing to frequently rebuild LLVM "in-tree", append: ```shell cmake -GNinja -Bbuild \ @@ -174,23 +174,29 @@ The same QoL CMake flags can be used to enable clang, ccache, and lld. Be sure t Be aware that the installed version of LLVM needs in general to match the committed version in `externals/llvm-project`. Using a different version may or may not work. -##### Options to enable MLIR debugging +##### (Optional) Append options that enable MLIR debugging * Enabling `--debug` and `--debug-only` flags (see [MLIR docs](https://mlir.llvm.org/getting_started/Debugging/)) for the `torch-mlir-opt` tool ```shell + \ -DCMAKE_BUILD_TYPE=RelWithDebInfo \ # or =Debug - -DLLVM_ENABLE_ASSERTIONS=ON \ + -DLLVM_ENABLE_ASSERTIONS=ON ``` -##### Options to run end-to-end tests +##### (Optional) Append options that enable end-to-end tests Running the end-to-end execution tests locally requires enabling the native PyTorch extension features and the JIT IR importer, which depends on the former and defaults to `ON` if not changed: ```shell + \ -DTORCH_MLIR_ENABLE_PYTORCH_EXTENSIONS=ON \ - -DTORCH_MLIR_ENABLE_JIT_IR_IMPORTER=ON \ + -DTORCH_MLIR_ENABLE_JIT_IR_IMPORTER=ON ``` +##### Run assembled command + +Execute the command once you've appended the options pertaining to your workflow. + #### Initiate Build After either cmake run (in-tree/out-of-tree), use one of the following commands to build the project: From 7fa51ccb924d44514d3d95074e96e35f74cf4139 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Fri, 20 Dec 2024 21:07:31 +0000 Subject: [PATCH 08/25] docs(development.md): extracts common CMake options into first subheader - was repeated 3 times! --- docs/development.md | 38 +++++++++++++++++--------------------- 1 file changed, 17 insertions(+), 21 deletions(-) diff --git a/docs/development.md b/docs/development.md index 2d403d47bcaf..e1a5d38915bf 100644 --- a/docs/development.md +++ b/docs/development.md @@ -95,7 +95,18 @@ sudo apt install clang ccache lld #### Configure for Building -##### Choose command LLVM built... +##### Append (not "run") command with "common" options + +```shell +cmake -GNinja -Bbuild \ + -DCMAKE_BUILD_TYPE=Release \ + -DLLVM_ENABLE_ASSERTIONS=ON \ + -DPython3_FIND_VIRTUALENV=ONLY \ + -DMLIR_ENABLE_BINDINGS_PYTHON=ON \ + -DLLVM_TARGETS_TO_BUILD=host +``` + +##### Choose options LLVM built... Two setups are possible to build: in-tree and out-of-tree. The in-tree setup is the most straightforward, as it will build LLVM dependencies as well. @@ -108,12 +119,7 @@ The following commands generate configuration files to build the project *in-tre If you don't anticipate needing to frequently rebuild LLVM "in-tree", append: ```shell -cmake -GNinja -Bbuild \ - -DCMAKE_BUILD_TYPE=Release \ - -DLLVM_ENABLE_ASSERTIONS=ON \ - -DPython3_FIND_VIRTUALENV=ONLY \ - -DMLIR_ENABLE_BINDINGS_PYTHON=ON \ - -DLLVM_TARGETS_TO_BUILD=host \ + \ `# For building LLVM "in-tree"` \ externals/llvm-project/llvm \ -DLLVM_ENABLE_PROJECTS=mlir \ @@ -122,15 +128,10 @@ cmake -GNinja -Bbuild \ ``` ####### ...Base + Optimization Options -This will build `libtorch` / `PyTorch` wheels from source and requires [the enablement mentioned earlier](#optional-enable-quicker-builds). If you encounter issues when you run this, try the [simplified build command](#base-options) instead. +This will build `libtorch` / `PyTorch` wheels from source and requires [the enablement mentioned earlier](#optional-enable-quicker-builds). If these options cause issues, use just the [base options](#base-options) for now. ```shell -cmake -GNinja -Bbuild \ - -DCMAKE_BUILD_TYPE=Release \ - -DLLVM_ENABLE_ASSERTIONS=ON \ - -DPython3_FIND_VIRTUALENV=ONLY \ - -DMLIR_ENABLE_BINDINGS_PYTHON=ON \ - -DLLVM_TARGETS_TO_BUILD=host \ + \ `# For building LLVM "in-tree"` \ externals/llvm-project/llvm \ -DLLVM_ENABLE_PROJECTS=mlir \ @@ -157,14 +158,9 @@ cmake -GNinja -Bbuild \ ###### ..."out-of-tree" -If you have built llvm-project separately in the directory `$LLVM_INSTALL_DIR`, you can also build the project *out-of-tree* using the following command as template: +If you have built llvm-project separately in the directory `$LLVM_INSTALL_DIR`, you can also build the project *out-of-tree* using the following options as a template: ```shell -cmake -GNinja -Bbuild \ - -DCMAKE_BUILD_TYPE=Release \ - -DLLVM_ENABLE_ASSERTIONS=ON \ - -DPython3_FIND_VIRTUALENV=ONLY \ - -DMLIR_ENABLE_BINDINGS_PYTHON=ON \ - -DLLVM_TARGETS_TO_BUILD=host \ + \ `# For building LLVM "out-of-tree"` \ -DMLIR_DIR="$LLVM_INSTALL_DIR/lib/cmake/mlir/" \ -DLLVM_DIR="$LLVM_INSTALL_DIR/lib/cmake/llvm/" From 993dc9fa6833c250be1be552017b72c92ab2cfeb Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Fri, 20 Dec 2024 21:22:58 +0000 Subject: [PATCH 09/25] docs(development.md): extracts common "in-tree" CMake options into first subheader --- docs/development.md | 21 +++++++-------------- 1 file changed, 7 insertions(+), 14 deletions(-) diff --git a/docs/development.md b/docs/development.md index e1a5d38915bf..362d261fc926 100644 --- a/docs/development.md +++ b/docs/development.md @@ -110,13 +110,7 @@ cmake -GNinja -Bbuild \ Two setups are possible to build: in-tree and out-of-tree. The in-tree setup is the most straightforward, as it will build LLVM dependencies as well. -###### ..."in-tree" using... - -The following commands generate configuration files to build the project *in-tree*, that is, using llvm/llvm-project as the main build. This will build LLVM as well as torch-mlir and its subprojects. - -####### ...Base Options - -If you don't anticipate needing to frequently rebuild LLVM "in-tree", append: +###### ..."in-tree" ```shell \ @@ -127,16 +121,15 @@ If you don't anticipate needing to frequently rebuild LLVM "in-tree", append: -DLLVM_EXTERNAL_TORCH_MLIR_SOURCE_DIR="$PWD" ``` -####### ...Base + Optimization Options -This will build `libtorch` / `PyTorch` wheels from source and requires [the enablement mentioned earlier](#optional-enable-quicker-builds). If these options cause issues, use just the [base options](#base-options) for now. +- NOTE: uses external/llvm-project/llvm as the main build, so LLVM will be built in additional to torch-mlir and its sub-projects. + +####### (Optional) Append options to enable build optimizations +This will build `libtorch` / `PyTorch` wheels from source and requires [the enablement mentioned earlier](#optional-enable-quicker-builds). If these options cause issues, just skip them for now. + +If you do anticipate needing to frequently rebuild LLVM "in-tree", append: ```shell \ - `# For building LLVM "in-tree"` \ - externals/llvm-project/llvm \ - -DLLVM_ENABLE_PROJECTS=mlir \ - -DLLVM_EXTERNAL_PROJECTS="torch-mlir" \ - -DLLVM_EXTERNAL_TORCH_MLIR_SOURCE_DIR="$PWD" \ `# use clang`\ -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \ `# use ccache to cache build results` \ From e769344e9b521225c1be581d99bcd3af3c76f803 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Fri, 20 Dec 2024 21:29:28 +0000 Subject: [PATCH 10/25] docs(development.md): extracts build optimization options into h5 header --- docs/development.md | 32 ++++++++++++++++---------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/docs/development.md b/docs/development.md index 362d261fc926..27899cb5044b 100644 --- a/docs/development.md +++ b/docs/development.md @@ -123,10 +123,24 @@ Two setups are possible to build: in-tree and out-of-tree. The in-tree setup is - NOTE: uses external/llvm-project/llvm as the main build, so LLVM will be built in additional to torch-mlir and its sub-projects. -####### (Optional) Append options to enable build optimizations +###### ..."out-of-tree" + +If you have built llvm-project separately in the directory `$LLVM_INSTALL_DIR`, you can also build the project *out-of-tree* using the following options as a template: +```shell + \ + `# For building LLVM "out-of-tree"` \ + -DMLIR_DIR="$LLVM_INSTALL_DIR/lib/cmake/mlir/" \ + -DLLVM_DIR="$LLVM_INSTALL_DIR/lib/cmake/llvm/" + . +``` +The same QoL CMake flags can be used to enable clang, ccache, and lld. Be sure to have built LLVM with `-DLLVM_ENABLE_PROJECTS=mlir`. + +Be aware that the installed version of LLVM needs in general to match the committed version in `externals/llvm-project`. Using a different version may or may not work. + +##### (Optional) Append options to enable build optimizations This will build `libtorch` / `PyTorch` wheels from source and requires [the enablement mentioned earlier](#optional-enable-quicker-builds). If these options cause issues, just skip them for now. -If you do anticipate needing to frequently rebuild LLVM "in-tree", append: +If you anticipate needing to frequently rebuild LLVM, append: ```shell \ @@ -149,20 +163,6 @@ If you do anticipate needing to frequently rebuild LLVM "in-tree", append: -DLIBTORCH_VARIANT=shared ``` -###### ..."out-of-tree" - -If you have built llvm-project separately in the directory `$LLVM_INSTALL_DIR`, you can also build the project *out-of-tree* using the following options as a template: -```shell - \ - `# For building LLVM "out-of-tree"` \ - -DMLIR_DIR="$LLVM_INSTALL_DIR/lib/cmake/mlir/" \ - -DLLVM_DIR="$LLVM_INSTALL_DIR/lib/cmake/llvm/" - . -``` -The same QoL CMake flags can be used to enable clang, ccache, and lld. Be sure to have built LLVM with `-DLLVM_ENABLE_PROJECTS=mlir`. - -Be aware that the installed version of LLVM needs in general to match the committed version in `externals/llvm-project`. Using a different version may or may not work. - ##### (Optional) Append options that enable MLIR debugging * Enabling `--debug` and `--debug-only` flags (see [MLIR docs](https://mlir.llvm.org/getting_started/Debugging/)) for the `torch-mlir-opt` tool From 6fb7e5f66403e8952ac0b3d184bd30dc531ec6d9 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Fri, 20 Dec 2024 21:56:20 +0000 Subject: [PATCH 11/25] docs(development.md): simplifies section about CMake options for LLVM --- docs/development.md | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/docs/development.md b/docs/development.md index 27899cb5044b..b89026d258f0 100644 --- a/docs/development.md +++ b/docs/development.md @@ -106,11 +106,9 @@ cmake -GNinja -Bbuild \ -DLLVM_TARGETS_TO_BUILD=host ``` -##### Choose options LLVM built... +##### Specify LLVM options -Two setups are possible to build: in-tree and out-of-tree. The in-tree setup is the most straightforward, as it will build LLVM dependencies as well. - -###### ..."in-tree" +###### If building "in-tree", append: ```shell \ @@ -123,9 +121,8 @@ Two setups are possible to build: in-tree and out-of-tree. The in-tree setup is - NOTE: uses external/llvm-project/llvm as the main build, so LLVM will be built in additional to torch-mlir and its sub-projects. -###### ..."out-of-tree" +###### If using "out-of-tree" build, append: -If you have built llvm-project separately in the directory `$LLVM_INSTALL_DIR`, you can also build the project *out-of-tree* using the following options as a template: ```shell \ `# For building LLVM "out-of-tree"` \ From 0e8584af3628075f4a7e77ace178983cbea406e9 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Fri, 20 Dec 2024 22:10:28 +0000 Subject: [PATCH 12/25] docs(development.md): moves notes to bottom of build optimization section --- docs/development.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/development.md b/docs/development.md index b89026d258f0..974cb5150db1 100644 --- a/docs/development.md +++ b/docs/development.md @@ -135,7 +135,6 @@ The same QoL CMake flags can be used to enable clang, ccache, and lld. Be sure t Be aware that the installed version of LLVM needs in general to match the committed version in `externals/llvm-project`. Using a different version may or may not work. ##### (Optional) Append options to enable build optimizations -This will build `libtorch` / `PyTorch` wheels from source and requires [the enablement mentioned earlier](#optional-enable-quicker-builds). If these options cause issues, just skip them for now. If you anticipate needing to frequently rebuild LLVM, append: @@ -160,6 +159,8 @@ If you anticipate needing to frequently rebuild LLVM, append: -DLIBTORCH_VARIANT=shared ``` +This will build `libtorch` / `PyTorch` wheels from source and requires [the enablement mentioned earlier](#optional-enable-quicker-builds). If these options cause issues, just skip them for now. + ##### (Optional) Append options that enable MLIR debugging * Enabling `--debug` and `--debug-only` flags (see [MLIR docs](https://mlir.llvm.org/getting_started/Debugging/)) for the `torch-mlir-opt` tool From cae34b54b4eee7d4dca90eb8940efd442081c550 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Fri, 20 Dec 2024 22:42:17 +0000 Subject: [PATCH 13/25] docs(development.md): moves notes to bottom of CMake end-to-end tests section --- docs/development.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/development.md b/docs/development.md index 974cb5150db1..d12c66a09df2 100644 --- a/docs/development.md +++ b/docs/development.md @@ -172,14 +172,15 @@ This will build `libtorch` / `PyTorch` wheels from source and requires [the enab ##### (Optional) Append options that enable end-to-end tests -Running the end-to-end execution tests locally requires enabling the native PyTorch extension features and the JIT IR importer, which depends on the -former and defaults to `ON` if not changed: ```shell \ -DTORCH_MLIR_ENABLE_PYTORCH_EXTENSIONS=ON \ -DTORCH_MLIR_ENABLE_JIT_IR_IMPORTER=ON ``` +Running the end-to-end execution tests locally requires enabling the native PyTorch extension features and the JIT IR importer, which depends on the +former and defaults to `ON` if not changed: + ##### Run assembled command Execute the command once you've appended the options pertaining to your workflow. From 9b582d6921be4a9ac5bf94ec77bb178d05390204 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Wed, 18 Dec 2024 18:02:42 +0000 Subject: [PATCH 14/25] docs(development.md): updates DCMAKE_BUILD_TYPE to "RelWithDebInfo" - the use-case for this doc leans more toward "development" rather than "release" --- docs/development.md | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/docs/development.md b/docs/development.md index d12c66a09df2..11011146c401 100644 --- a/docs/development.md +++ b/docs/development.md @@ -99,13 +99,16 @@ sudo apt install clang ccache lld ```shell cmake -GNinja -Bbuild \ - -DCMAKE_BUILD_TYPE=Release \ + `# Enables "--debug" and "--debug-only" flags for the "torch-mlir-opt" tool` \ + -DCMAKE_BUILD_TYPE=RelWithDebInfo \ -DLLVM_ENABLE_ASSERTIONS=ON \ -DPython3_FIND_VIRTUALENV=ONLY \ -DMLIR_ENABLE_BINDINGS_PYTHON=ON \ -DLLVM_TARGETS_TO_BUILD=host ``` +[About MLIR debugging](https://mlir.llvm.org/getting_started/Debugging/) + ##### Specify LLVM options ###### If building "in-tree", append: @@ -161,15 +164,6 @@ If you anticipate needing to frequently rebuild LLVM, append: This will build `libtorch` / `PyTorch` wheels from source and requires [the enablement mentioned earlier](#optional-enable-quicker-builds). If these options cause issues, just skip them for now. -##### (Optional) Append options that enable MLIR debugging - -* Enabling `--debug` and `--debug-only` flags (see [MLIR docs](https://mlir.llvm.org/getting_started/Debugging/)) for the `torch-mlir-opt` tool -```shell - \ - -DCMAKE_BUILD_TYPE=RelWithDebInfo \ # or =Debug - -DLLVM_ENABLE_ASSERTIONS=ON -``` - ##### (Optional) Append options that enable end-to-end tests ```shell From a036e37b63084c6bf2a07c7238374b2fb9bcd001 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Fri, 20 Dec 2024 22:17:23 +0000 Subject: [PATCH 15/25] docs(development.md): converts headers under CMake to numbered list - leverages numbering and indentation to reduce cognitive load --- docs/development.md | 143 ++++++++++++++++++++------------------------ 1 file changed, 66 insertions(+), 77 deletions(-) diff --git a/docs/development.md b/docs/development.md index 11011146c401..f1dc61a56859 100644 --- a/docs/development.md +++ b/docs/development.md @@ -95,89 +95,78 @@ sudo apt install clang ccache lld #### Configure for Building -##### Append (not "run") command with "common" options +1. Append (not "run") command with "common" options: -```shell -cmake -GNinja -Bbuild \ - `# Enables "--debug" and "--debug-only" flags for the "torch-mlir-opt" tool` \ - -DCMAKE_BUILD_TYPE=RelWithDebInfo \ - -DLLVM_ENABLE_ASSERTIONS=ON \ - -DPython3_FIND_VIRTUALENV=ONLY \ - -DMLIR_ENABLE_BINDINGS_PYTHON=ON \ - -DLLVM_TARGETS_TO_BUILD=host -``` - -[About MLIR debugging](https://mlir.llvm.org/getting_started/Debugging/) - -##### Specify LLVM options - -###### If building "in-tree", append: - -```shell - \ - `# For building LLVM "in-tree"` \ - externals/llvm-project/llvm \ - -DLLVM_ENABLE_PROJECTS=mlir \ - -DLLVM_EXTERNAL_PROJECTS="torch-mlir" \ - -DLLVM_EXTERNAL_TORCH_MLIR_SOURCE_DIR="$PWD" -``` - -- NOTE: uses external/llvm-project/llvm as the main build, so LLVM will be built in additional to torch-mlir and its sub-projects. - -###### If using "out-of-tree" build, append: - -```shell - \ - `# For building LLVM "out-of-tree"` \ - -DMLIR_DIR="$LLVM_INSTALL_DIR/lib/cmake/mlir/" \ - -DLLVM_DIR="$LLVM_INSTALL_DIR/lib/cmake/llvm/" - . -``` -The same QoL CMake flags can be used to enable clang, ccache, and lld. Be sure to have built LLVM with `-DLLVM_ENABLE_PROJECTS=mlir`. - -Be aware that the installed version of LLVM needs in general to match the committed version in `externals/llvm-project`. Using a different version may or may not work. - -##### (Optional) Append options to enable build optimizations - -If you anticipate needing to frequently rebuild LLVM, append: + ```shell + cmake -GNinja -Bbuild \ + `# Enables "--debug" and "--debug-only" flags for the "torch-mlir-opt" tool` \ + -DCMAKE_BUILD_TYPE=RelWithDebInfo \ + -DLLVM_ENABLE_ASSERTIONS=ON \ + -DPython3_FIND_VIRTUALENV=ONLY \ + -DMLIR_ENABLE_BINDINGS_PYTHON=ON \ + -DLLVM_TARGETS_TO_BUILD=host + ``` -```shell - \ - `# use clang`\ - -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \ - `# use ccache to cache build results` \ - -DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \ - `# use LLD to link in seconds, rather than minutes` \ - `# if using clang <= 13, replace --ld-path=ld.lld with -fuse-ld=lld` \ - -DCMAKE_EXE_LINKER_FLAGS_INIT="--ld-path=ld.lld" \ - -DCMAKE_MODULE_LINKER_FLAGS_INIT="--ld-path=ld.lld" \ - -DCMAKE_SHARED_LINKER_FLAGS_INIT="--ld-path=ld.lld" \ - `# Enabling libtorch binary cache instead of downloading the latest libtorch everytime.` \ - `# Testing against a mismatched version of libtorch may cause failures` \ - -DLIBTORCH_CACHE=ON \ - `# Enable an experimental path to build libtorch (and PyTorch wheels) from source,` \ - `# instead of downloading them` \ - -DLIBTORCH_SRC_BUILD=ON \ - `# Set the variant of libtorch to build / link against. (shared|static and optionally cxxabi11)` \ - -DLIBTORCH_VARIANT=shared -``` - -This will build `libtorch` / `PyTorch` wheels from source and requires [the enablement mentioned earlier](#optional-enable-quicker-builds). If these options cause issues, just skip them for now. - -##### (Optional) Append options that enable end-to-end tests + - [About MLIR debugging](https://mlir.llvm.org/getting_started/Debugging/) +1. Specify LLVM options + 1. **If building "in-tree"**, append: + + ```shell + \ + `# For building LLVM "in-tree"` \ + externals/llvm-project/llvm \ + -DLLVM_ENABLE_PROJECTS=mlir \ + -DLLVM_EXTERNAL_PROJECTS="torch-mlir" \ + -DLLVM_EXTERNAL_TORCH_MLIR_SOURCE_DIR="$PWD" + ``` + + - NOTE: uses external/llvm-project/llvm as the main build, so LLVM will be built in additional to torch-mlir and its sub-projects. + 1. **If using "out-of-tree" build**, append: + + ```shell + \ + `# For building LLVM "out-of-tree"` \ + -DMLIR_DIR="$LLVM_INSTALL_DIR/lib/cmake/mlir/" \ + -DLLVM_DIR="$LLVM_INSTALL_DIR/lib/cmake/llvm/" + ``` + + - Be sure to have built LLVM with `-DLLVM_ENABLE_PROJECTS=mlir`. + - Be aware that the installed version of LLVM needs in general to match the committed version in `externals/llvm-project`. Using a different version may or may not work. +1. **If you anticipate needing to frequently rebuild LLVM**, append: -```shell - \ - -DTORCH_MLIR_ENABLE_PYTORCH_EXTENSIONS=ON \ - -DTORCH_MLIR_ENABLE_JIT_IR_IMPORTER=ON -``` + ```shell + \ + `# use clang`\ + -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ \ + `# use ccache to cache build results` \ + -DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \ + `# use LLD to link in seconds, rather than minutes` \ + `# if using clang <= 13, replace --ld-path=ld.lld with -fuse-ld=lld` \ + -DCMAKE_EXE_LINKER_FLAGS_INIT="--ld-path=ld.lld" \ + -DCMAKE_MODULE_LINKER_FLAGS_INIT="--ld-path=ld.lld" \ + -DCMAKE_SHARED_LINKER_FLAGS_INIT="--ld-path=ld.lld" \ + `# Enabling libtorch binary cache instead of downloading the latest libtorch everytime.` \ + `# Testing against a mismatched version of libtorch may cause failures` \ + -DLIBTORCH_CACHE=ON \ + `# Enable an experimental path to build libtorch (and PyTorch wheels) from source,` \ + `# instead of downloading them` \ + -DLIBTORCH_SRC_BUILD=ON \ + `# Set the variant of libtorch to build / link against. (shared|static and optionally cxxabi11)` \ + -DLIBTORCH_VARIANT=shared + ``` -Running the end-to-end execution tests locally requires enabling the native PyTorch extension features and the JIT IR importer, which depends on the -former and defaults to `ON` if not changed: + - This will build `libtorch` / `PyTorch` wheels from source and requires [the enablement mentioned earlier](#optional-enable-quicker-builds). + - If these options cause issues, just skip them for now. +1. **If you need to enable local end-to-end tests**, append: -##### Run assembled command + ```shell + \ + -DTORCH_MLIR_ENABLE_PYTORCH_EXTENSIONS=ON \ + -DTORCH_MLIR_ENABLE_JIT_IR_IMPORTER=ON + ``` -Execute the command once you've appended the options pertaining to your workflow. + - NOTE: The JIT IR importer depends on the native PyTorch extension features and defaults to `ON` if not changed. +1. Run assembled command (once you've appended the options pertaining to your workflow) #### Initiate Build From 94cfac424bea9d9a0e0357b7aaf42c82660a73a1 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Tue, 28 Jan 2025 20:21:29 +0000 Subject: [PATCH 16/25] docs(development.md): combines common options with mandatory LLVM options --- docs/development.md | 34 ++++++++++++++++++---------------- 1 file changed, 18 insertions(+), 16 deletions(-) diff --git a/docs/development.md b/docs/development.md index f1dc61a56859..e2563ac54d61 100644 --- a/docs/development.md +++ b/docs/development.md @@ -95,23 +95,17 @@ sudo apt install clang ccache lld #### Configure for Building -1. Append (not "run") command with "common" options: - - ```shell - cmake -GNinja -Bbuild \ - `# Enables "--debug" and "--debug-only" flags for the "torch-mlir-opt" tool` \ - -DCMAKE_BUILD_TYPE=RelWithDebInfo \ - -DLLVM_ENABLE_ASSERTIONS=ON \ - -DPython3_FIND_VIRTUALENV=ONLY \ - -DMLIR_ENABLE_BINDINGS_PYTHON=ON \ - -DLLVM_TARGETS_TO_BUILD=host - ``` - - - [About MLIR debugging](https://mlir.llvm.org/getting_started/Debugging/) -1. Specify LLVM options - 1. **If building "in-tree"**, append: +1. Choose command with relevant LLVM options: + 1. **If building "in-tree"**, run/append: ```shell + cmake -GNinja -Bbuild \ + `# Enables "--debug" and "--debug-only" flags for the "torch-mlir-opt" tool` \ + -DCMAKE_BUILD_TYPE=RelWithDebInfo \ + -DLLVM_ENABLE_ASSERTIONS=ON \ + -DPython3_FIND_VIRTUALENV=ONLY \ + -DMLIR_ENABLE_BINDINGS_PYTHON=ON \ + -DLLVM_TARGETS_TO_BUILD=host \ \ `# For building LLVM "in-tree"` \ externals/llvm-project/llvm \ @@ -121,9 +115,16 @@ sudo apt install clang ccache lld ``` - NOTE: uses external/llvm-project/llvm as the main build, so LLVM will be built in additional to torch-mlir and its sub-projects. - 1. **If using "out-of-tree" build**, append: + 1. **If using "out-of-tree" build**, run/append: ```shell + cmake -GNinja -Bbuild \ + `# Enables "--debug" and "--debug-only" flags for the "torch-mlir-opt" tool` \ + -DCMAKE_BUILD_TYPE=RelWithDebInfo \ + -DLLVM_ENABLE_ASSERTIONS=ON \ + -DPython3_FIND_VIRTUALENV=ONLY \ + -DMLIR_ENABLE_BINDINGS_PYTHON=ON \ + -DLLVM_TARGETS_TO_BUILD=host \ \ `# For building LLVM "out-of-tree"` \ -DMLIR_DIR="$LLVM_INSTALL_DIR/lib/cmake/mlir/" \ @@ -132,6 +133,7 @@ sudo apt install clang ccache lld - Be sure to have built LLVM with `-DLLVM_ENABLE_PROJECTS=mlir`. - Be aware that the installed version of LLVM needs in general to match the committed version in `externals/llvm-project`. Using a different version may or may not work. + - [About MLIR debugging](https://mlir.llvm.org/getting_started/Debugging/) 1. **If you anticipate needing to frequently rebuild LLVM**, append: ```shell From c5996f4d351696a5f28959cb7e5f3017ff9a5cba Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Fri, 20 Dec 2024 23:06:02 +0000 Subject: [PATCH 17/25] docs(development.md): adds reminder to activate python environment before configuring with CMake --- docs/development.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/development.md b/docs/development.md index e2563ac54d61..a9b6783aa9a2 100644 --- a/docs/development.md +++ b/docs/development.md @@ -95,6 +95,7 @@ sudo apt install clang ccache lld #### Configure for Building +1. [Activate the Python environment](#set-up-the-python-environment) 1. Choose command with relevant LLVM options: 1. **If building "in-tree"**, run/append: From 7727257ff6e62d4617609aa52f5fb272e16a1c0d Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Tue, 17 Dec 2024 23:15:02 +0000 Subject: [PATCH 18/25] docs(development.md): removes unused libtorch variables CMake issued a warning upon fresh build: > Manually-specified variables were not used by the project: > LIBTORCH_CACHE > LIBTORCH_SRC_BUILD > LIBTORCH_VARIANT --- docs/development.md | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/docs/development.md b/docs/development.md index a9b6783aa9a2..65c7e96c97b5 100644 --- a/docs/development.md +++ b/docs/development.md @@ -147,18 +147,10 @@ sudo apt install clang ccache lld `# if using clang <= 13, replace --ld-path=ld.lld with -fuse-ld=lld` \ -DCMAKE_EXE_LINKER_FLAGS_INIT="--ld-path=ld.lld" \ -DCMAKE_MODULE_LINKER_FLAGS_INIT="--ld-path=ld.lld" \ - -DCMAKE_SHARED_LINKER_FLAGS_INIT="--ld-path=ld.lld" \ - `# Enabling libtorch binary cache instead of downloading the latest libtorch everytime.` \ - `# Testing against a mismatched version of libtorch may cause failures` \ - -DLIBTORCH_CACHE=ON \ - `# Enable an experimental path to build libtorch (and PyTorch wheels) from source,` \ - `# instead of downloading them` \ - -DLIBTORCH_SRC_BUILD=ON \ - `# Set the variant of libtorch to build / link against. (shared|static and optionally cxxabi11)` \ - -DLIBTORCH_VARIANT=shared + -DCMAKE_SHARED_LINKER_FLAGS_INIT="--ld-path=ld.lld" ``` - - This will build `libtorch` / `PyTorch` wheels from source and requires [the enablement mentioned earlier](#optional-enable-quicker-builds). + - This requires [the enablement mentioned earlier](#optional-enable-quicker-builds). - If these options cause issues, just skip them for now. 1. **If you need to enable local end-to-end tests**, append: From 5ec3bf51960546f472d59b3029c160dc8f4147c5 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Mon, 23 Dec 2024 20:17:04 +0000 Subject: [PATCH 19/25] docs(development.md): enunciates CMake configuration is required before building --- docs/development.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/development.md b/docs/development.md index 65c7e96c97b5..108f86de06c5 100644 --- a/docs/development.md +++ b/docs/development.md @@ -165,7 +165,7 @@ sudo apt install clang ccache lld #### Initiate Build -After either cmake run (in-tree/out-of-tree), use one of the following commands to build the project: +[Configure the build](#configure-for-building) if you haven't already done so. Then, use one of the following commands to build the project: ```shell # Build just torch-mlir (not all of LLVM) From dc8dd0ff5e94f14192e95deb1fccc48696295361 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Mon, 23 Dec 2024 20:17:48 +0000 Subject: [PATCH 20/25] docs(development.md): converts details under "Initiate Build" section to numbered list --- docs/development.md | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/docs/development.md b/docs/development.md index 108f86de06c5..6ce08c7c4a5c 100644 --- a/docs/development.md +++ b/docs/development.md @@ -165,21 +165,22 @@ sudo apt install clang ccache lld #### Initiate Build -[Configure the build](#configure-for-building) if you haven't already done so. Then, use one of the following commands to build the project: +1. [Configure the build](#configure-for-building) if you haven't already done so. +1. Use one of the following commands to build the project: -```shell -# Build just torch-mlir (not all of LLVM) -cmake --build build --target tools/torch-mlir/all + ```shell + # Build just torch-mlir (not all of LLVM) + cmake --build build --target tools/torch-mlir/all -# Run unit tests. -cmake --build build --target check-torch-mlir + # Run unit tests. + cmake --build build --target check-torch-mlir -# Run Python regression tests. -cmake --build build --target check-torch-mlir-python + # Run Python regression tests. + cmake --build build --target check-torch-mlir-python -# Build everything (including LLVM if in-tree) -cmake --build build -``` + # Build everything (including LLVM if in-tree) + cmake --build build + ``` ### Setup Python Environment to export the built Python packages From 647227b59ad54ee9a3daefbb1e9c1104233ade87 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Mon, 23 Dec 2024 20:25:13 +0000 Subject: [PATCH 21/25] docs(development.md): splits commands under "Initiate Build" into separate code blocks - makes them individually copyable from GitHub --- docs/development.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/development.md b/docs/development.md index 6ce08c7c4a5c..610b50f07239 100644 --- a/docs/development.md +++ b/docs/development.md @@ -171,13 +171,19 @@ sudo apt install clang ccache lld ```shell # Build just torch-mlir (not all of LLVM) cmake --build build --target tools/torch-mlir/all + ``` + ```shell # Run unit tests. cmake --build build --target check-torch-mlir + ``` + ```shell # Run Python regression tests. cmake --build build --target check-torch-mlir-python + ``` + ```shell # Build everything (including LLVM if in-tree) cmake --build build ``` From 73184107bb16612d676f89a8189cf89a15743872 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Mon, 23 Dec 2024 20:29:05 +0000 Subject: [PATCH 22/25] docs(development.md): extracts comments from commands under "Initiate Build" - makes it so that the GitHub copy button only grabs the code snippet itself --- docs/development.md | 35 +++++++++++++++++++---------------- 1 file changed, 19 insertions(+), 16 deletions(-) diff --git a/docs/development.md b/docs/development.md index 610b50f07239..9c98b7aeb467 100644 --- a/docs/development.md +++ b/docs/development.md @@ -167,26 +167,29 @@ sudo apt install clang ccache lld 1. [Configure the build](#configure-for-building) if you haven't already done so. 1. Use one of the following commands to build the project: + - Build just torch-mlir (not all of LLVM) - ```shell - # Build just torch-mlir (not all of LLVM) - cmake --build build --target tools/torch-mlir/all - ``` + ```shell + cmake --build build --target tools/torch-mlir/all + ``` - ```shell - # Run unit tests. - cmake --build build --target check-torch-mlir - ``` + - Run unit tests. - ```shell - # Run Python regression tests. - cmake --build build --target check-torch-mlir-python - ``` + ```shell + cmake --build build --target check-torch-mlir + ``` - ```shell - # Build everything (including LLVM if in-tree) - cmake --build build - ``` + - Run Python regression tests. + + ```shell + cmake --build build --target check-torch-mlir-python + ``` + + - Build everything (including LLVM if in-tree) + + ```shell + cmake --build build + ``` ### Setup Python Environment to export the built Python packages From d607f548bce706a9bb23319ba32b6e17d543da64 Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Mon, 23 Dec 2024 21:12:05 +0000 Subject: [PATCH 23/25] docs(development.md): moves vanilla build command to first bullet under "Initiate Build" section --- docs/development.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/development.md b/docs/development.md index 9c98b7aeb467..8af35eeb80ef 100644 --- a/docs/development.md +++ b/docs/development.md @@ -167,6 +167,12 @@ sudo apt install clang ccache lld 1. [Configure the build](#configure-for-building) if you haven't already done so. 1. Use one of the following commands to build the project: + - Build everything (including LLVM if in-tree) + + ```shell + cmake --build build + ``` + - Build just torch-mlir (not all of LLVM) ```shell @@ -185,12 +191,6 @@ sudo apt install clang ccache lld cmake --build build --target check-torch-mlir-python ``` - - Build everything (including LLVM if in-tree) - - ```shell - cmake --build build - ``` - ### Setup Python Environment to export the built Python packages #### Linux and macOS From 398db4f558ad8090f3ebdd19b29f7643ddea8ebc Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Mon, 23 Dec 2024 20:42:12 +0000 Subject: [PATCH 24/25] docs(development.md): restructures the "Initiate Build" section - less repetitive - emphasizes that the targets can be chained --- docs/development.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/development.md b/docs/development.md index 8af35eeb80ef..99cf16934055 100644 --- a/docs/development.md +++ b/docs/development.md @@ -166,31 +166,33 @@ sudo apt install clang ccache lld #### Initiate Build 1. [Configure the build](#configure-for-building) if you haven't already done so. -1. Use one of the following commands to build the project: - - Build everything (including LLVM if in-tree) +1. **If you want to...** + - **...build _everything_** (including LLVM if configured as "in-tree"), run: ```shell cmake --build build ``` - - Build just torch-mlir (not all of LLVM) + - **...build _just_ torch-mlir** (not all of LLVM), run: ```shell cmake --build build --target tools/torch-mlir/all ``` - - Run unit tests. + - **...run unit tests**, run: ```shell cmake --build build --target check-torch-mlir ``` - - Run Python regression tests. + - **...run Python regression tests**, run: ```shell cmake --build build --target check-torch-mlir-python ``` +TIP: add multiple target options to stack build phases + ### Setup Python Environment to export the built Python packages #### Linux and macOS From 84d17b6a7f8fd5563f8b8d68dd8177bdd603c73b Mon Sep 17 00:00:00 2001 From: Jacob Gordon Date: Tue, 21 Jan 2025 22:48:22 +0000 Subject: [PATCH 25/25] docs(development.md): simplifies options for specifying linker --- docs/development.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/docs/development.md b/docs/development.md index 99cf16934055..1dad06425bc9 100644 --- a/docs/development.md +++ b/docs/development.md @@ -144,10 +144,7 @@ sudo apt install clang ccache lld `# use ccache to cache build results` \ -DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER_LAUNCHER=ccache \ `# use LLD to link in seconds, rather than minutes` \ - `# if using clang <= 13, replace --ld-path=ld.lld with -fuse-ld=lld` \ - -DCMAKE_EXE_LINKER_FLAGS_INIT="--ld-path=ld.lld" \ - -DCMAKE_MODULE_LINKER_FLAGS_INIT="--ld-path=ld.lld" \ - -DCMAKE_SHARED_LINKER_FLAGS_INIT="--ld-path=ld.lld" + -DCMAKE_LINKER_TYPE=lld ``` - This requires [the enablement mentioned earlier](#optional-enable-quicker-builds).