Migration to AMDShark (#962)

Migration to AMDShark

Signed-off-by: pdhirajkumarprasad <[email protected]>

Author: pdhirajkumarprasad

.github/workflows/test_shark.yml renamed to .github/workflows/test_AMDShark.yml

@@ -1,4 +1,4 @@
-name: Test SHARK
+name: Test AMDShark
 
 on:
   workflow_dispatch:
@@ -16,7 +16,7 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  test-shark:
+  test-AMDShark:
     strategy:
       matrix:
         version: [3.11]
@@ -29,11 +29,11 @@ jobs:
         with:
           python-version: ${{matrix.version}}
 
-      - name: "Checkout SHARK"
+      - name: "Checkout AMDShark"
         uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
         with:
-          repository: "nod-ai/SHARK.git"
-          path: SHARK
+          repository: "nod-ai/AMDShark.git"
+          path: AMDShark
           ref: "main"
 
       - name: "Checkout iree-turbine"
@@ -43,12 +43,12 @@ jobs:
           # TODO: Let the ref be passed as a parameter to run integration tests.
           path: iree-turbine
 
-      # TODO: Replace with a sh script from shark repo
-      - name: "Install SHARK"
+      # TODO: Replace with a sh script from AMDShark repo
+      - name: "Install AMDShark"
         run: |
-          cd $GITHUB_WORKSPACE/SHARK
-          python${{ matrix.version }} -m venv shark.venv
-          source shark.venv/bin/activate
+          cd $GITHUB_WORKSPACE/AMDShark
+          python${{ matrix.version }} -m venv AMDShark.venv
+          source AMDShark.venv/bin/activate
           pip install -r requirements.txt --no-cache-dir
           pip install -e .
-          python apps/shark_studio/tests/api_test.py
+          python apps/AMDShark_studio/tests/api_test.py

.github/workflows/test_models.yml

@@ -28,9 +28,9 @@ jobs:
       HF_TOKEN: ${{ secrets.HF_TOKEN }}
       AZ_PRIVATE_CONNECTION: ${{ secrets.ONNXPRIVATESTORAGE_AZ_PRIVATE_CONNECTION }}
       ROCR_VISIBLE_DEVICES: ${{ matrix.visible-device }}
-      TORCH_HOME: /groups/aig_sharks/test-suite-ci-cache
-      HF_HOME: /groups/aig_sharks/test-suite-ci-cache
-      TURBINE_TANK_CACHE_DIR: /groups/aig_sharks/test-suite-ci-cache
+      TORCH_HOME: /groups/aig_AMDSharks/test-suite-ci-cache
+      HF_HOME: /groups/aig_AMDSharks/test-suite-ci-cache
+      TURBINE_TANK_CACHE_DIR: /groups/aig_AMDSharks/test-suite-ci-cache
     steps:
       # We are using a persistent Gentoo runner here, and this python action is not supported for the arch
       # - name: "Setting up Python"

.github/workflows/test_sdxl.yml

@@ -30,7 +30,7 @@ jobs:
         with:
           python-version: ${{matrix.version}}
 
-      - name: "Checkout SHARK-ModelDev"
+      - name: "Checkout AMDShark-ModelDev"
         uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
         with:
           ref: bump-punet-tom

README.md

@@ -1,4 +1,4 @@
-# SHARK Turbine
+# AMDShark Turbine
 
 This repo is Nod-AI's integration repository for various model bringup
 activities and CI. In 2023 and early 2024, it played a different role
@@ -9,7 +9,7 @@ were developed, including:
 * [Torch-MLIR ONNX Importer](https://github.com/llvm/torch-mlir/blob/main/python/torch_mlir/extras/onnx_importer.py)
 * [Torch-MLIR's ONNX C Importer](https://github.com/llvm/torch-mlir/tree/main/projects/onnx_c_importer)
 * [IREE Turbine](https://github.com/iree-org/iree-turbine)
-* [Sharktank and Shortfin](https://github.com/nod-ai/sharktank)
+* [AMDSharktank and Shortfin](https://github.com/nod-ai/AMDSharktank)
 
 As these have all found upstream homes, this repo is a bit bare. We will
 continue to use it as a staging ground for things that don't have a

models/README.md

@@ -14,12 +14,12 @@ YOUR_HF_TOKEN="insert token for headless"
 
 # clone and install dependencies
 sudo apt install -y git
-git clone https://github.com/nod-ai/SHARK-Turbine.git
-cd SHARK-Turbine
+git clone https://github.com/nod-ai/AMDShark-Turbine.git
+cd AMDShark-Turbine
 pip install -r core/requirements.txt
 pip install -r models/requirements.txt
 
-# do an editable install from the cloned SHARK-Turbine
+# do an editable install from the cloned AMDShark-Turbine
 pip install --editable core models
 
 # Log in with Hugging Face CLI if token setup is required

models/requirements.txt

@@ -4,7 +4,7 @@ transformers==4.50.0
 torchsde
 accelerate
 peft
-diffusers @ git+https://github.com/nod-ai/[email protected]shark
+diffusers @ git+https://github.com/nod-ai/[email protected]AMDShark
 brevitas @ git+https://github.com/Xilinx/brevitas.git@6695e8df7f6a2c7715b9ed69c4b78157376bb60b
 # turbine tank downloading/uploading
 azure-storage-blob
@@ -13,4 +13,4 @@ einops
 pytest
 scipy
 iree-turbine @ git+https://github.com/iree-org/iree-turbine.git@main
--e git+https://github.com/nod-ai/sharktank.git@main#egg=sharktank&subdirectory=sharktank
+-e git+https://github.com/nod-ai/AMDSharktank.git@main#egg=AMDSharktank&subdirectory=AMDSharktank

models/setup.py

@@ -35,12 +35,12 @@ def load_version_info():
 setup(
     name=f"turbine-models",
     version=f"{PACKAGE_VERSION}",
-    author="SHARK Authors",
+    author="AMDShark Authors",
     author_email="[email protected]",
-    description="SHARK Turbine Machine Learning Model Zoo",
+    description="AMDShark Turbine Machine Learning Model Zoo",
     long_description=README,
     long_description_content_type="text/markdown",
-    url="https://github.com/nod-ai/SHARK-Turbine",
+    url="https://github.com/nod-ai/AMDShark-Turbine",
     license="Apache-2.0",
     classifiers=[
         "Development Status :: 3 - Alpha",
@@ -54,7 +54,7 @@ def load_version_info():
         ],
     ),
     install_requires=[
-        "Shark-Turbine",
+        "AMDShark-Turbine",
         "protobuf",
         "sentencepiece",
         "transformers>=4.37.1",

models/turbine_models/custom_models/README.md

@@ -1,9 +1,9 @@
 # Instructions
 
-Clone and install SHARK-Turbine
+Clone and install AMDShark-Turbine
 ```
-git clone https://github.com/nod-ai/SHARK-Turbine.git
-cd SHARK-Turbine
+git clone https://github.com/nod-ai/AMDShark-Turbine.git
+cd AMDShark-Turbine
 python -m venv turbine_venv && source turbine_venv/bin/activate
 
 pip install --index-url https://download.pytorch.org/whl/cpu \

models/turbine_models/custom_models/llama_benchmark/README.md

@@ -1,9 +1,9 @@
 # Instructions
 
-Clone and install SHARK-Turbine
+Clone and install AMDShark-Turbine
 ```
-git clone https://github.com/nod-ai/SHARK-Turbine.git
-cd SHARK-Turbine
+git clone https://github.com/nod-ai/AMDShark-Turbine.git
+cd AMDShark-Turbine
 python -m venv turbine_venv && source turbine_venv/bin/activate
 
 pip install --upgrade -r requirements.txt

models/turbine_models/custom_models/llm_optimizations/streaming_llm/README.md

@@ -27,6 +27,6 @@ python python/turbine_models/custom_models/llm_runner.py --vmfb_path=/path/to/Ll
 ```
 
 ## Future Work:
-- [ ] Make window size configurable through python, everything is there but we'd need to initialize with a default value which would only be possible after we let `_create_initial_value` to take in initial value from GlobalAttribute somewhere [here](https://github.com/nod-ai/SHARK-Turbine/blob/18e8a4100b61adfd9425dd32f780dc5f90017813/python/shark_turbine/aot/support/ir_utils.py#L284-L316) . 
+- [ ] Make window size configurable through python, everything is there but we'd need to initialize with a default value which would only be possible after we let `_create_initial_value` to take in initial value from GlobalAttribute somewhere [here](https://github.com/nod-ai/AMDShark-Turbine/blob/18e8a4100b61adfd9425dd32f780dc5f90017813/python/AMDShark_turbine/aot/support/ir_utils.py#L284-L316) . 
 - [ ] Get flow.move to enable overlap of sliding window and src of data. (Currently need to evict when it's at least 2x size of window) For example by default our streamingLLM window_size is 256, so we evict at ~600(slightly more than 2x for safety) token.
 - [ ] Introduce Rerotation of RoPE to as seen [here](https://github.com/huggingface/transformers/blob/c2d283a64a7f33547952e3eb0fa6533fc375bcdd/src/transformers/cache_utils.py#L213-L218) to remove invasive modification of LlamaAttention module for streamingLLM.