Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/test_AMDShark.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,11 @@ jobs:
with:
python-version: ${{matrix.version}}

- name: "Checkout AMDShark"
- name: "Checkout AMD-Shark"
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
with:
repository: "nod-ai/AMDShark.git"
path: AMDShark
repository: "nod-ai/AMD-Shark.git"
path: AMD-Shark
ref: "main"

- name: "Checkout iree-turbine"
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ were developed, including:
* [Torch-MLIR ONNX Importer](https://github.com/llvm/torch-mlir/blob/main/python/torch_mlir/extras/onnx_importer.py)
* [Torch-MLIR's ONNX C Importer](https://github.com/llvm/torch-mlir/tree/main/projects/onnx_c_importer)
* [IREE Turbine](https://github.com/iree-org/iree-turbine)
* [AMDSharktank and Shortfin](https://github.com/nod-ai/AMDSharktank)
* [AMDSharktank and Shortfin](https://github.com/nod-ai/Sharktank)

As these have all found upstream homes, this repo is a bit bare. We will
continue to use it as a staging ground for things that don't have a
Expand Down
4 changes: 2 additions & 2 deletions models/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ YOUR_HF_TOKEN="insert token for headless"

# clone and install dependencies
sudo apt install -y git
git clone https://github.com/nod-ai/AMDShark-Turbine.git
cd AMDShark-Turbine
git clone https://github.com/nod-ai/AMD-Shark-ModelDev.git
cd AMD-Shark-ModelDev
pip install -r core/requirements.txt
pip install -r models/requirements.txt

Expand Down
4 changes: 2 additions & 2 deletions models/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ transformers==4.50.0
torchsde
accelerate
peft
diffusers @ git+https://github.com/nod-ai/[email protected]AMDShark
diffusers @ git+https://github.com/nod-ai/[email protected]Shark
brevitas @ git+https://github.com/Xilinx/brevitas.git@6695e8df7f6a2c7715b9ed69c4b78157376bb60b
# turbine tank downloading/uploading
azure-storage-blob
Expand All @@ -13,4 +13,4 @@ einops
pytest
scipy
iree-turbine @ git+https://github.com/iree-org/iree-turbine.git@main
-e git+https://github.com/nod-ai/AMDSharktank.git@main#egg=AMDSharktank&subdirectory=AMDSharktank
-e git+https://github.com/nod-ai/Sharktank.git@main#egg=Sharktank&subdirectory=Sharktank
2 changes: 1 addition & 1 deletion models/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ def load_version_info():
description="AMDShark Turbine Machine Learning Model Zoo",
long_description=README,
long_description_content_type="text/markdown",
url="https://github.com/nod-ai/AMDShark-Turbine",
url="https://github.com/nod-ai/AMD-SHARK-ModelDev",
license="Apache-2.0",
classifiers=[
"Development Status :: 3 - Alpha",
Expand Down
4 changes: 2 additions & 2 deletions models/turbine_models/custom_models/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

Clone and install AMDShark-Turbine
```
git clone https://github.com/nod-ai/AMDShark-Turbine.git
cd AMDShark-Turbine
git clone https://github.com/nod-ai/AMD-SHARK-ModelDev.git
cd AMD-SHARK-ModelDev
python -m venv turbine_venv && source turbine_venv/bin/activate

pip install --index-url https://download.pytorch.org/whl/cpu \
Expand Down
4 changes: 2 additions & 2 deletions models/turbine_models/custom_models/llama_benchmark/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

Clone and install AMDShark-Turbine
```
git clone https://github.com/nod-ai/AMDShark-Turbine.git
cd AMDShark-Turbine
git clone https://github.com/nod-ai/AMD-SHARK-ModelDev.git
cd AMD-SHARK-ModelDev
python -m venv turbine_venv && source turbine_venv/bin/activate

pip install --upgrade -r requirements.txt
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,6 @@ python python/turbine_models/custom_models/llm_runner.py --vmfb_path=/path/to/Ll
```

## Future Work:
- [ ] Make window size configurable through python, everything is there but we'd need to initialize with a default value which would only be possible after we let `_create_initial_value` to take in initial value from GlobalAttribute somewhere [here](https://github.com/nod-ai/AMDShark-Turbine/blob/18e8a4100b61adfd9425dd32f780dc5f90017813/python/AMDShark_turbine/aot/support/ir_utils.py#L284-L316) .
- [ ] Make window size configurable through python, everything is there but we'd need to initialize with a default value which would only be possible after we let `_create_initial_value` to take in initial value from GlobalAttribute somewhere [here](https://github.com/nod-ai/AMD-SHARK-ModelDev/blob/18e8a4100b61adfd9425dd32f780dc5f90017813/python/AMDShark_turbine/aot/support/ir_utils.py#L284-L316) .
- [ ] Get flow.move to enable overlap of sliding window and src of data. (Currently need to evict when it's at least 2x size of window) For example by default our streamingLLM window_size is 256, so we evict at ~600(slightly more than 2x for safety) token.
- [ ] Introduce Rerotation of RoPE to as seen [here](https://github.com/huggingface/transformers/blob/c2d283a64a7f33547952e3eb0fa6533fc375bcdd/src/transformers/cache_utils.py#L213-L218) to remove invasive modification of LlamaAttention module for streamingLLM.
- [ ] Introduce Rerotation of RoPE to as seen [here](https://github.com/huggingface/transformers/blob/c2d283a64a7f33547952e3eb0fa6533fc375bcdd/src/transformers/cache_utils.py#L213-L218) to remove invasive modification of LlamaAttention module for streamingLLM.
4 changes: 2 additions & 2 deletions models/turbine_models/tests/stateless_llama_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ def tearDownClass(cls):
cls.tokenizer = None
cls.mod = None

# See: https://github.com/nod-ai/AMDShark-Turbine/issues/601
# See: https://github.com/nod-ai/AMD-SHARK-ModelDev/issues/601
# Developed issues related to the pytorch 2.3 upgrade.
@unittest.expectedFailure
def test_vmfb_comparison(self):
Expand Down Expand Up @@ -139,7 +139,7 @@ def test_vmfb_comparison(self):
new_blob_name = new_blob_name[0] + "-pass.mlir"
turbine_tank.changeBlobName(blob_name, new_blob_name)

# See: https://github.com/nod-ai/AMDShark-Turbine/issues/601
# See: https://github.com/nod-ai/AMD-SHARK-ModelDev/issues/601
# Developed issues related to the pytorch 2.3 upgrade.
@unittest.expectedFailure
def test_streaming_vmfb_comparison(self):
Expand Down
Loading