Initial commit

Author: lucadellalib

.gitignore

@@ -0,0 +1,118 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+.hypothesis/
+.pytest_cache/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# celery beat schedule file
+celerybeat-schedule
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/*
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# PyCharm
+.idea/

.pre-commit-config.yaml

@@ -0,0 +1,22 @@
+repos:
+  - repo: https://github.com/pre-commit/pre-commit-hooks
+    rev: v2.5.0
+    hooks:
+      - id: trailing-whitespace
+        types: [file, text]
+      - id: end-of-file-fixer
+        types: [python]
+      - id: mixed-line-ending
+        types: [python]
+        args: ["--fix=lf"]
+
+  - repo: https://github.com/psf/black
+    rev: 24.3.0
+    hooks:
+      - id: black
+        types: [python]
+
+  - repo: https://github.com/pycqa/isort
+    rev: 5.13.2
+    hooks:
+      - id: isort

MANIFEST.in

@@ -0,0 +1,3 @@
+include llama3/py.typed
+global-exclude __pycache__
+global-exclude *.py[co]

NOTICE

@@ -0,0 +1,12 @@
+This project incorporates components from the projects listed below. The original copyright notices are set forth below.
+
+#############################################################################################################################################################
+
+1. Code in {llama3/llama3.py, main.py} adapted from:
+   https://github.com/meta-llama/llama3
+
+META LLAMA 3 COMMUNITY LICENSE AGREEMENT
+
+Meta Llama 3 is licensed under the Meta Llama 3 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.
+
+#############################################################################################################################################################

README.md

@@ -0,0 +1,78 @@
+# LLaMA 3
+
+A single-file implementation of [LLaMA 3](https://arxiv.org/abs/2407.21783), with support for jitting, KV caching and prompting.
+
+The original implementation can be found at https://github.com/meta-llama/llama3.
+
+---------------------------------------------------------------------------------------------------------
+
+## 🛠️️ Installation
+
+### Using Pip
+
+First of all, install [Python 3.8 or later](https://www.python.org). Open a terminal and run:
+
+```bash
+pip install git+https://github.com/lucadellalib/llama3@main#egg=llama3[all]
+```
+
+### From Source
+
+First of all, install [Python 3.8 or later](https://www.python.org).
+Clone or download and extract the repository, navigate to `<path-to-repository>`, open a terminal and run:
+
+```bash
+# Install the package locally in editable mode
+pip install -e .[all]
+```
+
+---------------------------------------------------------------------------------------------------------
+
+## ▶️ Quickstart
+
+### Importing the Model in Your Own Script
+
+```python
+import torch
+from llama3 import LlamaDecoder
+
+B, H, K = 3, 512, 30
+model = LlamaDecoder(K)
+print(model)
+
+# Process 50 timesteps
+input = torch.randn(B, 50, H)
+output, state = model(input)
+print(output.shape)
+
+# Process 2 additional timesteps
+input = torch.randn(B, 2, H)
+output, state = model(input, state=state)
+print(output.shape)
+
+# JIT the model
+model_jit = model.jit()
+output_jit, state_jit = model_jit(input)
+print(output.shape)
+```
+
+### Inference Example With Pretrained Checkpoint
+
+First of all, download the model weights and tokenizer (pretrained variant, e.g. Llama3.2-1B). Check the official
+website for instructions on how to [download the models](https://github.com/meta-llama/llama3#download).
+
+Navigate to `<path-to-repository>`, open a terminal and run:
+
+```bash
+python main.py --checkpoint_path <path-to-checkpoint>
+```
+
+It is recommended to run this script on a machine with at least 1 GPU.
+
+---------------------------------------------------------------------------------------------------------
+
+## 📧 Contact
+
+[[email protected]](mailto:[email protected])
+
+---------------------------------------------------------------------------------------------------------

llama3/__init__.py

@@ -0,0 +1,2 @@
+from llama3.llama3 import *
+from llama3.version import VERSION as __version__