@@ -17,8 +17,9 @@ We have released several foundation models (sparse or sparse-and-quantized) for
17
17
| -----------------------------------------------------------------------------------| ----------| ------------------------------------------------------------------------------------------------------| ------------------------------------------------------------------------------------------------------------------|
18
18
| [ Mistral-7B-v0.3] ( https://huggingface.co/mistralai/Mistral-7B-v0.3 ) | 50% | [ IntelLabs/sqft-mistral-7b-v0.3-50-base] ( https://huggingface.co/IntelLabs/sqft-mistral-7b-v0.3-50-base ) | [ IntelLabs/sqft-mistral-7b-v0.3-50-base-gptq] ( https://huggingface.co/IntelLabs/sqft-mistral-7b-v0.3-50-base-gptq ) |
19
19
| [ Phi-3-mini-4k-instruct] ( https://huggingface.co/microsoft/Phi-3-mini-4k-instruct ) | 50% | [ IntelLabs/sqft-phi-3-mini-4k-50-base] ( https://huggingface.co/IntelLabs/sqft-phi-3-mini-4k-50-base ) | [ IntelLabs/sqft-phi-3-mini-4k-50-base-gptq] ( https://huggingface.co/IntelLabs/sqft-phi-3-mini-4k-50-base-gptq ) |
20
- | [ Meta-Llama-3-8B] ( https://huggingface.co/meta-llama/Meta-Llama-3-8B ) | 50% | [ IntelLabs/sqft-llama-3-8b-50-base] ( ) | [ IntelLabs/sqft-llama-3-8b-50-base-gptq] ( ) |
21
- ` * ` ** Llama-3 models are under review**
20
+ | [ Meta-Llama-3-8B] ( https://huggingface.co/meta-llama/Meta-Llama-3-8B ) | 50% | IntelLabs/sqft-llama-3-8b-50-base<sup >* </sup > | IntelLabs/sqft-llama-3-8b-50-base-gptq<sup >* </sup > |
21
+
22
+ <sup >* </sup > * Llama-3 models are currently under internal review and will be released soon.*
22
23
23
24
[ // ] : # ( https://huggingface.co/IntelLabs/sqft-llama-3-8b-50-base )
24
25
[ // ] : # ( https://huggingface.co/IntelLabs/sqft-llama-3-8b-50-base-gptq )
@@ -333,10 +334,11 @@ lm_eval --model hf \
333
334
334
335
| Base Model | Task | Method | Fine-tuned Model |
335
336
| ------------------------------------------------------------------------------------------------| --------| -----------------------| -------------------------------------------------------------------------------------------------------------------------------------|
336
- | [ sqft-llama-3-8b-50-base] ( ) | GSM8K | SQFT + SparsePEFT | [ sqft-llama-3-8b-50-gptq-gsm8k-heu-adapter] ( ) |
337
- | [ sqft-llama-3-8b-50-base-gptq] ( ) | GSM8K | SQFT | [ sqft-sparsepeft-llama-3-8b-50-gsm8k-heu] ( ) |
338
- | [ sqft-llama-3-8b-50-base-gptq] ( ) | GSM8K | SQFT + QA-SparsePEFT | [ sqft-qa-sparsepeft-llama-3-8b-50-gptq-gsm8k-heu] ( ) |
339
- ` * ` ** Llama-3-8B fine-tuned models are under review**
337
+ | sqft-llama-3-8b-50-base<sup >* </sup > | GSM8K | SQFT + SparsePEFT | sqft-llama-3-8b-50-gptq-gsm8k-heu-adapter |
338
+ | sqft-llama-3-8b-50-base-gptq | GSM8K | SQFT | sqft-sparsepeft-llama-3-8b-50-gsm8k-heu |
339
+ | sqft-llama-3-8b-50-base-gptq | GSM8K | SQFT + QA-SparsePEFT | sqft-qa-sparsepeft-llama-3-8b-50-gptq-gsm8k-heu |
340
+
341
+ <sup >* </sup > * Llama-3 models are currently under internal review and will be released soon.*
340
342
341
343
[ // ] : # ( https://huggingface.co/IntelLabs/sqft-llama-3-8b-50-base )
342
344
@@ -360,4 +362,4 @@ If you find SQFT's code and paper helpful, please kindly cite:
360
362
year = {2024},
361
363
url = {}
362
364
}
363
- ```
365
+ ```
0 commit comments