Move model benchmarking #2016
Move model benchmarking #2016
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/2016
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New FailureAs of commit 381ef76 with merge base b375781 ( NEW FAILURE - The following job has failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
facebook-github-bot
added
the
CLA Signed
There was a problem hiding this comment.
Copilot reviewed 8 out of 16 changed files in this pull request and generated 1 comment.
Files not reviewed (8)
- benchmarks/models/llama/benchmark_results.txt: Language not supported
- benchmarks/models/llama/benchmarks.sh: Language not supported
- benchmarks/models/llama/bsr_bench_results.txt: Language not supported
- benchmarks/models/llama/bsr_benchmarks.sh: Language not supported
- benchmarks/models/llama/demo_summarize.sh: Language not supported
- benchmarks/models/llama/evals.sh: Language not supported
- benchmarks/models/sam/benchmark.sh: Language not supported
- benchmarks/models/sam/results.csv: Language not supported
Comments suppressed due to low confidence (1)
benchmarks/models/llama/_eval.py:335
- [nitpick] The variable name 'input' shadows the built-in function. Consider renaming it to 'model_input' or a similar descriptive name.
input = self.input_prep_func(inps)
| quantizer = Int4WeightOnlyGPTQQuantizer(group_size=groupsize, device=device) | ||
| model.setup_caches(max_batch_size=1, max_seq_length=calibration_seq_length) | ||
| model = quantizer.quantize(model, inputs).to(device) | ||
| else: |
There was a problem hiding this comment.
The else clause attached to the 'if "int4wo" in quantization and "gptq" in quantization:' block may execute unexpectedly when other quantization conditions are met. Consider restructuring these conditionals using elif to ensure each quantization case is handled independently.
Copilot is powered by AI, so mistakes are possible. Review output carefully before use.
No description provided.