Update on "int4 gptq shape fix"

Summary: redoing
5bf70c1
in a way that doesn't get reverted. note, needed to fix a device issue
as well.

Test Plan:

export MODEL_REPO=meta-llama/Llama-2-7b-chat-hf
python quantize.py --checkpoint_path checkpoints/$MODEL_REPO/model.pth --mode int4-gptq --calibration_tasks wikitext --calibration_limit 5
python eval.py --checkpoint_path checkpoints/$MODEL_REPO/model_int4-gptq.g32.cuda.pth --tasks wikitext --limit 5

Reviewers:

Subscribers:

Tasks:

Tags:

[ghstack-poisoned]

Author: HDCharles

Diff for: GPTQ.py

@@ -150,7 +150,7 @@ def __init__(
         }
 
         # trace model for one input
-        one_input = tuple([multi.values[0].cpu() for multi in inputs])
+        one_input = [multi.values[0].cpu() for multi in inputs]
         exported_model = torch._dynamo.export(
             model.cpu(), aten_graph=True, pre_dispatch=True, tracing_mode="fake"
         )(*one_input)