Update Readme

[drisspg]

Stacked PRs:

• ->Update Readme #1526

---

Update Readme

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/1526

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• ROCM Infra failures during checkout of PyTorch

✅ No Failures

As of commit d6a6302 with merge base 1c0ea5b ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[supriyar]

should this also mention DTensor composability?

[jerryzh168]

yeah I think so, @drisspg you can check out the sglang blog post for some desciprtions

[bhack]

Can we expand also few lines on autoquant that currently it is really a single line in the Readme?

E.g. I see some basic option here that it would nice to clarify as a basic use case:

ao/examples/sam2_amg_server/server.py

Lines 389 to 418 in 3eb18e7

	def set_autoquant(mask_generator, autoquant_type, min_sqnr):
	import torchao
	from torchao import autoquant
	# NOTE: Not baseline feature
	if autoquant_type == "autoquant":
	mask_generator.predictor.model.image_encoder = autoquant(
	mask_generator.predictor.model.image_encoder, min_sqnr=min_sqnr
	)
	elif autoquant_type == "autoquant-fp":
	mask_generator.predictor.model.image_encoder = autoquant(
	mask_generator.predictor.model.image_encoder,
	qtensor_class_list=torchao.quantization.DEFAULT_FLOAT_AUTOQUANT_CLASS_LIST,
	min_sqnr=min_sqnr,
	)
	elif autoquant_type == "autoquant-all":
	mask_generator.predictor.model.image_encoder = autoquant(
	mask_generator.predictor.model.image_encoder,
	qtensor_class_list=torchao.quantization.ALL_AUTOQUANT_CLASS_LIST,
	min_sqnr=min_sqnr,
	)
	else:
	raise ValueError(f"Unexpected autoquant type: {autoquant_type}")
	mask_generator.predictor._transforms_device = mask_generator.predictor.device
	torch.set_float32_matmul_precision("high")
	# NOTE: this fails when we run
	# python server.py ~/checkpoints/sam2 large --port 8000 --host localhost --fast --autoquant_type autoquant --unittest
	# https://gist.github.com/jerryzh168/d337cb5de0a1dec306069fe48ac8225e
	# mask_generator.predictor.model.sam_mask_decoder = autoquant(mask_generator.predictor.model.sam_mask_decoder, qtensor_class_list=DEFAULT_FLOAT_AUTOQUANT_CLASS_LIST, min_sqnr=40)