Skip to content

Fix per-feature boundary lost by ZCH index dedup (#4525) - #4525

Open
yingufan wants to merge 1 commit into
meta-pytorch:mainfrom
yingufan:export-D115457342
Open

Fix per-feature boundary lost by ZCH index dedup (#4525)#4525
yingufan wants to merge 1 commit into
meta-pytorch:mainfrom
yingufan:export-D115457342

Conversation

@yingufan

@yingufan yingufan commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Summary:

When a managed collision table has more than one feature, index dedup can move
ids from one feature to another.

All features of a table share one hash range (see _create_dedup_indices), so
fbgemm.jagged_unique_indices dedups them together. The op cannot tell which
feature each surviving id came from, so it just splits the total evenly over the
table's (feature, batch) slots. That is fine for a table with one feature. With
two features of very different lengths it is not: a 33 long feature and a 1280
long feature both come back as roughly 656, so ids end up under the wrong
feature.

This is a correctness problem and not just wrong bookkeeping, because ZCH decides
whether to insert an id from the name of the feature it arrived under: a feature
whose name ends in the read-only suffix is looked up but never written. Ids moved
to the wrong feature get written when they should not be.

The fix walks each surviving id back to the first feature it appeared in,
regroups the values so each feature is contiguous again, and recounts. Ids sent
by both features still collapse into one row, so we keep the dedup saving. It
only runs when a table actually has more than one feature, so the common path is
unchanged.

Also declares the reverse_indices field that _dedup_indices already appends
to. mc_modules has its own local EmbeddingCollectionContext that was missing
it, so sharding a ManagedCollisionCollection on its own with dedup enabled
raised AttributeError. It only worked through
ManagedCollisionEmbeddingCollection, whose context comes from embedding.py.

Differential Revision: D115457342

@meta-codesync

meta-codesync Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

@yingufan has exported this pull request. If you are a Meta employee, you can view the originating Diff in D115457342.

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Aug 11, 2026
@meta-codesync meta-codesync Bot changed the title Fix per-feature boundary lost by ZCH index dedup Fix per-feature boundary lost by ZCH index dedup (#4525) Aug 17, 2026
yingufan added a commit to yingufan/torchrec that referenced this pull request Aug 17, 2026
Summary:

`fbgemm.jagged_unique_indices` dedups a whole hash range at once, and
`_create_dedup_indices` puts every feature of a table into one range, so the op
returns the group's total unique count spread evenly over the group's
(feature, batch) slots rather than the real per-feature counts. A table binding a
single feature is unaffected. A table binding two features of very unequal length
gets ids relabeled across the boundary.

Managed collision modules read the feature an id arrives under as a control
signal, so this is a correctness bug rather than just wrong bookkeeping. On IG
Reels ESR the `media_embbedding_cache` HASH_ZCH table binds a ~33 long write
feature and a ~1280 long `_readonly` history feature; both come back as ~656, so
roughly 620 history ids per step are relabeled as candidates and inserted past
the read-only gate. The table reaches 100% occupancy within ~100 steps and, with
eviction disabled, only collides from then on.

Re-attribute every surviving row to the first feature it appeared in and regroup
the values so each feature is contiguous again. Rows shared by two features still
collapse into one, so the dedup saving is kept. Gated on a table actually binding
more than one feature, so the common path is untouched. Only per-feature totals
are made exact: the layout within a feature stays approximate, as it already was,
because reverse_indices undoes it downstream.

Also declares the `reverse_indices` field that `_dedup_indices` already appends
to. `mc_modules` shadows `EmbeddingCollectionContext` with a local dataclass that
lacks it, so a standalone sharded `ManagedCollisionCollection` with dedup on
raised `AttributeError`; only the `ManagedCollisionEmbeddingCollection` path
worked, since that context derives from the `embedding.py` class instead.

Differential Revision: D115457342
yingufan added a commit to yingufan/torchrec that referenced this pull request Aug 19, 2026
Summary:

When a managed collision table has more than one feature, index dedup can move
ids from one feature to another.

All features of a table share one hash range (see `_create_dedup_indices`), so
`fbgemm.jagged_unique_indices` dedups them together. The op cannot tell which
feature each surviving id came from, so it just splits the total evenly over the
table's (feature, batch) slots. That is fine for a table with one feature. With
two features of very different lengths it is not: a 33 long feature and a 1280
long feature both come back as roughly 656, so ids end up under the wrong
feature.

This is a correctness problem and not just wrong bookkeeping, because ZCH decides
whether to insert an id from the name of the feature it arrived under: a feature
whose name ends in the read-only suffix is looked up but never written. Ids moved
to the wrong feature get written when they should not be.

The fix walks each surviving id back to the first feature it appeared in,
regroups the values so each feature is contiguous again, and recounts. Ids sent
by both features still collapse into one row, so we keep the dedup saving. It
only runs when a table actually has more than one feature, so the common path is
unchanged.

Also declares the `reverse_indices` field that `_dedup_indices` already appends
to. `mc_modules` has its own local `EmbeddingCollectionContext` that was missing
it, so sharding a `ManagedCollisionCollection` on its own with dedup enabled
raised `AttributeError`. It only worked through
`ManagedCollisionEmbeddingCollection`, whose context comes from `embedding.py`.

Differential Revision: D115457342
Summary:

When a managed collision table has more than one feature, index dedup can move
ids from one feature to another.

All features of a table share one hash range (see `_create_dedup_indices`), so
`fbgemm.jagged_unique_indices` dedups them together. The op cannot tell which
feature each surviving id came from, so it just splits the total evenly over the
table's (feature, batch) slots. That is fine for a table with one feature. With
two features of very different lengths it is not: a 33 long feature and a 1280
long feature both come back as roughly 656, so ids end up under the wrong
feature.

This is a correctness problem and not just wrong bookkeeping, because ZCH decides
whether to insert an id from the name of the feature it arrived under: a feature
whose name ends in the read-only suffix is looked up but never written. Ids moved
to the wrong feature get written when they should not be.

The fix walks each surviving id back to the first feature it appeared in,
regroups the values so each feature is contiguous again, and recounts. Ids sent
by both features still collapse into one row, so we keep the dedup saving. It
only runs when a table actually has more than one feature, so the common path is
unchanged.

Also declares the `reverse_indices` field that `_dedup_indices` already appends
to. `mc_modules` has its own local `EmbeddingCollectionContext` that was missing
it, so sharding a `ManagedCollisionCollection` on its own with dedup enabled
raised `AttributeError`. It only worked through
`ManagedCollisionEmbeddingCollection`, whose context comes from `embedding.py`.

Differential Revision: D115457342
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. meta-exported

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant