Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reformulate l1 and dram memory maps #2414

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Conversation

nsmithtt
Copy link
Contributor

This commit is largely broken into 2 parts:

  • Reformulations of the l1 and dram memory map
  • Breaking the stream layout in half, to distinguish physical shard layout, from a virtual stream layout. This layout refactoring was also done such that their affine maps compose with the way the device memory maps have been redefined.

Memory Maps

This PR reformulates the l1 and dram memory maps to make them simpler, easier to use, and more compatible with the new direction of the direct to metal backend path.

Stream and Shard memref layouts

This PR also breaks the stream layout into two. This makes the interpretation of memory for each layout much more explicit and should make it easier for lowering passes to use them in conjunction with the device memory maps.

Stream Layout

Describes a stream layout of a memref buffer.

  • AffineMap: Provides affine map indexing into the associated data stream.

Only the stream_layout op should return memref's with this attribute. The stream layout attribute is necessary for two reasons:

  • It provides a way to reblock the data stream into a different shape (via affine map). Usually this would be some subblock of the original backing memory to chunk the data into smaller pieces.
  • The type itself is a signal to datamovement passes that the memref is a stream and should be treated as such.

Shard Layout

Describes shard layout of a memref buffer.

  • Stride: Stride of each dim in bytes.
  • Buffers: Number of back buffers used for double buffering, I/O latency hiding, etc

The shard layout attribute is a description of how each shard of a memref is laid out in memory. Memref's with this layout type implicitly mean their data is distributed across a grid of cores.

This commit is largely broken into 2 parts:
- Reformulations of the l1 and dram memory map
- Breaking the stream layout in half, to distinguish physical shard
  layout, from a virtual stream layout. This layout refactoring was also
  done such that their affine maps compose with the way the device
  memory maps have been redefined.

## Memory Maps

This PR reformulates the l1 and dram memory maps to make them simpler,
easier to use, and more compatible with the new direction of the direct
to metal backend path.

## Stream and Shard memref layouts

This PR also breaks the stream layout into two.  This makes the
interpretation of memory for each layout much more explicit and should
make it easier for lowering passes to use them in conjunction with the
device memory maps.

### Stream Layout

Describes a stream layout of a memref buffer.
- AffineMap: Provides affine map indexing into the associated data stream.

Only the stream_layout op should return memref's with this attribute.  The stream layout
attribute is necessary for two reasons:
  - It provides a way to reblock the data stream into a different shape (via affine map).
    Usually this would be some subblock of the original backing memory to chunk the data
    into smaller pieces.
  - The type itself is a signal to datamovement passes that the memref is a stream and
    should be treated as such.

### Shard Layout

Describes shard layout of a memref buffer.
- Stride: Stride of each dim in bytes.
- Buffers: Number of back buffers used for double buffering, I/O latency hiding, etc

The shard layout attribute is a description of how each shard of a memref is laid out in
memory. Memref's with this layout type implicitly mean their data is distributed across
a grid of cores.
@nsmithtt nsmithtt force-pushed the nsmith/device-maps branch from 591fdbe to 0077d1f Compare March 10, 2025 04:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant