[QST] What is PermutationMNK in TiledMMA in CUTLASS 3.4 changes?

[hyhieu]

Hi everyone,

I noticed that in CUTLASS 3.4 (#1286), there was a change in cute/atom/mma_atom.hpp that altered the signature of make_tiled_mma and the template of TiledMMA.

Specifically:

CUTLASS 3.4:

// @tparam MMA_Atom The MMA_Atom to use in the TiledMMA
// @tparam AtomLayoutMNK The MNK-tiling of the Atom to be performed.
// @tparam PermuationsMNK Permutations to apply to each MNK-mode before tiling for the Atom.
template <class MMA_Atom,
          class AtomLayoutMNK,
          class PermutationMNK = Tile<Underscore,Underscore,Underscore>>
struct TiledMMA : MMA_Atom { ... }

Before:

template <class MMA_Atom,
          class AtomLayoutMNK   = Layout<Shape<_1,_1,_1>>,
          class ValLayoutMNK    = Layout<Shape<_1,_1,_1>>,
          class PermutationsMNK = Tile<Underscore,Underscore,Underscore>>
struct TiledMMA : MMA_Atom { ... }

So, what is the PermutationMNK in the new interface? How does it achieve the effect of ValLayoutMNK and PermutationsMNK in the older version?

To be clear, thanks to this Zhihu post, I kind of understood that in the old version:

• AtomLayoutMNK means that the MMA_Atom is replicated into more threads, and so the CTA becomes accordingly.
• ValLayoutMNK means that the TiledMMA performs more compute by repeating the MMA_Atom, and so the CTA does not become bigger, even though the TiledShape computed by the TiledMMA still becomes bigger.

Now that ValLayoutMNK is gone, I noticed that this still does the job for me:

using TiledMma = decltype(
  make_tiled_mma(
    MmaAtom{},
    Layout<Shape<_1, _1, _1>>{},  // this should be AtomLayoutMNK
    Layout<Shape<_2, _2, _2>>{}   // this should be PermutationMNK, but what does it do now?
  )
);

but I don't know what is going on with what I passed to make_tiled_mma as PermutationsMNK.

Hoping for an explanation.

Thanks!

[ccecka]

Thanks for the question, my update to the CuTe documentation and examples is still pending but should be available soon.

The easiest way to think about it is that the Permutation parameter is a Tiler for the MNK modes of the MMA. That is, it is a set of three layouts that are applied to the M-mode, N-mode, and K-mode individually before applying the TV-layouts and projective slicing. These layouts can act to permute the M-mode, N-mode, and K-mode individually to make the TV-partitioning patterns more manageable/intuitive and effectively interleave individual MMAs.

I'll start with some examples and then get to your case. Let's start with SM80_8x8x4_F64F64F64F64_TN

    TiledMMA tiled_mma = make_tiled_mma(SM80_8x8x4_F64F64F64F64_TN{});
    print_latex(tiled_mma);

The above is equivalent to

    TiledMMA tiled_mma = make_tiled_mma(SM80_8x8x4_F64F64F64F64_TN{},
                                        Layout<Shape<_1,_1,_1>>{},
                                        Tile<_8,_8,_4>{});
    print_latex(tiled_mma);

as the atom already has a natural tile size of 8x8x4. But we can immediately expand this "tile size" up to 8x16x8 instead:

    TiledMMA tiled_mma = make_tiled_mma(SM80_8x8x4_F64F64F64F64_TN{},
                                        Layout<Shape<_1,_1,_1>>{},     // AtomLayout
                                        Tile<_8,_16,_8>{});            // Tiler
    print_latex(tiled_mma);

This doesn't actually affect the partitioning of input/output tensors because, by convention, only a single atom is ever partitioned out. It will affect the output of tile_size and get_layoutC_MN and get_layoutC_TV etc, which could affect any TiledCopy that rely on those partitioning patterns by being built on this TiledMMA. Regardless, you'll find the resulting tensors from partition_C etc to be exactly the same since the atom partitioning is exactly the same.

Continuing, I see those four values that T0 write to in C are separate, but I would prefer them to be next to each other. That is, I would like to permute the N-mode.

    TiledMMA tiled_mma = make_tiled_mma(SM80_8x8x4_F64F64F64F64_TN{},
                                        Layout<Shape<_1,_1,_1>>{},     // AtomLayout
                                        Tile<_8,                       // Permutation on M, equivalent to 8:1, identity
                                             Layout<Shape <_2,_4,_2>, 
                                                    Stride<_1,_4,_2>>, // Permutation on N, size 16
                                             _8>{});                   // Permutation on K, equivalent to 8:1, identity
    print_latex(tiled_mma);

That layout (2,4,2):(1,4,2) is read like a scatter permutation, telling the n-coords of the original image where to go in the new image: The first 2 elements stay put, the next 4 of those 2-element groups get sent to n-coord 4, and the next 2 of those 8-element groups get sent to n-coord 2. This permutes only the n-mode (in C and B accordingly) and brings the access of all threads to be contiguous in n-coordinates. This is convenient when designing layouts for shared memory or registers, for example. The two MMA instructions contained within the image above are now effectively interleaved in the logical n-coordinates.

Your example is doing something a bit silly, but perfectly functional.

using TiledMma = decltype(
  make_tiled_mma(
    MmaAtom{},
    Layout<Shape<_1, _1, _1>>{},  // this should be AtomLayoutMNK
    Layout<Shape<_2, _2, _2>>{}   // this should be PermutationMNK, but what does it do now?
  )
);

Because make_tiled_mma massages the inputs to get reasonable defaults by:

make_tiled_mma(MMA_Atom<MMA_Op> const& mma_atom,
               MMAThrLayout     const& thr_layout   = {},
               Permutations     const& permutations = {})
{
  auto thr_layout_mnk  = append<3>(thr_layout, Layout<_1,_0>{});
  auto permutation_mnk = append<3>(permutations, _);

  return TiledMMA<MMA_Atom<MMA_Op>,
                  decltype(thr_layout_mnk),
                  decltype(permutation_mnk)>{mma_atom, thr_layout_mnk};
}

You're getting the following as a result

TiledMMA<AtomType,
         Layout<Shape<_1,_1,_1>>,       // AtomLayoutMNK
         Tile<Layout<Shape<_2,_2,_2>>,  // Permutation on M, (2,2,2):(1,2,4), identity of size 8
              Underscore,               // Permutation on N, noop identity
              Underscore>>              // Permutation on K, noop identity

So you can see that your Layout is becoming a rank-3 Tiler, but the permutations on each mode are all identities anyway and have little to no effect.

Hope that helps! I'll try to get explanations and examples like these into the documentation as soon as possible.

[hyhieu]

This is an amazing explanation. Thank you, Cris. Cannot wait to see more examples and read the documentation from you.

[MARD1NO]

Thanks for the question, my update to the CuTe documentation and examples is still pending but should be available soon.

The easiest way to think about it is that the Permutation parameter is a Tiler for the MNK modes of the MMA. That is, it is a set of three layouts that are applied to the M-mode, N-mode, and K-mode individually before applying the TV-layouts and projective slicing. These layouts can act to permute the M-mode, N-mode, and K-mode individually to make the TV-partitioning patterns more manageable/intuitive and effectively interleave individual MMAs.

I'll start with some examples and then get to your case. Let's start with SM80_8x8x4_F64F64F64F64_TN

    TiledMMA tiled_mma = make_tiled_mma(SM80_8x8x4_F64F64F64F64_TN{});
    print_latex(tiled_mma);

The above is equivalent to

    TiledMMA tiled_mma = make_tiled_mma(SM80_8x8x4_F64F64F64F64_TN{},
                                        Layout<Shape<_1,_1,_1>>{},
                                        Tile<_8,_8,_4>{});
    print_latex(tiled_mma);

as the atom already has a natural tile size of 8x8x4. But we can immediately expand this "tile size" up to 8x16x8 instead:

    TiledMMA tiled_mma = make_tiled_mma(SM80_8x8x4_F64F64F64F64_TN{},
                                        Layout<Shape<_1,_1,_1>>{},     // AtomLayout
                                        Tile<_8,_16,_8>{});            // Tiler
    print_latex(tiled_mma);

This doesn't actually affect the partitioning of input/output tensors because, by convention, only a single atom is ever partitioned out. It will affect the output of tile_size and get_layoutC_MN and get_layoutC_TV etc, which could affect any TiledCopy that rely on those partitioning patterns by being built on this TiledMMA. Regardless, you'll find the resulting tensors from partition_C etc to be exactly the same since the atom partitioning is exactly the same.

Continuing, I see those four values that T0 write to in C are separate, but I would prefer them to be next to each other. That is, I would like to permute the N-mode.

    TiledMMA tiled_mma = make_tiled_mma(SM80_8x8x4_F64F64F64F64_TN{},
                                        Layout<Shape<_1,_1,_1>>{},     // AtomLayout
                                        Tile<_8,                       // Permutation on M, equivalent to 8:1, identity
                                             Layout<Shape <_2,_4,_2>, 
                                                    Stride<_1,_4,_2>>, // Permutation on N, size 16
                                             _8>{});                   // Permutation on K, equivalent to 8:1, identity
    print_latex(tiled_mma);

That layout (2,4,2):(1,4,2) is read like a scatter permutation, telling the n-coords of the original image where to go in the new image: The first 2 elements stay put, the next 4 of those 2-element groups get sent to n-coord 4, and the next 2 of those 8-element groups get sent to n-coord 2. This permutes only the n-mode (in C and B accordingly) and brings the access of all threads to be contiguous in n-coordinates. This is convenient when designing layouts for shared memory or registers, for example. The two MMA instructions contained within the image above are now effectively interleaved in the logical n-coordinates.

Your example is doing something a bit silly, but perfectly functional.

using TiledMma = decltype(
  make_tiled_mma(
    MmaAtom{},
    Layout<Shape<_1, _1, _1>>{},  // this should be AtomLayoutMNK
    Layout<Shape<_2, _2, _2>>{}   // this should be PermutationMNK, but what does it do now?
  )
);

Because make_tiled_mma massages the inputs to get reasonable defaults by:

make_tiled_mma(MMA_Atom<MMA_Op> const& mma_atom,
               MMAThrLayout     const& thr_layout   = {},
               Permutations     const& permutations = {})
{
  auto thr_layout_mnk  = append<3>(thr_layout, Layout<_1,_0>{});
  auto permutation_mnk = append<3>(permutations, _);

  return TiledMMA<MMA_Atom<MMA_Op>,
                  decltype(thr_layout_mnk),
                  decltype(permutation_mnk)>{mma_atom, thr_layout_mnk};
}

You're getting the following as a result

TiledMMA<AtomType,
         Layout<Shape<_1,_1,_1>>,       // AtomLayoutMNK
         Tile<Layout<Shape<_2,_2,_2>>,  // Permutation on M, (2,2,2):(1,2,4), identity of size 8
              Underscore,               // Permutation on N, noop identity
              Underscore>>              // Permutation on K, noop identity

So you can see that your Layout is becoming a rank-3 Tiler, but the permutations on each mode are all identities anyway and have little to no effect.

Hope that helps! I'll try to get explanations and examples like these into the documentation as soon as possible.

Nice explanation! But I still have problem about this:

Layout<Shape <_2,_4,_2>, 
              Stride<_1,_4,_2>>

it seems it takes element idx is: 0 1 4 5 8 9 12 13 2 3 6 7 10 11 14 15

But in your case it need to take in n axis like: 0 1 8 9 2 3 10 11 4 5 12 13 6 7 14 15？。。。

[ccecka]

The layout you write is

Layout<Shape <_2,_2,_4>,
       Stride<_1,_8,_2>>
// 0 1 8 9 2 3 10 11 4 5 12 13 6 7 14 15

which I would describe as the "gather permutation" for the above, mapping the n-coords of the new image to their n-coords of the old image.

The layout I wrote is

Layout<Shape <_2,_4,_2>, 
       Stride<_1,_4,_2>>
// 0 1 4 5 8 9 12 13 2 3 6 7 10 11 14 15

which I would describe as the "scatter permutation" for the above, mapping the n-coords of the old image to their n-coords of the new image.

As you might expect from the descriptions, these are inverses of one another:

auto A = Layout<Shape<_2,_4,_2>,Stride<_1,_4,_2>>{};
auto B = Layout<Shape<_2,_2,_4>,Stride<_1,_8,_2>>{};
CUTE_STATIC_ASSERT_V(A == right_inverse(B));
CUTE_STATIC_ASSERT_V(B == right_inverse(A));

The TiledMMA takes the "scatter" version of these two.

[MARD1NO]

Oh! I understand it, so much thanks! :D

[wanghongyu2001]

Hi, AtomLayoutMNK mean what in this code

// @tparam MMA_Atom The MMA_Atom to use in the TiledMMA
// @tparam AtomLayoutMNK The MNK-tiling of the Atom to be performed.
// @tparam PermuationsMNK Permutations to apply to each MNK-mode before tiling for the Atom.
template <class MMA_Atom,
          class AtomLayoutMNK,
          class PermutationMNK = Tile<Underscore,Underscore,Underscore>>
struct TiledMMA : MMA_Atom { ... }

I didn't understand this sentence “AtomLayoutMNK means that the MMA_Atom is replicated into more threads, and so the CTA becomes accordingly. ”

[SimpleTheoryOfTypes]

which I would describe as the "scatter permutation" for the above, mapping the n-coords of the old image to their n-coords of the new image.

It seems that whether a construct in CUTLASS takes a "scatter permutation" version or a "gather permutation" version depends on the convention. Is there an efficient way for users to quickly identify the convention of a layout argument from a cutlass declaration (for example, in the case of TiledMMA, how to determine if its PermutationsMNK argument follows the "scatter permutation" without reading the actual implementation)?

[Ddd195]

How does this upper-right matrix represent its Layout in N dimension as Layout<Shape <_2,_4,_2>,Stride<_1,_4,_2>>

Thank you very much for your reply！
@ccecka

[thakkarV]

Have you read the mma atom readme in cute docs? It explains how these are transcribed

[Ddd195]

Thank you for reminding me! I found the answer I was looking for.
But it seems to use "new matrix" according to Layout<Shape <_2,_4,_2>, Stride<_1,_4,_2>> to get "old matrix", but normally, why not design the make_tiled_mma parameter to be an arrangement of the "old" to "new" matrix.
For instance, currently, given that I desire the "new" matrix where all the values of the T0 thread are arranged together, I am obliged to determine the permutation from the "old" to the "new" matrix by myself, namely 0 1 8 9 2 3 10 11 4 5 12 13 6 7 14 15. Then, via the inverse operation, obtain the 0, 1, 4, 5, 8, 9, 13, 2, 3, 6, 7, 10, 11, 14 15 layout, and subsequently fill this layout into make_tiled_mma.
Thanks for your reply.

[jcao-ai]

Is it right to think AtomLayoutMNK as intra-warp tiling, and PermutationMNK as inside-warp tiling ? Further more, permutation is intrinsic benefit brought by it.

[thakkarV]

That's a very Ampere centric view. Atom layout is just a tiling of atoms over "threads". In hopper the atom spans 128 threads.

[jcao-ai]

thanks, you are right.

[botbw]

This doesn't actually affect the partitioning of input/output tensors because, by convention, only a single atom is ever partitioned out. It will affect the output of tile_size and get_layoutC_MN and get_layoutC_TV etc, which could affect any TiledCopy that rely on those partitioning patterns by being built on this TiledMMA.

@ccecka Please correct me if I got it wrong: from my understanding, the current PermutationMNK is only for building TiledCopy from smem to reg but has no impact on MMA operation, and partition_X, gemm (dispatch 4) will correctly handle the previous ValLayoutMNK by deriving the val-repeat mode?

[botbw]

I could have been more clear -- The PermutationMNK affects the entire partitioning pattern of the MMA. Everything is affected (except perhaps the shape of the partitioned tensors) which includes strides, offsets, anything that is created downstream of MMAs like TiledCopy, etc. Everything must be consistent with the partitioning pattern of the MMA.

@ccecka Thanks for the explanation. Is there any reason for removing ValLayoutMNK as it doesn't seem to conflict with PermutationMNK?

[ccecka]

They are two alternate representations for the same abstract "TiledMMA" concept. The PermutationMNK version is a bit cheaper and more consistent with other Tiler concepts.

[botbw]

They are two alternate representations for the same abstract "TiledMMA" concept. The PermutationMNK version is a bit cheaper and more consistent with other Tiler concepts.

@ccecka I'm still a bit confused, wouldn't ValLayoutMNK affect partitioning and the number of allocated registers?

For example, if I adjust ValLayoutMNK of a TiledCopy, I see the left-most modes of partition_S/D outputs change accordingly; but when I adjust PermutationMNK, the shapes of partition_A/B/C outputs never change.

A solid case can be:

Change Tile<_32, _32, _16>{} to Tile<_128, _128, _64>{} here:

cutlass/examples/cute/tutorial/sgemm_sm80.cu

Line 376 in e94e888

Tile<_32,_32,_16>{}); // 32x32x16 Tiled MMA for LDSM

And the tiling changed from

tCgC : ((_2,_2),_4,(_2,_4)):((1024,_8),_32,(16384,32768))
tCrA : ((_2,_2),_4,(_2,_2,_2)):((_1,_2),_4,(_16,_32,_64))
tCrB : (_2,_8,(_2,_2,_2)):(_1,_2,(_16,_32,_64))
tCrC : ((_2,_2),_4,(_2,_4)):((_1,_2),_4,(_16,_32))

tXsA : ((_8,_1),_4,(_2,_2),(_1,_3)):((_1,_0),_2048,(144,288),(_0,_8192))
tXrA : (((_4,_2),_1),_4,_4):(((_1,_16),_0),_4,_32)
tXsB : ((_8,_1),_4,(_2,_2),(_1,_3)):((_1,_0),_2048,(144,288),(_0,_8192))
tXrB : (((_4,_2),_1),_4,_4):(((_1,_16),_0),_4,_32)

to

tCgC : ((_2,_2),_4,_8):((1024,_8),_32,16384)
tCrA : ((_2,_2),_4,(_2,_2,_2)):((_1,_2),_4,(_16,_32,_64))
tCrB : (_2,_8,(_2,_2,_2)):(_1,_2,(_16,_32,_64))
tCrC : ((_2,_2),_4,_8):((_1,_2),_4,_16)

tXsA : ((_8,(_2,(_2,_2,_2))),_1,_1,(_1,_3)):((_1,(_4096,(72,144,288))),_0,_0,(_0,_8192))
tXrA : ((_8,_16),_1,_1):((_1,_8),_0,_0)
tXsB : ((_8,(_2,(_2,_2,_2))),_1,_1,(_1,_3)):((_1,(_4096,(72,144,288))),_0,_0,(_0,_8192))
tXrB : ((_8,_16),_1,_1):((_1,_8),_0,_0)

I understand the left-most mode tXr(A|B) tiling becomes greater and rest(M|N|K) modes become 1 because it now handles the entire bM * bN block using a single TiledMMA, but why tCr(A|B|C) don't behave in the same way (I might expected flatten tCrC shape to be (atom_reg * val_repeat_m * val_repeat_n, tile_repeat_m, tile_repeat_n) == (4 * 4 * 8, 1, 1).

I tried to read through the code and inspect the usage of permutation.

cutlass/include/cute/atom/mma_atom.hpp

Lines 238 to 275 in e94e888

	// Tile a tensor or a layout from shape
	// (M,N,...)
	// to shape
	// ((ThrV,(ThrM,ThrN)),(FrgV,(RestM,RestN,...)))
	// where
	// ThrV: The threads local to an MMA. layout<0>(ThrLayoutVMNK): ThrV -> thread_idx
	// ThrM: The threads tiled in M. layout<1>(ThrLayoutVMNK): ThrM -> thread_idx
	// ThrN: The threads tiled in N. layout<2>(ThrLayoutVMNK): ThrN -> thread_idx
	// FrgV: The values local to an MMA.
	// RestM: The values tiled in M.
	// RestN: The values tiled in N.
	template <class CTensor>
	CUTE_HOST_DEVICE constexpr
	auto
	thrfrg_C(CTensor&& ctensor) const
	{
	CUTE_STATIC_ASSERT_V(rank(ctensor) >= Int<2>{});
	// Reorder the tensor for the TiledAtom
	auto t_tile = make_tile(permutation_mnk<0>(),
	permutation_mnk<1>());
	auto t_tensor = logical_divide(ctensor, t_tile); // (PermM,PermN)
	// Tile the tensor for the Atom
	auto c_tile = make_tile(make_layout(size<0>(AtomShape_MNK{})),
	make_layout(size<1>(AtomShape_MNK{})));
	auto c_tensor = zipped_divide(t_tensor, c_tile); // ((AtomM,AtomN),(RestM,RestN))
	// Transform the Atom mode from (M,K) to (Thr,Val)
	auto tv_tensor = c_tensor.compose(AtomLayoutC_TV{},_); // ((ThrV,FrgV),(RestM,RestN))
	// Tile the tensor for the C-threads
	auto thr_tile = make_tile(_,
	make_tile(make_layout(size<1>(thr_layout_vmnk_)),
	make_layout(size<2>(thr_layout_vmnk_))));
	auto thr_tensor = zipped_divide(tv_tensor, thr_tile); // ((ThrV,(ThrM,ThrN)),(FrgV,(RestM,RestN)))
	return thr_tensor;
	}

It seems like when partitioning tensors (C for example), only thread repeat thr_layout_vmnk_ is used, and there is no explicit logic to deal with value repeat tile_size_mnk (although they might appear in rest modes if input tensor shape is tile_size_mnk, but it totally depends on input tensor rather than its defination). From print_latex_mma, I can see that TiledMMA expands its value dimension, but how does this (or even will this) reflect in partitioning and register allocation?

Any insight or code hint will be appreciated.

[ccecka]

TiledMMA partitions for a single MMA Instruction while TiledCopy partitions for potentially multiple CPY instructions. When a TiledCopy is constructed from a TiledMMA, the TiledCopy is for the entire TiledMMA and not just one MMA instruction.

For instance, one LDSM can load data for 1 MMA, more than one MMA, or less than one MMA -- we want to guarantee that when a TiledCopy is executed, an entire tile of data is now available. The appropriate granularity is whatever is specified in the entire TiledMMA.

[botbw]

TiledMMA partitions for a single MMA Instruction while TiledCopy partitions for potentially multiple CPY instructions. When a TiledCopy is constructed from a TiledMMA, the TiledCopy is for the entire TiledMMA and not just one MMA instruction.

For instance, one LDSM can load data for 1 MMA, more than one MMA, or less than one MMA -- we want to guarantee that when a TiledCopy is executed, an entire tile of data is now available. The appropriate granularity is whatever is specified in the entire TiledMMA.

@ccecka I see. It's much clearer to me now, thanks a lot!

[cermeng]

@ccecka I'm wondering when PermuationsMNK would be practically used? While the MMA operation's thread/value layouts are fixed according to PTX instruction specifications, applying permutations can create inconsistent layouts with the PTX instruction expectations.

Could you provide a concrete use case where explicitly specifying PermutationsMNK becomes necessary since I hardly ever find one to use the param PermuationsMNK

[ccecka]

While the MMA operation's thread/value layouts are fixed according to PTX instruction specifications, applying permutations can create inconsistent layouts with the PTX instruction expectations.

This is false. By independently permuting the M-mode, N-mode, and K-mode of the partitioning pattern of a TiledMMA, the thread/value layouts fixed to PTX instructions become interleaved, but always valid.

Could you provide a concrete use case where explicitly specifying PermutationsMNK becomes necessary since I hardly ever find one to use the param PermuationsMNK

It's never necessary (the [inverse] permutation can be applied to SMEM instead of the MMA to yield an identical access pattern), but it can greatly simplify the definition of shared memory layouts for good access patterns -- consideration of vectorization and bank conflicts.

For example, it is used here:

cutlass/test/unit/gemm/device/default_gemm_configuration.hpp

Line 810 in b244379

Tile<Layout<Shape<_16,_2>,Stride<_2,_1>>, // 32x32x1 MMA with perm for load vectorization

and here

cutlass/test/unit/gemm/device/default_gemm_configuration.hpp

Line 1266 in b244379

Tile<Layout<Shape<_16,_2>,Stride<_2,_1>>, // 32x32x4 MMA with perm for load vectorization

to simplify the SMEM Layout definition (and bring the SMEM layout more in line with what was already familiar and used in CUTLASS 2.x).

[cermeng]

@ccecka I really appreciate your quick clarification! To confirm my understanding, I'd like to walk through an example.

Referring to your earlier example:

    TiledMMA tiled_mma = make_tiled_mma(SM80_8x8x4_F64F64F64F64_TN{},
                                        Layout<Shape<_1,_1,_1>>{},     // AtomLayout
                                        Tile<_8,_16,_8>{});            // Tiler
    TiledMMA tiled_mma_perm = make_tiled_mma(SM80_8x8x4_F64F64F64F64_TN{},
                                        Layout<Shape<_1,_1,_1>>{},     // AtomLayout
                                        Tile<_8,                       // Permutation on M, equivalent to 8:1, identity
                                             Layout<Shape <_2,_4,_2>, 
                                                    Stride<_1,_4,_2>>, // Permutation on N, size 16
                                             _8>{});        

Then the sub-matrix of C (highlighted as the gray box in the figure below) involved in a single MMA PTX instruction will differ between these two TiledMMA configurations, but they are both valid. Furthermore, the difference of sub-matrix is consistent with PermutationMNK. Correct me if I'm wrong.

[ccecka]

Exactly!