Do not outline elementwise operations

[newling]

What observations motivate this PR?

I compared the input.opt.ll files with and without coalescing. One difference I noticed was in the number of calls to the elementwise outlined function. Specifically, grepping in the output directory of the run.py script after 2 runs creating directories no_coalescing and with_coalescing:

grep "call void @generic_elementwise" no_coalescing/matmul_truncf_128_256_bf16_f32/input.opt.ll | wc -l
16
grep "call void @generic_elementwise" with_coalescing/matmul_truncf_128_256_bf16_f32/input.opt.ll | wc -l
0

Above notice that there are no function calls to the elementwise func when coalescing is enabled. i.e. the calls must have been inlined. Note that for matmul the number of calls is the same:

grep "call void @generic_matmul" no_coalescing/matmul_truncf_128_256_bf16_f32/input.opt.ll | wc -l
128
grep "call void @generic_matmul" with_coalescing/matmul_truncf_128_256_bf16_f32/input.opt.ll | wc -l
128

Background fact: we go out of program memory quite badly without coalescing (we hit 30 kB, much more than the allowed 16 kB). With coalescing, we fit.

So it would seem that inlining the function calls dramatically reduces the memory required. I can't explain this.

With the above observation, this PR removes function outlining for the elementwise ops, so that they are effectively always inline. With this change, program fits in memory even without coalescing.

[Abhishek-Varma]

This seems like a very good find @newling ! Thank you!

Another supporting argument that adds to what you mentioned above: since we don't outline linalg.fill (a producer fused to matmul - part of a single loop - PROLOGUE), there shouldn't be any need to outline an elementwise (a consumer fused to matmul - part of a single loop - EPILOGUE).

LGTM % one review comment.

[Abhishek-Varma]

No need for this example. If required, you can add one linalg.matmul to an already existing test case above.

[newling]

I like this test, I think it's good to have a sequence of tests which slowly change what's being tested from the preceding test, makes diagnosing failures easier. I should use a more descriptive name though...