[SYCL] Extract args directly from kernel if we can

[Pennycook]

In some cases, all values that need to be passed as kernel arguments are stored within the kernel function object, and their offsets can be calculated using the integration header or equivalent built-ins. In such cases, we can therefore set kernel arguments directly without staging via MArgs.

This first attempt is limited to the simplest cases where all kernel arguments are either standard layout types or pointers. It may be possible to extend this approach to cover other cases, but only if some classes are redesigned.

The implementation currently stores some information (e.g., the number of kernel arguments) inside of the handler, because there is no way to access the kernel type within handler::finalize().

---

Some notes for reviewers:

• This depends on the new hasSpecialCaptures functionality introduced in [SYCL] Add hasSpecialCaptures() constexpr function #18386, which returns true for kernels that only capture standard layout classes and pointers.

• There are some seemingly unrelated changes in kernel_desc.hpp and to some of the unit tests. These changes were necessary because hasSpecialCaptures requires getParamDesc to be constexpr. I think this wasn't picked up during [SYCL] Add hasSpecialCaptures() constexpr function #18386 because hasSpecialCaptures wasn't previously being run for every kernel.

• I'm not really satisfied by the solution of adding a lot more member variables, but it was the best way I could think of to limit the scope of the changes required. Long-term, it would be better to try and move everything (including the complicated cases) to extract everything directly from the lambda, to design an abstraction that unifies the MArgs and MKernelFuncPtr paths, or to find a way to access the required values without them being stored in the handler (e.g., using something like [SYCL] Optimize kernel name based cache lookup #18081).

[Pennycook]

This is failing with:

# .---command stdout------------
# | There are new symbols in the new library. It is a non-breaking change. Refer to sycl/doc/developer/ABIPolicyGuide.md for further instructions.
# | The following symbols are new to the object file:
# | 
# | ?prepareForDirectArgumentCopy@handler@_V1@sycl@@AEAAXPEBXHP6A?AUkernel_param_desc_t@detail@23@H@Z@Z
# `-----------------------------

I've never added anything to handler before, and the intent here wasn't to make this function part of the public API. What did I do wrong, and what do I have to change?

[sergey-semenov]

I've never added anything to handler before, and the intent here wasn't to make this function part of the public API. What did I do wrong, and what do I have to change?

The added function is private, so it's not part of the API, but since it's defined in the library source code it becomes part of the ABI. I wouldn't say that's a problem (e.g. it allows applications to benefit from some changes to that function without recompilation), but you can move the definition to the header if you'd like to keep it out of the library ABI boundary.

[aelovikov-intel]

I'd prefer we use an std::variant to represent that.

[aelovikov-intel]

This is wrong in its current version, I think. Line 768 might have left KernelFunc in moved-out state. How exactly is this beneficial? We've already created a copy in MHostKernel, what extra overhead are you trying to eliminate?

Edit: is #18413 doing exactly that?

[Pennycook]

Exactly, this is why I started looking at #18413.

I'm going to work on merging these two PRs today. The end result should be that:

• We defer extracting arguments from the kernel until we reach handler::finalize()
• We skip extracting arguments from the kernel if it's a simple case

What I'm trying to eliminate is the overhead of:

• Creating an MArgs vector
• Checking that MArgs is sorted
• Copying the arguments into MArgs before they're passed to UR

[aelovikov-intel]

Can we delay args extraction (by passing a callback from the user's code maybe) until here, so that non-special arguments would naturally be copied directly by the nature of the modified implementation?