Adding hidden_states to generator state to enable embedding scenarios

[vortex-captain]

Description

This PR enables users to get hidden_states output as text embedding from language models by calling GenAI C# interface. (See #1127 for creating models with hidden_states output).

If "hidden_states" is specified as a non-empty string in genai_config["model"]["decoder"]["outputs"], an extra output will be added to the generator state via Generators::Logits. Otherwise The behavior of Logits remains unchanged.

Test

Convert a model with hidden_states output:

python -m onnxruntime_genai.models.builder -m Qwen/Qwen2.5-0.5B-Instruct -c <model-cache-dir> -e cpu -p int4 -o ./qwen_w_embedding --extra_options include_hidden_states=true

Item hidden_states is added to genai_config["model"]["decoder"]["outputs"]:

"outputs": {
    "hidden_states": "hidden_states",
    "logits": "logits",
    "present_key_names": "present.%d.key",
    "present_value_names": "present.%d.value"
},

Then, in C# code:

Generator generator = ...;
generator.GenerateNextToken();
generator.GetOutput("hidden_states");

[vortex-captain]

We might need an extra bool flag in genai_config to indicate whether to remove hidden_states from outputs