Not nullable array Hits are sometimes null

[hesehus-bizzkit-mibu]

Elastic.Clients.Elasticsearch version:
8.17.3

Elasticsearch version:
8.17.2

.NET runtime version:
net9.0

Operating system version:
Docker/Windows

Description of the problem including expected versus actual behavior:
This might seem like an edge case but since two different teams ran in to this little gotcha individually. I would just make sure you where aware of it.

Accessing SearchResponse<T>.Hits may cause System.NullReferenceException

Steps to reproduce:
Simplified project to show behavior:

NullableElasticClient.csproj

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net9.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Elastic.Clients.Elasticsearch" Version="8.17.3" />
  </ItemGroup>

</Project>

Program.cs

using Elastic.Clients.Elasticsearch;
using Elastic.Clients.Elasticsearch.QueryDsl;
using Elastic.Transport;
using System.Text.Json;

var settings = new ElasticsearchClientSettings(new Uri("https://localhost:9200"))
    .DisableDirectStreaming()
    .Authentication(new BasicAuthentication("elastic", "YOUR_ELASTIC_PASSWORD"));

var client = new ElasticsearchClient(settings);

await client.Indices.CreateAsync("my_index");

var response1 = await client.SearchAsync<MyDoc>(s => s
    .Index("my_index")
    .Query(q => q.MatchAll(new MatchAllQuery()))
);

Console.WriteLine(JsonSerializer.Serialize(response1.Hits));

var response2 = await client.SearchAsync<MyDoc>(s => s
    .Index("my_index")
    .Query(q => q.MatchAll(new MatchAllQuery()))
    .FilterPath("hits.hits._source")
);

try
{
    Console.WriteLine(JsonSerializer.Serialize(response2.Hits));
}
catch (Exception e)
{
    Console.WriteLine("Error:"+e.Message);
}

await client.IndexAsync(new MyDoc { Id = 42, Name="name" }, i => i
            .Index("my_index")
            .Refresh(Refresh.True));

var response3 = await client.SearchAsync<MyDoc>(s => s
    .Index("my_index")
    .Query(q => q.MatchAll(new MatchAllQuery()))
    .FilterPath("hits.hits._source")
);

Console.WriteLine(JsonSerializer.Serialize(response3.Hits));

class MyDoc
{
    public int Id { get; set; }
    public required string Name { get; set; }
}

Expected behavior

When the result set is empty in query 2 the expected output is [] but instead the exception is thrown. This example is of cause simplified but there exist valid reasons for filtering aggressively in returned data.

Looking at

using System.Collections.Generic;
using System.Linq;
using System.Text.Json.Serialization;

namespace Elastic.Clients.Elasticsearch;

public partial class SearchResponse<TDocument>
{
	[JsonIgnore]
	public IReadOnlyCollection<Core.Search.Hit<TDocument>> Hits => HitsMetadata.Hits;

	[JsonIgnore]
	public IReadOnlyCollection<TDocument> Documents => HitsMetadata.Hits.Select(s => s.Source).ToReadOnlyCollection();

	[JsonIgnore]
	public long Total => HitsMetadata?.Total?.Item1?.Value ?? HitsMetadata?.Total?.Item2 ?? -1;
}

It would be expected that Hits would guard against null similar to Total.

---

Thanks for a great product.

[flobernd]

Hi @hesehus-bizzkit-mibu , thanks for providing this context.

I guess it won't hurt to guard against null here.

Just FYI: Using FilterPath always involves a certain risk since you actively remove properties from the response. This might even fundamentally break the client in some cases.