TextFilter with multiple values?

[skttl]

I'm trying to do a text filter on a single field with multiple possible values like this:

new TextFilter(
    FieldName: "FullTextContent",
    Values: ["salmon", "samlon"],
    Negate: false
);

I'm expecting this to return results where FullTextContent contains either salmon or samlon. But I don't get any results.

If I do them individually (one per search) they do return the content correctly.

new TextFilter(
    FieldName: "FullTextContent",
    Values: ["salmon"],
    Negate: false
);

new TextFilter(
    FieldName: "FullTextContent",
    Values: ["samlon"],
    Negate: false
);

Am I expecting too much from the TextFilter, or am I doing it wrong?

[kjac]

Interesting. It should definitively yield results with either "salmon" OR "samlon".

I'll have someone (or me) look into this as soon as possible.

[Zeegaan]

This works for me with the source code atleast, I've made a test case to cover the issue here:

field names are often with lower case, unless you specifically added one manually via code(which looks like you did, just making sure here 😅 )

Here is my search on a site with the package installed aswell, which works:

[skttl]

Weird - I can't get it to do the same...

Heres the filters and results for a combined search:

And here is the same, with just salmon

Can you spot the difference?

I'm indexing FullTextContent like this:

public class FullTextContentIndexer(ICacheService cacheService, IOptions<FullTextSearchOptions> options) : IContentIndexer
{
    public async Task<IEnumerable<IndexField>> GetIndexFieldsAsync(IContentBase content, string?[] cultures, bool published, CancellationToken cancellationToken)
    {
        if (published is false 
            || options.Value.DisallowedContentTypeAliases.InvariantContains(content.ContentType.Alias) 
            || options.Value.DisallowedPropertyAliases.Any(x => content.GetValue<bool>(x)))
        {
            return [];
        }

        var allCultures = cultures.WhereNotNull();
        var fullTextContent = (await cacheService.GetFromCache(content.Id)).Where(x => allCultures.InvariantContains(x.Culture));
        
        var fullTextContentFields = fullTextContent
            .Select(x => new IndexField(
                options.Value.FullTextContentField, 
                new IndexValue { Texts = [x.Text] }, 
                x.Culture, 
                null
            )
        );

        return [..fullTextContentFields];

    }
}

[Zeegaan]

Might be different for something added with a IContentIndexer, I'll test it out 😁

[Zeegaan]

Nope, still works just fine for me 😅
I tried both with invariant and variant cultures, no custom indexing configuration or anything, just out of the box:

This was my indexer, i just indexed every time for simplicity here:

public class FullTextContentIndexer() : IContentIndexer
{
    public async Task<IEnumerable<IndexField>> GetIndexFieldsAsync(IContentBase content, string?[] cultures, bool published, CancellationToken cancellationToken)
    {
        return [new IndexField(
            "FulLTextContent", 
            new IndexValue { Texts = ["Salmon", "Something Else", "etc"] }, 
            "en-US", 
            null
        ),
        new IndexField(
            "FulLTextContent", 
            new IndexValue { Texts = ["Salmon", "Something Else", "etc"] }, 
            "da-DK", 
            null
        )];
    }
}

[skttl]

Could it be the order terms maybe? I have the samlon first, then salmon, you have salmon first.

What is the best way I can make a minimal reproduction?

A blank site with a single node, and then that content indexer?

And then some code to do the search?

[skttl]

I found it!

I had samlon in query, and then salmon and samlon in the text filter. I guess they have an AND relationship, so when samlon is not found in the document, it's skipped.

Is the score sorter used when only adding filters? The ordering is different now I nullified the query parameter.

[Zeegaan]

Aha that makes sense! 😁
They do have an AND relationship, as I would expect that if you search on "salmon", the filters will then further augment your search 💪

Just taking a look and textfilters aren't boosted it seems, it's "only" the query itself that gets boosted, here is the code for the query itself:

[skttl]

Just taking a look and textfilters aren't boosted it seems, it's "only" the query itself that gets boosted

Is that by design, or something that is just missing?

[Zeegaan]

I think it was just missed tbh, but I'm unsure if it's feasable! 😁
I'll create a task to look into if it's possible to boost text filters, it should be possible but you never know 🙈

[kjac]

I am working with Shannon (read: I put Shannon to work 🙈 ) to support boosting on text fields.

It's a Lucene query limitation that has been preventing this in the past, but it sounds like there are new options available moving forward, which might sort this out