Skip to content

ngrams fails with a bare negative number when the result is too large #30806

Description

@SEPURI-SAI-KRISHNA

ngrams sizes its result array with int arithmetic, so the size wraps negative for large arrays and the query fails with a bare negative number as its only error message.

Repro

SELECT ngrams(repeat(1, 100000), 50000);
Query failed: -1794917296

That message is the NegativeArraySizeException text leaking through as the query failure; it gives no indication of what went wrong.

Cause

core/trino-main/src/main/java/io/trino/operator/scalar/ArrayNgramsFunction.java

int elementsPerRecord = toIntExact(min(array.getPositionCount(), n));
int totalRecords = array.getPositionCount() - elementsPerRecord + 1;
int[] ids = new int[totalRecords * elementsPerRecord];   // int multiplication

The element count is maximized when elementsPerRecord is about half the array length, giving roughly arrayLength^2 / 4. That exceeds Integer.MAX_VALUE once the array reaches about 92,000 entries. For the repro above:

elementsPerRecord = 50000
totalRecords      = 50001
exact product     = 2500050000
int product       = -1794917296   ->  NegativeArraySizeException

A 100,000-element array is reachable through ordinary SQL, since repeat permits up to MAX_RESULT_ENTRIES = 100_000.

Related existing handling

The sibling functions all guard their result size and fail with a categorized INVALID_FUNCTION_ARGUMENT:

  • combinationscheckCondition(combinationCount * (long) combinationLength <= MAX_RESULT_ELEMENTS, ..., "combinations exceed max size")
  • repeatMAX_RESULT_ENTRIES = 100_000
  • sequenceMAX_RESULT_ENTRIES = 10_000

ngrams has no equivalent guard.

Environment

Reproduced on master.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions