Skip to content

//xls/jit:function_jit_test flaky due to fuzz test timeouts #3347

@mikex-oss

Description

@mikex-oss

Describe the bug
The fuzz tests TestPackedArrayWithType and TestPackedTupleWithType are flaky when run in all configurations, in OSS and internally.

To Reproduce

bazel test -c opt //xls/jit:function_jit_test --runs_per_test=1000 --test_filter=FunctionJitTest.TestPackedArrayWithType --test_sharding_strategy=disabled --test_timeout=60

TIMEOUT in 63 out of 1000 in 75.3s

bazel test -c opt //xls/jit:function_jit_test --runs_per_test=1000 --test_filter=FunctionJitTest.TestPackedTupleWithType --test_sharding_strategy=disabled --test_timeout=60

TIMEOUT in 10 out of 1000 in 75.1s

Expected behavior
The tests are not flaky.

Additional context

Some diagnostics for flaky tests run internally with an extended timeout of 5 minutes produces the following results:

TestPackedArrayWithType (timeout 300s):

Worker  0 (-c opt):                      test passed in 942 out of 1000 runs.
Worker  1 (-c opt --config asan):        test passed in 932 out of 1000 runs.
Worker  2 (-c opt --config msan):        test passed in 898 out of 1000 runs.
Worker  3 (-c opt --config tsan):        test passed in 938 out of 1000 runs.
Worker  4 (-c opt --config ubsan):       test passed in 955 out of 1000 runs.

TestPackedTupleWithType (timeout 300s):

Worker  0 (-c opt):                      test passed in 997 out of 1000 runs.
Worker  1 (-c opt --config asan):        test passed in 995 out of 1000 runs.
Worker  2 (-c opt --config msan):        test passed in 949 out of 1000 runs.
Worker  3 (-c opt --config tsan):        test passed in 995 out of 1000 runs.
Worker  4 (-c opt --config ubsan):       test passed in 992 out of 1000 runs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions