[AUTOCUT] Gradle Check Flaky Test Report for ClientYamlTestSuiteIT

[opensearch-ci-bot]

Flaky Test Report for ClientYamlTestSuiteIT

Noticed the ClientYamlTestSuiteIT has some flaky, failing tests that failed during post-merge actions.

Details

Git Reference	Merged Pull Request	Build Details	Test Name
0df6e9e	15098	43940	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=indices.delete_index_template/10_basic/Delete index template which is not used by data stream but index pattern matches}
c85ee68	15988	48303	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/360_date_histogram/Date histogram aggregation w/ shared field range test}
481fd38	16098	48529	org.opensearch.test.rest.ClientYamlTestSuiteIT.classMethod org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=indices.segments/10_basic/basic segments test}
d64c5f8	15866	47801	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/360_date_histogram/Date histogram aggregation w/ shared field range test}
2742e74	15700	47023	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/360_date_histogram/Date histogram aggregation w/ shared field range test}
7a54f71	14707	42236	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}
8b62948	13953	39797	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}
ed24a14	14881	43113	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}
fdd3ea7	15023	43647	org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}

The other pull requests, besides those involved in post-merge actions, that contain failing tests with the ClientYamlTestSuiteIT class are:

• 15024
• 14443
• 15124
• 15568
• 16294
• 15131
• 13853
• 13829
• 15333
• 15258
• 15922
• 13865
• 13788
• 13821
• 13620
• 14446
• 13232
• 13461
• 14383
• 15039
• 15012
• 15365
• 14069
• 14464
• 15545
• 14454
• 15289
• 14718
• 14774
• 15882
• 14613
• 14711
• 13573
• 14486
• 15194
• 15410
• 15509
• 15607
• 14817
• 15145
• 15256
• 15557
• 15603
• 13190
• 13929
• 14611
• 14630
• 15100
• 15153
• 15358
• 15381
• 15578
• 16038
• 14513
• 14415
• 12782
• 13772
• 13888
• 14118
• 14195
• 14684
• 15128
• 15326
• 15430
• 15649
• 15679

For more details on the failed tests refer to OpenSearch Gradle Check Metrics dashboard.

[getsaurabh02]

@prudhvigodithi can we confirm if these failures are pointing to an individual test or the overall test class?

[getsaurabh02]

Also, can we highlight the branch the test is failing on, in the issue description?

[prudhvigodithi]

Hey @getsaurabh02 today the gradle check runs on a commit ref, the issue description, the markdown table created has the commit that has the flaky failure. Here for this issue the commit 8b62948 has a flaky failure for the test class ClientYamlTestSuiteIT and the failing test is org.opensearch.test.rest.ClientYamlTestSuiteIT.test {p0=search.aggregation/20_terms/string profiler via global ordinals}.

Sometimes there are can be more than one test failing for the same commit or for the different one, here is the example #14381, in that case we group them under one issue created for a specific test class name.

[sandeshkr419]

@finnegancarroll do you want to take a look on these failures?

[sandeshkr419]

@prudhvigodithi Does the ci-bot closes out flaky tests if they don't fail for a certain time / runs?

[prudhvigodithi]

Hey @sandeshkr419 the automation wont re-open the issue once closed. So the idea is once the flaky tests are fixed, the user can close the issue and automation wont re-open if there are no new failures with respect to same flaky test. For now we can close the issue and monitor this flaky test issue.
Thanks