Skip to content

Commit f826aca

Browse files
vahmed-hamdyvahmed-hamdy
committed
[FLINK-36651][Connectors/Elasticsearch] Fix IT test not compatible with 1.20, drop main branch tests for 1.19 and 1.18
1 parent da2ef1f commit f826aca

File tree

8 files changed

+29
-57
lines changed

8 files changed

+29
-57
lines changed

‎.github/workflows/push_pr.yml

Lines changed: 2 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -28,13 +28,8 @@ jobs:
2828
compile_and_test:
2929
strategy:
3030
matrix:
31-
flink: [ 1.18-SNAPSHOT ]
32-
jdk: [ '8, 11, 17' ]
33-
include:
34-
- flink: 1.19-SNAPSHOT
35-
jdk: '8, 11, 17, 21'
36-
- flink: 1.20-SNAPSHOT
37-
jdk: '8, 11, 17, 21'
31+
flink: [ 1.19-SNAPSHOT, 1.20-SNAPSHOT ]
32+
jdk: [ '8, 11, 17, 21' ]
3833
uses: apache/flink-connector-shared-utils/.github/workflows/ci.yml@ci_utils
3934
with:
4035
flink_version: ${{ matrix.flink }}

‎.github/workflows/weekly.yml

Lines changed: 4 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -30,27 +30,21 @@ jobs:
3030
strategy:
3131
matrix:
3232
flink_branches: [{
33-
flink: 1.18-SNAPSHOT,
34-
jdk: '8, 11, 17',
35-
branch: main
36-
}, {
3733
flink: 1.19-SNAPSHOT,
38-
jdk: '8, 11, 17, 21',
3934
branch: main
4035
}, {
4136
flink: 1.20-SNAPSHOT,
42-
jdk: '8, 11, 17, 21',
4337
branch: main
44-
}, {
45-
flink: 1.18.1,
46-
branch: v3.0
4738
}, {
4839
flink: 1.19.0,
4940
branch: v3.0
41+
}, {
42+
flink: 1.20.0,
43+
branch: main
5044
}]
5145
uses: apache/flink-connector-shared-utils/.github/workflows/ci.yml@ci_utils
5246
with:
5347
flink_version: ${{ matrix.flink_branches.flink }}
5448
connector_branch: ${{ matrix.flink_branches.branch }}
55-
jdk_version: ${{ matrix.flink_branches.jdk || '8, 11' }}
49+
jdk_version: ${{ matrix.flink_branches.jdk || '8, 11, 17, 21' }}
5650
run_dependency_convergence: false

‎flink-connector-elasticsearch-base/archunit-violations/61c75739-865e-4e9d-9da6-40bcde6a923c

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
Field <org.apache.flink.connector.elasticsearch.sink.ElasticsearchWriter$DefaultBulkResponseInspector.failureHandler> is annotated with <org.apache.flink.annotation.VisibleForTesting> in (ElasticsearchWriter.java:0)
2+
Method <org.apache.flink.connector.elasticsearch.sink.ElasticsearchSink.getBulkResponseInspectorFactory()> is annotated with <org.apache.flink.annotation.VisibleForTesting> in (ElasticsearchSink.java:0)
3+
Method <org.apache.flink.connector.elasticsearch.sink.ElasticsearchSink.getDeliveryGuarantee()> is annotated with <org.apache.flink.annotation.VisibleForTesting> in (ElasticsearchSink.java:0)
4+
Method <org.apache.flink.connector.elasticsearch.sink.ElasticsearchSink.getNetworkClientConfig()> is annotated with <org.apache.flink.annotation.VisibleForTesting> in (ElasticsearchSink.java:0)
5+
Method <org.apache.flink.connector.elasticsearch.sink.ElasticsearchSinkBuilderBase.build()> calls method <org.apache.flink.api.java.ClosureCleaner.clean(java.lang.Object, org.apache.flink.api.common.ExecutionConfig$ClosureCleanerLevel, boolean)> in (ElasticsearchSinkBuilderBase.java:370)
6+
Method <org.apache.flink.connector.elasticsearch.sink.ElasticsearchWriter.blockingFlushAllActions()> is annotated with <org.apache.flink.annotation.VisibleForTesting> in (ElasticsearchWriter.java:0)
7+
Method <org.apache.flink.connector.elasticsearch.table.IndexGeneratorFactory.createRuntimeIndexGenerator(java.lang.String, [Ljava.lang.String;, [Lorg.apache.flink.table.types.DataType;, org.apache.flink.connector.elasticsearch.table.IndexGeneratorFactory$IndexHelper, java.time.ZoneId)> has parameter of type <[Lorg.apache.flink.table.types.DataType;> in (IndexGeneratorFactory.java:0)
8+
Method <org.apache.flink.connector.elasticsearch.table.RowElasticsearchEmitter$1.getMetricGroup()> calls constructor <org.apache.flink.metrics.groups.UnregisteredMetricsGroup.<init>()> in (RowElasticsearchEmitter.java:74)

‎flink-connector-elasticsearch-base/archunit-violations/stored.rules

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,11 @@
11
#
2-
#Thu May 12 12:03:32 CEST 2022
2+
#Sat Nov 02 19:05:02 GMT 2024
3+
Return\ and\ argument\ types\ of\ methods\ annotated\ with\ @Public\ must\ be\ annotated\ with\ @Public.=06cb39ec-97e2-4e4e-8aa7-cd92504b759c
4+
Connector\ production\ code\ must\ not\ depend\ on\ non-public\ API\ outside\ of\ connector\ packages=325d89d0-9c19-495f-86df-7d609b6d63dd
5+
ITCASE\ tests\ should\ use\ a\ MiniCluster\ resource\ or\ extension=dd583797-83e1-414c-a38d-330773978813
36
Production\ code\ must\ not\ call\ methods\ annotated\ with\ @VisibleForTesting=4539e81b-9c72-4fd7-b7b3-ffc2e546b968
47
Options\ for\ connectors\ and\ formats\ should\ reside\ in\ a\ consistent\ package\ and\ be\ public\ API.=aca99410-c331-4bc2-a45b-1363eb8dd20c
58
Tests\ inheriting\ from\ AbstractTestBase\ should\ have\ name\ ending\ with\ ITCase=de342dd1-c974-42c9-8f64-ef182ba8c56d
6-
Classes\ in\ API\ packages\ should\ have\ at\ least\ one\ API\ visibility\ annotation.=143fca50-a3b9-4a4b-983e-40dd2c424f62
79
Return\ and\ argument\ types\ of\ methods\ annotated\ with\ @PublicEvolving\ must\ be\ annotated\ with\ @Public(Evolving).=09532d6b-7cb1-4621-90af-755558763875
8-
Return\ and\ argument\ types\ of\ methods\ annotated\ with\ @Public\ must\ be\ annotated\ with\ @Public.=06cb39ec-97e2-4e4e-8aa7-cd92504b759c
9-
ITCASE\ tests\ should\ use\ a\ MiniCluster\ resource\ or\ extension=dd583797-83e1-414c-a38d-330773978813
10-
Connector\ production\ code\ must\ not\ depend\ on\ non-public\ API\ outside\ of\ connector\ packages=325d89d0-9c19-495f-86df-7d609b6d63dd
10+
Classes\ in\ API\ packages\ should\ have\ at\ least\ one\ API\ visibility\ annotation.=143fca50-a3b9-4a4b-983e-40dd2c424f62
11+
Connector\ production\ code\ must\ depend\ only\ on\ public\ API\ when\ outside\ of\ connector\ packages=61c75739-865e-4e9d-9da6-40bcde6a923c

‎flink-connector-elasticsearch-base/src/test/java/org/apache/flink/connector/elasticsearch/sink/ElasticsearchWriterITCase.java

Lines changed: 2 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,6 @@
1717

1818
package org.apache.flink.connector.elasticsearch.sink;
1919

20-
import org.apache.flink.api.common.operators.MailboxExecutor;
2120
import org.apache.flink.api.connector.sink2.SinkWriter;
2221
import org.apache.flink.api.connector.sink2.SinkWriter.Context;
2322
import org.apache.flink.api.java.tuple.Tuple2;
@@ -31,14 +30,13 @@
3130
import org.apache.flink.metrics.groups.SinkWriterMetricGroup;
3231
import org.apache.flink.metrics.groups.UnregisteredMetricsGroup;
3332
import org.apache.flink.metrics.testutils.MetricListener;
33+
import org.apache.flink.runtime.mailbox.SyncMailboxExecutor;
3434
import org.apache.flink.runtime.metrics.MetricNames;
3535
import org.apache.flink.runtime.metrics.groups.InternalSinkWriterMetricGroup;
3636
import org.apache.flink.runtime.metrics.groups.UnregisteredMetricGroups;
3737
import org.apache.flink.runtime.testutils.MiniClusterResourceConfiguration;
3838
import org.apache.flink.test.junit5.MiniClusterExtension;
39-
import org.apache.flink.util.FlinkRuntimeException;
4039
import org.apache.flink.util.TestLoggerExtension;
41-
import org.apache.flink.util.function.ThrowingRunnable;
4240

4341
import org.apache.http.HttpHost;
4442
import org.elasticsearch.action.ActionListener;
@@ -329,7 +327,7 @@ private static ElasticsearchWriter<Tuple2<Integer, String>> createWriter(
329327
new DefaultBulkResponseInspector(),
330328
new NetworkClientConfig(null, null, null, null, null, null, null, null),
331329
metricGroup,
332-
new TestMailbox());
330+
new SyncMailboxExecutor());
333331
}
334332

335333
private TestingSinkWriterMetricGroup getSinkWriterMetricGroup() {
@@ -481,29 +479,4 @@ GetResponse getResponse(String index, int id) throws IOException {
481479
return client.get(new GetRequest(index, Integer.toString(id)), RequestOptions.DEFAULT);
482480
}
483481
}
484-
485-
private static class TestMailbox implements MailboxExecutor {
486-
487-
@Override
488-
public void execute(
489-
ThrowingRunnable<? extends Exception> command,
490-
String descriptionFormat,
491-
Object... descriptionArgs) {
492-
try {
493-
command.run();
494-
} catch (Exception e) {
495-
throw new RuntimeException("Unexpected error", e);
496-
}
497-
}
498-
499-
@Override
500-
public void yield() throws InterruptedException, FlinkRuntimeException {
501-
Thread.sleep(100);
502-
}
503-
504-
@Override
505-
public boolean tryYield() throws FlinkRuntimeException {
506-
return false;
507-
}
508-
}
509482
}

‎flink-connector-elasticsearch6/archunit-violations/f0b21088-63c4-4e1c-bcda-d54e4a224391

Whitespace-only changes.

‎flink-connector-elasticsearch6/archunit-violations/stored.rules

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,11 @@
11
#
2-
#Thu May 12 11:59:51 CEST 2022
2+
#Sat Nov 02 19:05:27 GMT 2024
3+
Return\ and\ argument\ types\ of\ methods\ annotated\ with\ @Public\ must\ be\ annotated\ with\ @Public.=82e2a33f-9c4b-49ed-95d5-2319a9ae5e1f
4+
Connector\ production\ code\ must\ not\ depend\ on\ non-public\ API\ outside\ of\ connector\ packages=a2f3489b-bdd7-430b-8723-6ac3502c173e
5+
ITCASE\ tests\ should\ use\ a\ MiniCluster\ resource\ or\ extension=db3972e4-f3a3-45b2-9643-27cba0cef09d
36
Production\ code\ must\ not\ call\ methods\ annotated\ with\ @VisibleForTesting=a4b5cf92-8e7d-43af-9d55-c693e352a3bc
47
Options\ for\ connectors\ and\ formats\ should\ reside\ in\ a\ consistent\ package\ and\ be\ public\ API.=795761fe-68cf-40d9-8ed8-bc6f47cb2292
58
Tests\ inheriting\ from\ AbstractTestBase\ should\ have\ name\ ending\ with\ ITCase=25e52d29-fa7e-42fa-a571-b5c76235df52
6-
Classes\ in\ API\ packages\ should\ have\ at\ least\ one\ API\ visibility\ annotation.=07e22212-0fdc-435a-9a8b-aca438213316
79
Return\ and\ argument\ types\ of\ methods\ annotated\ with\ @PublicEvolving\ must\ be\ annotated\ with\ @Public(Evolving).=3010cd85-0143-4cff-9095-1f24a1979f7f
8-
Return\ and\ argument\ types\ of\ methods\ annotated\ with\ @Public\ must\ be\ annotated\ with\ @Public.=82e2a33f-9c4b-49ed-95d5-2319a9ae5e1f
9-
ITCASE\ tests\ should\ use\ a\ MiniCluster\ resource\ or\ extension=db3972e4-f3a3-45b2-9643-27cba0cef09d
10-
Connector\ production\ code\ must\ not\ depend\ on\ non-public\ API\ outside\ of\ connector\ packages=a2f3489b-bdd7-430b-8723-6ac3502c173e
10+
Classes\ in\ API\ packages\ should\ have\ at\ least\ one\ API\ visibility\ annotation.=07e22212-0fdc-435a-9a8b-aca438213316
11+
Connector\ production\ code\ must\ depend\ only\ on\ public\ API\ when\ outside\ of\ connector\ packages=f0b21088-63c4-4e1c-bcda-d54e4a224391

‎pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ under the License.
5151
</modules>
5252

5353
<properties>
54-
<flink.version>1.18.0</flink.version>
54+
<flink.version>1.20.0</flink.version>
5555

5656
<jackson-bom.version>2.15.3</jackson-bom.version>
5757
<junit4.version>4.13.2</junit4.version>

0 commit comments

Comments
 (0)