Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate existing patch series to patchable #1005

Merged
merged 33 commits into from
Feb 27, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
196c36d
Convert Druid 30.0.0 patches to patchable
nightkr Feb 17, 2025
5c975e4
Update a few dates for Druid 26.0.0 patches
nightkr Feb 17, 2025
6d09f6d
Delete patches for unsupported Hadoop versions
nightkr Feb 17, 2025
c2dc228
Add patchable commands for getting the current patch and worktree dirs
nightkr Feb 17, 2025
ca1d0b8
More Druid date fixes
nightkr Feb 17, 2025
367377a
Make mailsplitting slightly more lenient
nightkr Feb 17, 2025
9298873
Migrate Hadoop 3.3.4 to patchable
nightkr Feb 17, 2025
8030aa6
Revise Hadoop 3.3.4 patch naming
nightkr Feb 18, 2025
0e97b59
Convert Hadoop 3.3.6 patches to Patchable
nightkr Feb 18, 2025
d51aa18
Make patchable discover images-repo specifically
nightkr Feb 18, 2025
6c154af
Add metadata to Hadoop 3.4.0 patches
nightkr Feb 18, 2025
d5ee460
Convert Hadoop 3.4.0 to patchable
nightkr Feb 18, 2025
ef49dfe
Hadoop 3.3.6 rewording
nightkr Feb 18, 2025
6cf0e71
Convert HBase 2.4.18 to patchable
nightkr Feb 18, 2025
81d193c
Convert HBase 2.6.0 to patchable
nightkr Feb 18, 2025
45057eb
Convert HBase 2.6.1 to patchable
nightkr Feb 18, 2025
728d962
Convert Hive 3.1.3 to patchable
nightkr Feb 18, 2025
029bcb0
Convert Hive 4.0.0 to patchable
nightkr Feb 18, 2025
da15609
Convert Kafka 3.7.1 to patchable
nightkr Feb 18, 2025
ad60339
Fix metadata for Kafka 3.7.2, 3.8.1, 3.9.0 patches
nightkr Feb 18, 2025
8f619f5
Convert omid to patchable
nightkr Feb 18, 2025
7c77250
Convert spark-k8s to patchable
nightkr Feb 18, 2025
656d6f7
Convert Superset to patchable
nightkr Feb 19, 2025
2dd16f8
Convert Trino to patchable
nightkr Feb 19, 2025
5b3385f
Convert trino-storage-connector to patchable
nightkr Feb 19, 2025
ddf9634
Convert Zookeeper to patchable
nightkr Feb 19, 2025
d03557b
Changelog
nightkr Feb 19, 2025
d202a27
Normalize git URLs
nightkr Feb 19, 2025
7b2ace7
Merge branch 'main' into feature/patchable-migration
nightkr Feb 20, 2025
9b3f5a7
Convert NiFi 1.27.0 to patchable
nightkr Feb 20, 2025
7e99abc
Normalize patchable metadata for NiFi 1.28.1 and 2.2.0
nightkr Feb 20, 2025
7149c0e
Remove duplicate subject tag from Druid 30.0.0 FMPP patch
nightkr Feb 27, 2025
a8866cb
Merge branch 'main' into feature/patchable-migration
nightkr Feb 27, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .patchable
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Marker file for Patchable to know that this is the image repository
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ All notable changes to this project will be documented in this file.
- trino-cli: Add version 470 ([#999]).
- trino-storage-connector: Add version 470 ([#999]).
- superset: Add version `4.1.1` ([#991]).
- Add Patchable patch management tool ([#1003], [#1007]).
- Add Patchable patch management tool ([#1003], [#1005], [#1007]).
- nifi: Add 1.28.1, 2.2.0 ([#1006]).

### Changed
Expand Down Expand Up @@ -78,6 +78,7 @@ All notable changes to this project will be documented in this file.
[#999]: https://github.com/stackabletech/docker-images/pull/999
[#1000]: https://github.com/stackabletech/docker-images/pull/1000
[#1003]: https://github.com/stackabletech/docker-images/pull/1003
[#1005]: https://github.com/stackabletech/docker-images/pull/1005
[#1006]: https://github.com/stackabletech/docker-images/pull/1006
[#1007]: https://github.com/stackabletech/docker-images/pull/1007

Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
From a8bec93ee6d0a4364676333168229aa0ec56657e Mon Sep 17 00:00:00 2001
From 098e0333cb3977164c62ab0f29aafaf9b1ac6c7c Mon Sep 17 00:00:00 2001
From: Lars Francke <[email protected]>
Date: Thu, 12 Dec 2024 17:59:17 +0100
Date: Wed, 10 Jul 2024 17:07:13 +0200
Subject: Removes all traces of the druid ranger extension

---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
From c19288cd84492d76f924152f2d4f0d0fc0499ed6 Mon Sep 17 00:00:00 2001
From cb547abec8bb002984bcb598c75f5031ea9513e1 Mon Sep 17 00:00:00 2001
From: Lars Francke <[email protected]>
Date: Thu, 12 Dec 2024 17:59:17 +0100
Date: Wed, 10 Jul 2024 17:07:13 +0200
Subject: Include Prometheus emitter in distribution

---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From 85cacbcc47c88a56acd60d91fbf0412040523c8d Mon Sep 17 00:00:00 2001
From 2722e90c01f02f804f1030f6aa91d07638e5e0a3 Mon Sep 17 00:00:00 2001
From: Lars Francke <[email protected]>
Date: Thu, 12 Dec 2024 17:59:17 +0100
Subject: Stop building unused extensions.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From 4229d1c0d096e10dce72929224a7b4c2284fb417 Mon Sep 17 00:00:00 2001
From 6a6cd8806bffe6b8f5da14d0d9f9b75fb79ac3cd Mon Sep 17 00:00:00 2001
From: Lars Francke <[email protected]>
Date: Thu, 12 Dec 2024 17:59:17 +0100
Subject: Updates all dependencies that have a new patch release available.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From d55895a2525286a5198a3b327c3ce503bc852ead Mon Sep 17 00:00:00 2001
From 1e4c0f607abfe1362941af5f53e04cd0b845f41f Mon Sep 17 00:00:00 2001
From: Lars Francke <[email protected]>
Date: Thu, 12 Dec 2024 17:59:17 +0100
Subject: Include jackson-dataformat-xml dependency.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From d1ae8732e2eee44abb5c831f5363c69e75e64a9a Mon Sep 17 00:00:00 2001
From c7d74ad665618125e09f365bae0ecaa2876b6a87 Mon Sep 17 00:00:00 2001
From: Lars Francke <[email protected]>
Date: Thu, 12 Dec 2024 17:59:17 +0100
Subject: Stop building the tar.gz distribution.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
From ff7d6a5ea07ea30653b47f6ef6844103a7ac3349 Mon Sep 17 00:00:00 2001
From 56541040ce6a5c36d53fdda71316d2a7ab8245c0 Mon Sep 17 00:00:00 2001
From: Lukas Voetmand <[email protected]>
Date: Thu, 12 Dec 2024 17:59:17 +0100
Date: Fri, 6 Sep 2024 17:53:52 +0200
Subject: Update CycloneDX plugin

---
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From bdd52ae32874b686d6ddfa3179f6af787444662f Mon Sep 17 00:00:00 2001
From a8442f203b70216cd7b3ec9cefe8f7627fa0d7e2 Mon Sep 17 00:00:00 2001
From: Malte Sander <[email protected]>
Date: Thu, 12 Dec 2024 17:59:17 +0100
Subject: Fix CVE-2024-36114
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From 736165ab0fe73e0bef765f2cfd21cd800baddbc1 Mon Sep 17 00:00:00 2001
From 2d634afe93690d295ddf69751b03e824cbd9f934 Mon Sep 17 00:00:00 2001
From: Lars Francke <[email protected]>
Date: Thu, 12 Dec 2024 06:35:21 +0100
Subject: Update FMPP version
Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
Removes all traces of the druid ranger extension

From b6665733d54d730bafcd4c238b11bedd4e412667 Mon Sep 17 00:00:00 2001
From: Lars Francke <[email protected]>

Date: Wed, 10 Jul 2024 17:07:13 +0200
Subject: Removes all traces of the druid ranger extension

---
0 files changed
distribution/pom.xml | 2 --
pom.xml | 1 -
2 files changed, 3 deletions(-)

diff --git a/distribution/pom.xml b/distribution/pom.xml
index 0f17a8c877..d7cd645767 100644
index bdbbd8b4c0..e27329e96d 100644
--- a/distribution/pom.xml
+++ b/distribution/pom.xml
@@ -252,8 +252,6 @@
Expand All @@ -20,10 +22,10 @@ index 0f17a8c877..d7cd645767 100644
<argument>-c</argument>
<argument>org.apache.druid.extensions:druid-catalog</argument>
diff --git a/pom.xml b/pom.xml
index cfca79dc6e..2acb812cbe 100644
index 9051ed24c5..3ab467e468 100644
--- a/pom.xml
+++ b/pom.xml
@@ -199,7 +199,6 @@
@@ -198,7 +198,6 @@
<module>extensions-core/simple-client-sslcontext</module>
<module>extensions-core/druid-basic-security</module>
<module>extensions-core/google-extensions</module>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
Include Prometheus emitter in distribution

From 0cee640900bb8119ba6be12911578927aeb191ed Mon Sep 17 00:00:00 2001
From: Lars Francke <[email protected]>

Date: Mon, 17 Feb 2025 16:42:34 +0100
Subject: Include Prometheus emitter in distribution

---
0 files changed
distribution/pom.xml | 46 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 46 insertions(+)

diff --git a/distribution/pom.xml b/distribution/pom.xml
index d7cd645767..eda1ddcfab 100644
index e27329e96d..d5918710ef 100644
--- a/distribution/pom.xml
+++ b/distribution/pom.xml
@@ -464,6 +464,52 @@
Expand Down
Original file line number Diff line number Diff line change
@@ -1,18 +1,20 @@
Stop building unused extensions.

From 62f340d66dae20da8d7566f992b56223d29d4174 Mon Sep 17 00:00:00 2001
From: Lars Francke <[email protected]>
Date: Mon, 17 Feb 2025 16:42:34 +0100
Subject: Stop building unused extensions.

By default Druid builds all community extensions and then discards them
while assembling the final distribution. This patch removes unused
extensions from the build.
---
0 files changed
pom.xml | 32 +-------------------------------
1 file changed, 1 insertion(+), 31 deletions(-)

diff --git a/pom.xml b/pom.xml
index 2acb812cbe..38e0ddc61a 100644
index 3ab467e468..171a98f803 100644
--- a/pom.xml
+++ b/pom.xml
@@ -201,39 +201,9 @@
@@ -200,39 +200,9 @@
<module>extensions-core/google-extensions</module>
<module>extensions-core/druid-catalog</module>
<module>extensions-core/testing-tools</module>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,19 @@
Updates all dependencies that have a new patch release available.

From 59ce2824b13ad95f728a5a5afc1e5b18ff547a38 Mon Sep 17 00:00:00 2001
From: Lars Francke <[email protected]>

Date: Mon, 17 Feb 2025 16:42:49 +0100
Subject: Updates all dependencies that have a new patch release available.

---
0 files changed
extensions-core/druid-pac4j/pom.xml | 5 +++-
extensions-core/kubernetes-extensions/pom.xml | 2 +-
extensions-core/orc-extensions/pom.xml | 2 +-
extensions-core/parquet-extensions/pom.xml | 2 +-
pom.xml | 29 ++++++++++---------
processing/pom.xml | 2 +-
6 files changed, 24 insertions(+), 18 deletions(-)

diff --git a/extensions-core/druid-pac4j/pom.xml b/extensions-core/druid-pac4j/pom.xml
index 282e0e5b15..523a2ca305 100644
index 88a570fd72..d3714ca0e2 100644
--- a/extensions-core/druid-pac4j/pom.xml
+++ b/extensions-core/druid-pac4j/pom.xml
@@ -38,7 +38,10 @@
Expand All @@ -23,10 +29,10 @@ index 282e0e5b15..523a2ca305 100644
</properties>

diff --git a/extensions-core/kubernetes-extensions/pom.xml b/extensions-core/kubernetes-extensions/pom.xml
index e3e77a99af..1304740ff3 100644
index 8bf105ea86..d445888da1 100644
--- a/extensions-core/kubernetes-extensions/pom.xml
+++ b/extensions-core/kubernetes-extensions/pom.xml
@@ -35,7 +35,7 @@
@@ -34,7 +34,7 @@
</parent>

<properties>
Expand All @@ -36,7 +42,7 @@ index e3e77a99af..1304740ff3 100644


diff --git a/extensions-core/orc-extensions/pom.xml b/extensions-core/orc-extensions/pom.xml
index b7eb007979..2c210c42c0 100644
index ade9bd8f37..db0ad77894 100644
--- a/extensions-core/orc-extensions/pom.xml
+++ b/extensions-core/orc-extensions/pom.xml
@@ -31,7 +31,7 @@
Expand All @@ -49,7 +55,7 @@ index b7eb007979..2c210c42c0 100644
<dependencies>
<dependency>
diff --git a/extensions-core/parquet-extensions/pom.xml b/extensions-core/parquet-extensions/pom.xml
index 371d2e7673..ad0b874db0 100644
index d3404a62d2..9cbb6d51d5 100644
--- a/extensions-core/parquet-extensions/pom.xml
+++ b/extensions-core/parquet-extensions/pom.xml
@@ -201,7 +201,7 @@
Expand All @@ -62,7 +68,7 @@ index 371d2e7673..ad0b874db0 100644
</profile>
</profiles>
diff --git a/pom.xml b/pom.xml
index 73fb14c1fc..26b24b8c14 100644
index 171a98f803..38c6b8dbb5 100644
--- a/pom.xml
+++ b/pom.xml
@@ -74,7 +74,7 @@
Expand Down Expand Up @@ -110,9 +116,9 @@ index 73fb14c1fc..26b24b8c14 100644
<resilience4j.version>1.3.1</resilience4j.version>
<slf4j.version>1.7.36</slf4j.version>
<jna.version>5.13.0</jna.version>
@@ -120,17 +120,17 @@
however it is required in some cases when running against mockito 4.x (mockito 4.x is required for Java <11.
We use the following property to pick the proper artifact based on Java version (see pre-java-11 profile) -->
@@ -119,17 +122,17 @@
<!-- mockito-inline artifact was removed in mockito 5.3 (mockito 5.x is required for Java >17),
however it is required in some cases when running against mockito 4.x (mockito 4.x is required for Java <11. We use the following property to pick the proper artifact based on Java version (see pre-java-11 profile) -->
<mockito.inline.artifact>core</mockito.inline.artifact>
- <aws.sdk.version>1.12.638</aws.sdk.version>
- <caffeine.version>2.8.0</caffeine.version>
Expand All @@ -134,10 +140,10 @@ index 73fb14c1fc..26b24b8c14 100644
<jdk.strong.encapsulation.argLine><!-- empty placeholder --></jdk.strong.encapsulation.argLine>
<jdk.security.manager.allow.argLine><!-- empty placeholder --></jdk.security.manager.allow.argLine>
diff --git a/processing/pom.xml b/processing/pom.xml
index affd900fe6..0daad4fa56 100644
index 3a62790fb0..c3afa9fd27 100644
--- a/processing/pom.xml
+++ b/processing/pom.xml
@@ -37,7 +37,7 @@
@@ -36,7 +36,7 @@
<sigar.base.version>1.6.5</sigar.base.version>
<sigar.version>${sigar.base.version}.132</sigar.version>
<ipaddress.version>5.3.4</ipaddress.version>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
Include jackson-dataformat-xml dependency.

From 062c9f9bf3b27752bb4546e468e3b48befc893fe Mon Sep 17 00:00:00 2001
From: Lars Francke <[email protected]>
Date: Mon, 17 Feb 2025 16:42:49 +0100
Subject: Include jackson-dataformat-xml dependency.

This allows us to use XmlLayout for Log4jV2.
By including it here as a dependency we can make sure that we always have
the matching version and we don't need to include it manually later in the
build.
---
0 files changed
server/pom.xml | 5 +++++
1 file changed, 5 insertions(+)

diff --git a/server/pom.xml b/server/pom.xml
index 410b51480e..b7dcf46111 100644
index ec2f4dec4f..3861f18ed1 100644
--- a/server/pom.xml
+++ b/server/pom.xml
@@ -205,6 +205,11 @@
Expand Down
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
Stop building the tar.gz distribution.

From c2a67767f6e06dcf7ea2bad4f7616cf69717b3b6 Mon Sep 17 00:00:00 2001
From: Lars Francke <[email protected]>
Date: Mon, 17 Feb 2025 16:42:49 +0100
Subject: Stop building the tar.gz distribution.

All we do is build Druid tar and gzip it only to immediately uncompress it
again. So, instead we just skip the compression step entirely.
---
distribution/src/assembly/assembly.xml | 2 +-
distribution/src/assembly/assembly.xml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/distribution/src/assembly/assembly.xml b/distribution/src/assembly/assembly.xml
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,17 @@
From eed0684b8097203e9d61c52093eb6dfe0960850d Mon Sep 17 00:00:00 2001
From: Lukas Voetmand <[email protected]>
Date: Fri, 6 Sep 2024 17:53:52 +0200
Subject: Update CycloneDX plugin

---
pom.xml | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/pom.xml b/pom.xml
index 9051ed2..10a2c85 100644
index 38c6b8dbb5..de1a3bddc4 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1728,7 +1728,11 @@
@@ -1700,7 +1700,11 @@
<plugin>
<groupId>org.cyclonedx</groupId>
<artifactId>cyclonedx-maven-plugin</artifactId>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
Fix CVE-2024-36114
From 0d9d6f564c52234c1eba4762c465a95f52cf8f0a Mon Sep 17 00:00:00 2001
From: Malte Sander <[email protected]>
Date: Thu, 12 Dec 2024 17:59:17 +0100
Subject: Fix CVE-2024-36114

see https://github.com/stackabletech/vulnerabilities/issues/834

Aircompressor is a library with ports of the Snappy, LZO, LZ4, and
Expand All @@ -17,12 +21,15 @@ have been fixed. When decompressing data from untrusted users, this can
be exploited for a denial-of-service attack by crashing the JVM, or to
leak other sensitive information from the Java process. There are no
known workarounds for this issue.
---
pom.xml | 6 ++++++
1 file changed, 6 insertions(+)

diff --git a/pom.xml b/pom.xml
index 9051ed24c5..e839295b61 100644
index de1a3bddc4..7beda6238f 100644
--- a/pom.xml
+++ b/pom.xml
@@ -283,6 +283,12 @@
@@ -255,6 +255,12 @@
</pluginRepositories>
<dependencyManagement>
<dependencies>
Expand Down
31 changes: 31 additions & 0 deletions druid/stackable/patches/30.0.0/0009-Update-FMPP-version.patch
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
From f6634ac02aa2fab11811915283e92c7b9ae1af5e Mon Sep 17 00:00:00 2001
From: Lars Francke <[email protected]>
Date: Thu, 12 Dec 2024 06:35:21 +0100
Subject: Update FMPP version

This is because FMPP Maven Plugin depends on FMPP in version 0.9.14
which itself depends on a Freemarker version that has not been pinned.
Instead it specifies a "range" which resolves to a SNAPSHOT version
which we don't want.
---
sql/pom.xml | 7 +++++++
1 file changed, 7 insertions(+)

diff --git a/sql/pom.xml b/sql/pom.xml
index 6669d43e52..1d02e4ef58 100644
--- a/sql/pom.xml
+++ b/sql/pom.xml
@@ -384,6 +384,13 @@
<plugin>
<groupId>com.googlecode.fmpp-maven-plugin</groupId>
<artifactId>fmpp-maven-plugin</artifactId>
+ <dependencies>
+ <dependency>
+ <groupId>net.sourceforge.fmpp</groupId>
+ <artifactId>fmpp</artifactId>
+ <version>0.9.16</version>
+ </dependency>
+ </dependencies>
<executions>
<execution>
<id>generate-fmpp-sources</id>
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
Fix CVE-2023-34455
From 4832e1270c2f541ad3724455034cbec394ba6263 Mon Sep 17 00:00:00 2001
From: Razvan-Daniel Mihai <[email protected]>
Date: Tue, 28 Jan 2025 17:29:59 +0100
Subject: Fix CVE-2023-34455

see https://github.com/stackabletech/vulnerabilities/issues/558

At the end of build process, Druid downloads dependencies directly from a remote
Expand All @@ -8,6 +12,9 @@ The hadoop client depends on a vulnerable version of the snappy library which
is then also downloaded even though a newer version is already on the system.

This patch removes the vulnerable jars.
---
distribution/pom.xml | 14 ++++++++++++++
1 file changed, 14 insertions(+)

diff --git a/distribution/pom.xml b/distribution/pom.xml
index d5918710ef..2d5bfc6ab4 100644
Expand Down
Loading