Skip to content

#757 Add support for transparent GPG/PGP decryption of files on read #868

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yruslan merged 6 commits into from
Aug 5, 2026
Merged

#757 Add support for transparent GPG/PGP decryption of files on read #868

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
e035225
#757 Add initial support for reading GPG (PGP) encrypted files via Bo…
yruslan Aug 3, 2026
64fa1d7
#757 Add `gpg_private_key` and `gpg_private_key_passphrase` options t…
yruslan Aug 4, 2026
e4a699a
#757 Fix BouncyCastle provider resolution when shaded, and close the …
yruslan Aug 5, 2026
1d1332c
#757 Fix Hadoop FSInputStream leak for GPG-encrypted files. Ensure th…
yruslan Aug 5, 2026
2559553
#757 Add a test for reading GPG-encrypted variable-record-length file…
yruslan Aug 5, 2026
b799427
#757 Fix another potential leek of Hadoop FSInputStream in BufferedFs…
yruslan Aug 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 23 additions & 7 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1678,17 +1678,33 @@ You can chain multiple transformers by providing transformer classes as a comma-
3. Custom transformers run before binary properties are calculated. So inserting or removing fields changes the copybook
layout.

### GPG Transparent Decryption (Experimental)

Cobrix supports transparent decryption of GPG/PGP-encrypted files. You just need to provide the content of ASCII-armored
private key, and the passphrase (if non-empty).

```scala
val df = spark.read
.format("cobol")
.option("copybook", someCopybook)
.option("gpg_private_key", gpgPrivateKey)
.option("gpg_private_key_passphrase", gpgPrivateKeyPassphrase)
.load("/some/path")
```

## Summary of all available options

##### File reading options

| Option (usage example) | Description |
|----------------------------------------|:---------------------------------------------------------------------------------------------------------------|
| .option("data_paths", "/path1,/path2") | Allows loading data from multiple unrelated paths on the same filesystem. |
| .option("file_start_offset", "0") | Specifies the number of bytes to skip at the beginning of each file. |
| .option("file_end_offset", "0") | Specifies the number of bytes to skip at the end of each file. |
| .option("record_start_offset", "0") | Specifies the number of bytes to skip at the beginning of each record before applying copybook fields to data. |
| .option("record_end_offset", "0") | Specifies the number of bytes to skip at the end of each record after applying copybook fields to data. |
| Option (usage example) | Description |
|-----------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------|
| .option("data_paths", "/path1,/path2") | Allows loading data from multiple unrelated paths on the same filesystem. |
| .option("file_start_offset", "0") | Specifies the number of bytes to skip at the beginning of each file. |
| .option("file_end_offset", "0") | Specifies the number of bytes to skip at the end of each file. |
| .option("record_start_offset", "0") | Specifies the number of bytes to skip at the beginning of each record before applying copybook fields to data. |
| .option("record_end_offset", "0") | Specifies the number of bytes to skip at the end of each record after applying copybook fields to data. |
| .option("gpg_private_key", ascGpgKeyContent) | Specifies the ASCII-armored GPG private key for decrypting data files. |
| .option("gpg_private_key_passphrase", "passphrase") | Specifies the passphrase for the ASCII-armored GPG private key used for decrypting data files. If not specified, empty passphrase will be used. |
Comment on lines +1699 to +1707

@coderabbitai coderabbitai Bot Aug 4, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Use an h3 heading for this subsection.

Line [1697] jumps from ## Summary of all available options to ##### File reading options. Change it to ### File reading options so the document outline complies with markdownlint MD001.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@README.md` around lines 1699 - 1707, Change the “File reading options”
subsection heading in the README from level five to level three, using “### File
reading options” beneath “## Summary of all available options” to maintain the
required heading hierarchy.

Source: Linters/SAST tools


##### Copybook parsing options

Expand Down
2 changes: 2 additions & 0 deletions build.sbt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,8 @@ lazy val assemblySettings = Seq(
assembly / assemblyShadeRules:= Seq(
// Spark may rely on a different version of ANTLR runtime. Renaming the package helps avoid the binary incompatibility
ShadeRule.rename("org.antlr.**" -> "za.co.absa.cobrix.cobol.parser.shaded.org.antlr.@1").inAll,
// Environments might rely on different versions of GPG support
ShadeRule.rename("org.bouncycastle.**" -> "za.co.absa.cobrix.cobol.parser.shaded.org.bouncycastle.@1").inAll,
// The SLF4j API and implementation are provided by Spark
ShadeRule.zap("org.slf4j.**").inAll
),
Expand Down
4 changes: 4 additions & 0 deletions cobol-parser/src/main/scala/za/co/absa/cobrix/cobol/reader/parameters/CobolParameters.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ import za.co.absa.cobrix.cobol.reader.policies.SchemaRetentionPolicy.SchemaReten
* @param asciiCharset A charset for ASCII data
* @param fieldCodePage Specifies a mapping between a field name and the code page
* @param isUtf16BigEndian If true UTF-16 is considered big-endian.
* @param gpgPrivateKey GPG private key content
* @param gpgPrivateKeyPassphrase GPG private key passphrase
* @param floatingPointFormat A format of floating-point numbers
* @param recordStartOffset A number of bytes to skip at the beginning of the record before parsing a record according to a copybook
* @param recordEndOffset A number of bytes to skip at the end of each record
Expand Down Expand Up @@ -84,6 +86,8 @@ case class CobolParameters(
asciiCharset: Option[String],
fieldCodePage: Map[String, String],
isUtf16BigEndian: Boolean,
gpgPrivateKey: Option[String],
gpgPrivateKeyPassphrase: Option[String],
floatingPointFormat: FloatingPointFormat,
recordStartOffset: Int,
recordEndOffset: Int,
Expand Down
9 changes: 9 additions & 0 deletions ...rser/src/main/scala/za/co/absa/cobrix/cobol/reader/parameters/CobolParametersParser.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,8 @@ object CobolParametersParser extends Logging {
val PARAM_RECORD_TRAILER_NAME2 = "file_trailer_field"
val PARAM_IS_XCOM = "is_xcom"
val PARAM_IS_TEXT = "is_text"
val PARAM_GPG_PRIVATE_KEY = "gpg_private_key"
val PARAM_GPG_PRIVATE_KEY_PASSPHRASE = "gpg_private_key_passphrase"

// Schema transformation parameters
val PARAM_GENERATE_RECORD_ID = "generate_record_id"
Expand Down Expand Up @@ -302,6 +304,9 @@ object CobolParametersParser extends Logging {
)
}

val gpgPrivateKey = params.get(PARAM_GPG_PRIVATE_KEY)
val gpgPrivateKeyPassphraseOpt = params.get(PARAM_GPG_PRIVATE_KEY_PASSPHRASE)

val variableSizeOccursPolicy = VariableSizeOccursPolicy(params.getOrElse(PARAM_VARIABLE_SIZE_OCCURS, "false"))

val writerParameters = if (isWriter) {
Expand All @@ -326,6 +331,8 @@ object CobolParametersParser extends Logging {
asciiCharset,
getFieldCodepageMap(params),
params.getOrElse(PARAM_IS_UTF16_BIG_ENDIAN, "true").toBoolean,
gpgPrivateKey,
gpgPrivateKeyPassphraseOpt,
getFloatingPointFormat(params),
params.getOrElse(PARAM_RECORD_START_OFFSET, "0").toInt,
params.getOrElse(PARAM_RECORD_END_OFFSET, "0").toInt,
Expand Down Expand Up @@ -493,6 +500,8 @@ object CobolParametersParser extends Logging {
asciiCharset = parameters.asciiCharset,
fieldCodePage = parameters.fieldCodePage,
isUtf16BigEndian = parameters.isUtf16BigEndian,
gpgPrivateKey = parameters.gpgPrivateKey,
gpgPrivateKeyPassphrase = parameters.gpgPrivateKeyPassphrase,
floatingPointFormat = parameters.floatingPointFormat,
redefineRuleExpressions = ruleExpressionMap,
variableSizeOccurs = parameters.variableSizeOccurs,
Expand Down
4 changes: 4 additions & 0 deletions cobol-parser/src/main/scala/za/co/absa/cobrix/cobol/reader/parameters/ReaderParameters.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ import za.co.absa.cobrix.cobol.reader.policies.SchemaRetentionPolicy.SchemaReten
* @param asciiCharset A charset for ASCII data
* @param fieldCodePage Specifies a mapping between a field name and the code page
* @param isUtf16BigEndian If true UTF-16 strings are considered big-endian.
* @param gpgPrivateKey GPG private key content
* @param gpgPrivateKeyPassphrase GPG private key passphrase
* @param floatingPointFormat A format of floating-point numbers
* @param redefineRuleExpressions A map of REDEFINE field names to expressions that determine which redefine alternative to use when parsing records.
* @param variableSizeOccurs Specifies how to handle OCCURS DEPENDING ON when the actual number of elements in arrays is less than the maximum array size
Expand Down Expand Up @@ -95,6 +97,8 @@ case class ReaderParameters(
asciiCharset: Option[String] = None,
fieldCodePage: Map[String, String] = Map.empty[String, String],
isUtf16BigEndian: Boolean = true,
gpgPrivateKey: Option[String] = None,
gpgPrivateKeyPassphrase: Option[String] = None,
floatingPointFormat: FloatingPointFormat = FloatingPointFormat.IBM,
redefineRuleExpressions: Map[String, ExpressionEvaluator] = Map.empty,
variableSizeOccurs: VariableSizeOccursPolicy = VariableSizeOccursPolicy.MaxSize,
Expand Down
37 changes: 37 additions & 0 deletions data/test41_copybook.cob
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
****************************************************************************
* *
* Copyright 2018 ABSA Group Limited *
* *
* Licensed under the Apache License, Version 2.0 (the "License"); *
* you may not use this file except in compliance with the License. *
* You may obtain a copy of the License at *
* *
* http://www.apache.org/licenses/LICENSE-2.0 *
* *
* Unless required by applicable law or agreed to in writing, software *
* distributed under the License is distributed on an "AS IS" BASIS, *
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. *
* See the License for the specific language governing permissions and *
* limitations under the License. *
* *
****************************************************************************

01 RECORD.
05 ID PIC S9(4) COMP.
05 COMPANY.
10 SHORT-NAME PIC X(10).
10 COMPANY-ID-NUM PIC 9(5) COMP-3.
10 COMPANY-ID-STR
REDEFINES COMPANY-ID-NUM PIC X(3).
05 METADATA.
10 CLIENTID PIC X(15).
10 REGISTRATION-NUM PIC X(10).
10 NUMBER-OF-ACCTS PIC 9(03) COMP-3.
10 ACCOUNT.
12 ACCOUNT-DETAIL OCCURS 80
DEPENDING ON NUMBER-OF-ACCTS.
15 ACCOUNT-NUMBER PIC X(24).
15 ACCOUNT-TYPE-N PIC 9(5) COMP-3.
15 ACCOUNT-TYPE-X REDEFINES
ACCOUNT-TYPE-N PIC X(3).

Binary file added data/test41_data/example.bin.gpg
View file
Open in desktop
Binary file not shown.
10 changes: 10 additions & 0 deletions data/test41_expected/test41a.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{"ID":1,"COMPANY":{"SHORT_NAME":"FOO INCORP","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":1,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"000000000000001100220033","ACCOUNT_TYPE_N":0,"ACCOUNT_TYPE_X":""}]}}}
{"ID":2,"COMPANY":{"SHORT_NAME":"BARCOMPANY","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":1,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"002000000022004000010001","ACCOUNT_TYPE_N":0,"ACCOUNT_TYPE_X":""}]}}}
{"ID":3,"COMPANY":{"SHORT_NAME":"EXAMPLE.CO","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":1,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"000000000000001234567890","ACCOUNT_TYPE_N":0,"ACCOUNT_TYPE_X":""}]}}}
{"ID":4,"COMPANY":{"SHORT_NAME":"EXAMPLE330","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":2,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"000000000000009876543210","ACCOUNT_TYPE_N":0,"ACCOUNT_TYPE_X":""},{"ACCOUNT_NUMBER":"000000000000001234555561","ACCOUNT_TYPE_N":1,"ACCOUNT_TYPE_X":""}]}}}
{"ID":5,"COMPANY":{"SHORT_NAME":"EXAMPLE3","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":1,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"000000012131415161718192","ACCOUNT_TYPE_N":0,"ACCOUNT_TYPE_X":""}]}}}
{"ID":6,"COMPANY":{"SHORT_NAME":"EXAMPLE4","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":3,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"000000000000002000400012","ACCOUNT_TYPE_N":0,"ACCOUNT_TYPE_X":""},{"ACCOUNT_NUMBER":"000000000000003000400102","ACCOUNT_TYPE_N":1,"ACCOUNT_TYPE_X":""},{"ACCOUNT_NUMBER":"000000005006001200301000","ACCOUNT_TYPE_N":2,"ACCOUNT_TYPE_X":""}]}}}
{"ID":7,"COMPANY":{"SHORT_NAME":"EXAMPLE7","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":2,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"000000100423412301203120","ACCOUNT_TYPE_N":0,"ACCOUNT_TYPE_X":""},{"ACCOUNT_NUMBER":"000000000030928973981723","ACCOUNT_TYPE_N":1,"ACCOUNT_TYPE_X":""}]}}}
{"ID":8,"COMPANY":{"SHORT_NAME":"FOOBAR8","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":3,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"000000389871238792010200","ACCOUNT_TYPE_N":0,"ACCOUNT_TYPE_X":""},{"ACCOUNT_NUMBER":"000000036719283719283713","ACCOUNT_TYPE_N":1,"ACCOUNT_TYPE_X":""},{"ACCOUNT_NUMBER":"000001992837819827389172","ACCOUNT_TYPE_N":2,"ACCOUNT_TYPE_X":""}]}}}
{"ID":9,"COMPANY":{"SHORT_NAME":"DUMMY_CO9","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":1,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"000000731928300100002312","ACCOUNT_TYPE_N":0,"ACCOUNT_TYPE_X":""}]}}}
{"ID":10,"COMPANY":{"SHORT_NAME":"NEWEXCOM10","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":2,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"000000004909239000000233","ACCOUNT_TYPE_N":2,"ACCOUNT_TYPE_X":""},{"ACCOUNT_NUMBER":"000000000984120003123900","ACCOUNT_TYPE_N":1,"ACCOUNT_TYPE_X":""}]}}}
10 changes: 10 additions & 0 deletions data/test41_expected/test41b.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
{"File_Id":0,"Record_Id":0,"Record_Byte_Length":2202,"ID":1,"COMPANY":{"SHORT_NAME":"FOO INCORP","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":1,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"000000000000001100220033","ACCOUNT_TYPE_N":0,"ACCOUNT_TYPE_X":""}]}}}
{"File_Id":0,"Record_Id":1,"Record_Byte_Length":2202,"ID":2,"COMPANY":{"SHORT_NAME":"BARCOMPANY","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":1,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"002000000022004000010001","ACCOUNT_TYPE_N":0,"ACCOUNT_TYPE_X":""}]}}}
{"File_Id":0,"Record_Id":2,"Record_Byte_Length":2202,"ID":3,"COMPANY":{"SHORT_NAME":"EXAMPLE.CO","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":1,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"000000000000001234567890","ACCOUNT_TYPE_N":0,"ACCOUNT_TYPE_X":""}]}}}
{"File_Id":0,"Record_Id":3,"Record_Byte_Length":2202,"ID":4,"COMPANY":{"SHORT_NAME":"EXAMPLE330","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":2,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"000000000000009876543210","ACCOUNT_TYPE_N":0,"ACCOUNT_TYPE_X":""},{"ACCOUNT_NUMBER":"000000000000001234555561","ACCOUNT_TYPE_N":1,"ACCOUNT_TYPE_X":""}]}}}
{"File_Id":0,"Record_Id":4,"Record_Byte_Length":2202,"ID":5,"COMPANY":{"SHORT_NAME":"EXAMPLE3","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":1,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"000000012131415161718192","ACCOUNT_TYPE_N":0,"ACCOUNT_TYPE_X":""}]}}}
{"File_Id":0,"Record_Id":5,"Record_Byte_Length":2202,"ID":6,"COMPANY":{"SHORT_NAME":"EXAMPLE4","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":3,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"000000000000002000400012","ACCOUNT_TYPE_N":0,"ACCOUNT_TYPE_X":""},{"ACCOUNT_NUMBER":"000000000000003000400102","ACCOUNT_TYPE_N":1,"ACCOUNT_TYPE_X":""},{"ACCOUNT_NUMBER":"000000005006001200301000","ACCOUNT_TYPE_N":2,"ACCOUNT_TYPE_X":""}]}}}
{"File_Id":0,"Record_Id":6,"Record_Byte_Length":2202,"ID":7,"COMPANY":{"SHORT_NAME":"EXAMPLE7","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":2,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"000000100423412301203120","ACCOUNT_TYPE_N":0,"ACCOUNT_TYPE_X":""},{"ACCOUNT_NUMBER":"000000000030928973981723","ACCOUNT_TYPE_N":1,"ACCOUNT_TYPE_X":""}]}}}
{"File_Id":0,"Record_Id":7,"Record_Byte_Length":2202,"ID":8,"COMPANY":{"SHORT_NAME":"FOOBAR8","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":3,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"000000389871238792010200","ACCOUNT_TYPE_N":0,"ACCOUNT_TYPE_X":""},{"ACCOUNT_NUMBER":"000000036719283719283713","ACCOUNT_TYPE_N":1,"ACCOUNT_TYPE_X":""},{"ACCOUNT_NUMBER":"000001992837819827389172","ACCOUNT_TYPE_N":2,"ACCOUNT_TYPE_X":""}]}}}
{"File_Id":0,"Record_Id":8,"Record_Byte_Length":2202,"ID":9,"COMPANY":{"SHORT_NAME":"DUMMY_CO9","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":1,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"000000731928300100002312","ACCOUNT_TYPE_N":0,"ACCOUNT_TYPE_X":""}]}}}
{"File_Id":0,"Record_Id":9,"Record_Byte_Length":2202,"ID":10,"COMPANY":{"SHORT_NAME":"NEWEXCOM10","COMPANY_ID_NUM":0,"COMPANY_ID_STR":""},"METADATA":{"CLIENTID":"","REGISTRATION_NUM":"","NUMBER_OF_ACCTS":2,"ACCOUNT":{"ACCOUNT_DETAIL":[{"ACCOUNT_NUMBER":"000000004909239000000233","ACCOUNT_TYPE_N":2,"ACCOUNT_TYPE_X":""},{"ACCOUNT_NUMBER":"000000000984120003123900","ACCOUNT_TYPE_N":1,"ACCOUNT_TYPE_X":""}]}}}
8 changes: 8 additions & 0 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,7 @@
<mockito.version>4.11.0</mockito.version>
<scodec_core.version>1.11.10</scodec_core.version>
<slf4j.version>1.7.25</slf4j.version>
<bouncycastle.version>1.84</bouncycastle.version>
</properties>

<dependencies>
Expand Down Expand Up @@ -225,6 +226,13 @@
<scope>test</scope>
</dependency>

<!-- GPG -->
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcpg-jdk18on</artifactId>
<version>${bouncycastle.version}</version>
</dependency>

<!-- Test scope dependencies -->
<dependency>
<groupId>org.scalatest</groupId>
Expand Down
4 changes: 4 additions & 0 deletions project/Dependencies.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ object Dependencies {
private val antlrValue = "4.9.3"
private val slf4jVersion = "1.7.25"
private val jacksonVersion = "2.15.4"
private val bouncycastleVersion = "1.84"

private val scalatestVersion = "3.2.19"
private val mockitoVersion = "4.11.0"
Expand Down Expand Up @@ -56,6 +57,9 @@ object Dependencies {
"org.apache.spark" %% "spark-sql" % sparkVersion(scalaVersion) % Provided,
"org.apache.spark" %% "spark-streaming" % sparkVersion(scalaVersion) % Provided,

// libraries
"org.bouncycastle" % "bcpg-jdk18on" % bouncycastleVersion,

// test
"org.scalatest" %% "scalatest" % scalatestVersion % Test,
"org.mockito" % "mockito-core" % mockitoVersion % Test
Expand Down
6 changes: 6 additions & 0 deletions spark-cobol/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,12 @@
<version>${project.version}</version>
</dependency>

<!-- GPG support -->
<dependency>
<groupId>org.bouncycastle</groupId>
<artifactId>bcpg-jdk18on</artifactId>
</dependency>

Comment on lines +52 to +57

@coderabbitai coderabbitai Bot Aug 4, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Inspect shade/relocation configuration in both POMs.
fd -t f 'pom.xml' | while IFS= read -r f; do
  echo "=== $f ==="
  rg -n -C8 'maven-shade-plugin|<relocation>|<relocations>|shadedPattern|bouncycastle' "$f"
done

Repository: AbsaOSS/cobrix

Length of output: 3790

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "=== root pom relevant sections ==="
sed -n '90,220p' pom.xml
sed -n '350,520p' pom.xml

echo "=== spark-cobol pom relevant sections ==="
sed -n '1,120p' spark-cobol/pom.xml

echo "=== build.sbt relevant shading lines ==="
sed -n '140,165p' build.sbt

echo "=== all occurrences of bouncycastle relocate-like references ==="
rg -n -i 'relocate|relocations|bouncycastle|bcpg-jdk18on|bcprov' . -g '!target' -g '!\.git' -g '!out'

Repository: AbsaOSS/cobrix

Length of output: 13347

Add BouncyCastle relocation to the Maven shade configuration.

build.sbt relocates org.bouncycastle.**, but the root POM’s maven-shade-plugin configuration only has <execution> and no <relocations>. Add the same BouncyCastle relocation for Maven so the released spark-cobol jar does not expose BouncyCastle classes that can collide with the cluster classpath.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@spark-cobol/pom.xml` around lines 52 - 57, Add an org.bouncycastle.**
relocation rule to the root POM’s maven-shade-plugin configuration, alongside
the existing execution settings, matching the relocation already defined in
build.sbt. Ensure the shaded spark-cobol artifact rewrites all BouncyCastle
packages without changing the dependency declaration.

<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
Expand Down
Loading
Loading