Dremio merge 2025 05 21 09 39 (apache#72)

* main: Update docker.io/prom/prometheus Docker tag to v3.4.0 (apache#1602)

* Site: Update production configuration page (apache#1606)

* main: Update dependency com.google.cloud:google-cloud-storage-bom to v2.52.3 (apache#1623)

* main: Update dependency boto3 to v1.38.19 (apache#1622)

* Remove Bouncy Castle dependency usage from PemUtils (apache#1318)

- Added PEM format parsing in PemUtils
- Added unit test for PemUtils for empty file and multiple PEM objects
- Removed Bouncy Castle Provider dependency from service common module
- Removed Bouncy Castle Provider dependency from quarkus service module

* Site: Add a page for policy management (apache#1600)

* [Policy Store | Management Spec] Add policy privileges to spec and update admin service impl (apache#1529)

This PR adds new policy related privileges to polaris-management-api.yml and update PolarisAdminService to allow granting new privileges

* Spec: Add SigV4 Auth Support for Catalog Federation (apache#1506)

* Spec changes for SigV4 Auth Support for Catalog Federation

* Extract service identity info as a nested object

* nit: fix admin tool log level and comments (apache#1626)

The previous WARNING log levels seems to work, but WARN
aligns better with standard Quarkus log levels.

Fixes apache#1612

* Doc: switch to use iceberg-aws-bundle jar (apache#1609)

* main: Update dependency org.mockito:mockito-core to v5.18.0 (apache#1630)

* main: Update dependency boto3 to v1.38.20 (apache#1631)

* Require explicit user-consent to enable HadoopFileIO (apache#1532)

Using `HadoopFileIO` in Polaris can enable "hidden features" that users are likely not aware of. This change requires users to manually update the configuration to be able to use `HadoopFileIO` in way that highlights the consequences of enabling it.

This PR updates Polaris in multiple ways:
* The default of `SUPPORTED_CATALOG_STORAGE_TYPES` is changed to not include the `FILE` storage type.
* Respect the `ALLOW_SPECIFYING_FILE_IO_IMPL` configuration on namespaces, tables and views to prevent setting an `io-impl` value for anything but one of the configured, supported storage-types.
* Unify validation code in a new class `IcebergPropertiesValidation`.
* Using `FILE` or `HadoopFileIO` now _also_ requires the explicit configuration `ALLOW_INSECURE_STORAGE_TYPES_ACCEPTING_SECURITY_RISKS=true`.
* Added production readiness checks that trigger when `ALLOW_INSECURE_STORAGE_TYPES_ACCEPTING_SECURITY_RISKS` is `true` or `SUPPORTED_CATALOG_STORAGE_TYPES` contains `FILE` (defaults and per-realm).
* The two new readiness checks are considered _severe_. Severe readiness-errors prevent the server from starting up - unless the user explicitly configured `polaris.readiness.ignore-security-issues=true`.

Log messages and configuration options explicitly use "clear" phrases highlighting the consequences.

With these changes it is intentionally extremely difficult to start Polaris with HadoopFileIO. People who work around all these safety nets must have realized that what they are doing.

A lot of the test code relies on `FILE`/`HadoopFileIO`, those tests got all the configurations to let those tests continue to work as they are, bypassing the added security safeguards.

---------

Co-authored-by: Dmitri Bourlatchkov <[email protected]>

---------

Co-authored-by: Mend Renovate <[email protected]>
Co-authored-by: Yufei Gu <[email protected]>
Co-authored-by: David Handermann <[email protected]>
Co-authored-by: Honah (Jonas) J. <[email protected]>
Co-authored-by: Rulin Xing <[email protected]>
Co-authored-by: Dmitri Bourlatchkov <[email protected]>
Co-authored-by: MonkeyCanCode <[email protected]>

Author: 8 people

client/python/pyproject.toml

@@ -34,7 +34,7 @@ dependencies = [
     "python-dateutil>=2.8.2",
     "pydantic>=2.0.0",
     "typing-extensions>=4.7.1",
-    "boto3==1.38.18",
+    "boto3==1.38.20",
 ]
 
 [project.urls]

getting-started/eclipselink/docker-compose.yml

@@ -76,7 +76,7 @@ services:
       retries: 15
     command: [
       /opt/spark/bin/spark-sql,
-      --packages, "org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.9.0,software.amazon.awssdk:bundle:2.28.17,software.amazon.awssdk:url-connection-client:2.28.17,org.apache.iceberg:iceberg-gcp-bundle:1.9.0,org.apache.iceberg:iceberg-azure-bundle:1.9.0",
+      --packages, "org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.9.0,org.apache.iceberg:iceberg-aws-bundle:1.9.0,org.apache.iceberg:iceberg-gcp-bundle:1.9.0,org.apache.iceberg:iceberg-azure-bundle:1.9.0",
       --conf, "spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions",
       --conf, "spark.sql.catalog.quickstart_catalog=org.apache.iceberg.spark.SparkCatalog",
       --conf, "spark.sql.catalog.quickstart_catalog.type=rest",

getting-started/jdbc/docker-compose.yml

@@ -76,7 +76,7 @@ services:
       retries: 15
     command: [
       /opt/spark/bin/spark-sql,
-      --packages, "org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.9.0,software.amazon.awssdk:bundle:2.28.17,software.amazon.awssdk:url-connection-client:2.28.17,org.apache.iceberg:iceberg-gcp-bundle:1.9.0,org.apache.iceberg:iceberg-azure-bundle:1.9.0",
+      --packages, "org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.9.0,org.apache.iceberg:iceberg-aws-bundle:1.9.0,org.apache.iceberg:iceberg-gcp-bundle:1.9.0,org.apache.iceberg:iceberg-azure-bundle:1.9.0",
       --conf, "spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions",
       --conf, "spark.sql.catalog.polaris=org.apache.iceberg.spark.SparkCatalog",
       --conf, "spark.sql.catalog.polaris.type=rest",

getting-started/spark/notebooks/SparkPolaris.ipynb

@@ -256,7 +256,7 @@
     "\n",
     "spark = (SparkSession.builder\n",
     "  .config(\"spark.sql.catalog.spark_catalog\", \"org.apache.iceberg.spark.SparkSessionCatalog\")\n",
-    "  .config(\"spark.jars.packages\", \"org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.9.0,org.apache.hadoop:hadoop-aws:3.4.0,software.amazon.awssdk:bundle:2.23.19,software.amazon.awssdk:url-connection-client:2.23.19\")\n",
+    "  .config(\"spark.jars.packages\", \"org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.9.0,org.apache.iceberg:iceberg-aws-bundle:1.9.0\")\n",
     "  .config('spark.sql.iceberg.vectorization.enabled', 'false')\n",
     "         \n",
     "  # Configure the 'polaris' catalog as an Iceberg rest catalog\n",

getting-started/telemetry/docker-compose.yml

@@ -60,7 +60,7 @@ services:
     entrypoint: '/bin/sh -c "chmod +x /polaris/create-catalog.sh && /polaris/create-catalog.sh"'
 
   prometheus:
-    image: docker.io/prom/prometheus:v3.3.1
+    image: docker.io/prom/prometheus:v3.4.0
     ports:
       - "9093:9090"
     depends_on:

gradle/libs.versions.toml

@@ -43,7 +43,6 @@ assertj-core = { module = "org.assertj:assertj-core", version = "3.27.3" }
 auth0-jwt = { module = "com.auth0:java-jwt", version = "4.5.0" }
 awssdk-bom = { module = "software.amazon.awssdk:bom", version = "2.31.45" }
 azuresdk-bom = { module = "com.azure:azure-sdk-bom", version = "1.2.34" }
-bouncycastle-bcprov = { module = "org.bouncycastle:bcprov-jdk18on", version = "1.80" }
 caffeine = { module = "com.github.ben-manes.caffeine:caffeine", version = "3.2.0" }
 cel-bom = { module = "org.projectnessie.cel:cel-bom", version = "0.5.3" }
 commons-codec1 = { module = "commons-codec:commons-codec", version = "1.18.0" }
@@ -52,7 +51,7 @@ commons-text = { module = "org.apache.commons:commons-text", version = "1.13.1"
 docker-java-api = { module = "com.github.docker-java:docker-java-api", version = "3.5.0" }
 eclipselink = { module = "org.eclipse.persistence:eclipselink", version = "4.0.6" }
 errorprone = { module = "com.google.errorprone:error_prone_core", version = "2.38.0" }
-google-cloud-storage-bom = { module = "com.google.cloud:google-cloud-storage-bom", version = "2.52.2" }
+google-cloud-storage-bom = { module = "com.google.cloud:google-cloud-storage-bom", version = "2.52.3" }
 guava = { module = "com.google.guava:guava", version = "33.4.8-jre" }
 h2 = { module = "com.h2database:h2", version = "2.3.232" }
 dnsjava = { module = "dnsjava:dnsjava", version = "3.6.3" }
@@ -81,7 +80,7 @@ junit-bom = { module = "org.junit:junit-bom", version = "5.12.2" }
 logback-classic = { module = "ch.qos.logback:logback-classic", version = "1.5.18" }
 micrometer-bom = { module = "io.micrometer:micrometer-bom", version = "1.15.0" }
 microprofile-fault-tolerance-api = { module = "org.eclipse.microprofile.fault-tolerance:microprofile-fault-tolerance-api", version = "4.1.2" }
-mockito-core = { module = "org.mockito:mockito-core", version = "5.17.0" }
+mockito-core = { module = "org.mockito:mockito-core", version = "5.18.0" }
 mockito-junit-jupiter = { module = "org.mockito:mockito-junit-jupiter", version = "5.17.0" }
 mongodb-driver-sync = { module = "org.mongodb:mongodb-driver-sync", version = "5.5.0" }
 opentelemetry-bom = { module = "io.opentelemetry:opentelemetry-bom", version = "1.50.0" }

integration-tests/src/main/java/org/apache/polaris/service/it/env/ManagementApi.java

@@ -115,6 +115,16 @@ public void addGrant(String catalogName, String catalogRoleName, GrantResource g
     }
   }
 
+  public void revokeGrant(String catalogName, String catalogRoleName, GrantResource grant) {
+    try (Response response =
+        request(
+                "v1/catalogs/{cat}/catalog-roles/{role}/grants",
+                Map.of("cat", catalogName, "role", catalogRoleName))
+            .post(Entity.json(grant))) {
+      assertThat(response).returns(CREATED.getStatusCode(), Response::getStatus);
+    }
+  }
+
   public void grantCatalogRoleToPrincipalRole(
       String principalRoleName, String catalogName, CatalogRole catalogRole) {
     try (Response response =

integration-tests/src/main/java/org/apache/polaris/service/it/test/PolarisPolicyServiceIntegrationTest.java

@@ -18,7 +18,9 @@
  */
 package org.apache.polaris.service.it.test;
 
+import static jakarta.ws.rs.core.Response.Status.NOT_FOUND;
 import static org.apache.polaris.service.it.env.PolarisClient.polarisClient;
+import static org.assertj.core.api.Assertions.assertThat;
 
 import com.google.common.collect.ImmutableMap;
 import jakarta.ws.rs.client.Entity;
@@ -33,6 +35,7 @@
 import java.util.Map;
 import java.util.Optional;
 import java.util.UUID;
+import java.util.stream.Stream;
 import org.apache.iceberg.Schema;
 import org.apache.iceberg.catalog.Namespace;
 import org.apache.iceberg.catalog.TableIdentifier;
@@ -47,9 +50,16 @@
 import org.apache.polaris.core.admin.model.CatalogRole;
 import org.apache.polaris.core.admin.model.FileStorageConfigInfo;
 import org.apache.polaris.core.admin.model.GrantResource;
+import org.apache.polaris.core.admin.model.GrantResources;
+import org.apache.polaris.core.admin.model.NamespaceGrant;
+import org.apache.polaris.core.admin.model.NamespacePrivilege;
 import org.apache.polaris.core.admin.model.PolarisCatalog;
+import org.apache.polaris.core.admin.model.PolicyGrant;
+import org.apache.polaris.core.admin.model.PolicyPrivilege;
 import org.apache.polaris.core.admin.model.PrincipalWithCredentials;
 import org.apache.polaris.core.admin.model.StorageConfigInfo;
+import org.apache.polaris.core.admin.model.TableGrant;
+import org.apache.polaris.core.admin.model.TablePrivilege;
 import org.apache.polaris.core.catalog.PolarisCatalogHelpers;
 import org.apache.polaris.core.entity.CatalogEntity;
 import org.apache.polaris.core.policy.PredefinedPolicyTypes;
@@ -68,6 +78,7 @@
 import org.apache.polaris.service.types.PolicyAttachmentTarget;
 import org.apache.polaris.service.types.PolicyIdentifier;
 import org.assertj.core.api.Assertions;
+import org.assertj.core.api.InstanceOfAssertFactories;
 import org.junit.jupiter.api.AfterAll;
 import org.junit.jupiter.api.AfterEach;
 import org.junit.jupiter.api.BeforeAll;
@@ -86,6 +97,8 @@ public class PolarisPolicyServiceIntegrationTest {
       Optional.ofNullable(System.getenv("INTEGRATION_TEST_ROLE_ARN"))
           .orElse("arn:aws:iam::123456789012:role/my-role");
 
+  private static final String CATALOG_ROLE_1 = "catalogrole1";
+  private static final String CATALOG_ROLE_2 = "catalogrole2";
   private static final String EXAMPLE_TABLE_MAINTENANCE_POLICY_CONTENT = "{\"enable\":true}";
   private static final Namespace NS1 = Namespace.of("NS1");
   private static final Namespace NS2 = Namespace.of("NS2");
@@ -225,9 +238,9 @@ public void before(TestInfo testInfo) {
             extraPropertiesBuilder.build());
     CatalogGrant catalogGrant =
         new CatalogGrant(CatalogPrivilege.CATALOG_MANAGE_CONTENT, GrantResource.TypeEnum.CATALOG);
-    managementApi.createCatalogRole(currentCatalogName, "catalogrole1");
-    managementApi.addGrant(currentCatalogName, "catalogrole1", catalogGrant);
-    CatalogRole catalogRole = managementApi.getCatalogRole(currentCatalogName, "catalogrole1");
+    managementApi.createCatalogRole(currentCatalogName, CATALOG_ROLE_1);
+    managementApi.addGrant(currentCatalogName, CATALOG_ROLE_1, catalogGrant);
+    CatalogRole catalogRole = managementApi.getCatalogRole(currentCatalogName, CATALOG_ROLE_1);
     managementApi.grantCatalogRoleToPrincipalRole(
         principalRoleName, currentCatalogName, catalogRole);
 
@@ -487,6 +500,176 @@ NS2_T1, new Schema(Types.NestedField.optional(1, "string", Types.StringType.get(
     restCatalog.dropTable(NS2_T1);
   }
 
+  @Test
+  public void testGrantsOnPolicy() {
+    restCatalog.createNamespace(NS1);
+    try {
+      policyApi.createPolicy(
+          currentCatalogName,
+          NS1_P1,
+          PredefinedPolicyTypes.DATA_COMPACTION,
+          EXAMPLE_TABLE_MAINTENANCE_POLICY_CONTENT,
+          "test policy");
+      managementApi.createCatalogRole(currentCatalogName, CATALOG_ROLE_2);
+      Stream<PolicyGrant> policyGrants =
+          Arrays.stream(PolicyPrivilege.values())
+              .map(
+                  p ->
+                      new PolicyGrant(
+                          Arrays.asList(NS1.levels()),
+                          NS1_P1.getName(),
+                          p,
+                          GrantResource.TypeEnum.POLICY));
+      policyGrants.forEach(g -> managementApi.addGrant(currentCatalogName, CATALOG_ROLE_2, g));
+
+      Assertions.assertThat(managementApi.listGrants(currentCatalogName, CATALOG_ROLE_2))
+          .extracting(GrantResources::getGrants)
+          .asInstanceOf(InstanceOfAssertFactories.list(GrantResource.class))
+          .map(gr -> ((PolicyGrant) gr).getPrivilege())
+          .containsExactlyInAnyOrder(PolicyPrivilege.values());
+
+      PolicyGrant policyReadGrant =
+          new PolicyGrant(
+              Arrays.asList(NS1.levels()),
+              NS1_P1.getName(),
+              PolicyPrivilege.POLICY_READ,
+              GrantResource.TypeEnum.POLICY);
+      managementApi.revokeGrant(currentCatalogName, CATALOG_ROLE_2, policyReadGrant);
+
+      Assertions.assertThat(managementApi.listGrants(currentCatalogName, CATALOG_ROLE_2))
+          .extracting(GrantResources::getGrants)
+          .asInstanceOf(InstanceOfAssertFactories.list(GrantResource.class))
+          .map(gr -> ((PolicyGrant) gr).getPrivilege())
+          .doesNotContain(PolicyPrivilege.POLICY_READ);
+    } finally {
+      policyApi.purge(currentCatalogName, NS1);
+    }
+  }
+
+  @Test
+  public void testGrantsOnNonExistingPolicy() {
+    restCatalog.createNamespace(NS1);
+
+    try {
+      managementApi.createCatalogRole(currentCatalogName, CATALOG_ROLE_2);
+      Stream<PolicyGrant> policyGrants =
+          Arrays.stream(PolicyPrivilege.values())
+              .map(
+                  p ->
+                      new PolicyGrant(
+                          Arrays.asList(NS1.levels()),
+                          NS1_P1.getName(),
+                          p,
+                          GrantResource.TypeEnum.POLICY));
+      policyGrants.forEach(
+          g -> {
+            try (Response response =
+                managementApi
+                    .request(
+                        "v1/catalogs/{cat}/catalog-roles/{role}/grants",
+                        Map.of("cat", currentCatalogName, "role", "catalogrole2"))
+                    .put(Entity.json(g))) {
+
+              assertThat(response.getStatus()).isEqualTo(NOT_FOUND.getStatusCode());
+            }
+          });
+    } finally {
+      policyApi.purge(currentCatalogName, NS1);
+    }
+  }
+
+  @Test
+  public void testGrantsOnNamespace() {
+    restCatalog.createNamespace(NS1);
+    try {
+      managementApi.createCatalogRole(currentCatalogName, CATALOG_ROLE_2);
+      List<NamespacePrivilege> policyPrivilegesOnNamespace =
+          List.of(
+              NamespacePrivilege.POLICY_LIST,
+              NamespacePrivilege.POLICY_CREATE,
+              NamespacePrivilege.POLICY_DROP,
+              NamespacePrivilege.POLICY_WRITE,
+              NamespacePrivilege.POLICY_READ,
+              NamespacePrivilege.POLICY_FULL_METADATA,
+              NamespacePrivilege.NAMESPACE_ATTACH_POLICY,
+              NamespacePrivilege.NAMESPACE_DETACH_POLICY);
+      Stream<NamespaceGrant> namespaceGrants =
+          policyPrivilegesOnNamespace.stream()
+              .map(
+                  p ->
+                      new NamespaceGrant(
+                          Arrays.asList(NS1.levels()), p, GrantResource.TypeEnum.NAMESPACE));
+      namespaceGrants.forEach(g -> managementApi.addGrant(currentCatalogName, CATALOG_ROLE_2, g));
+
+      Assertions.assertThat(managementApi.listGrants(currentCatalogName, CATALOG_ROLE_2))
+          .extracting(GrantResources::getGrants)
+          .asInstanceOf(InstanceOfAssertFactories.list(GrantResource.class))
+          .map(gr -> ((NamespaceGrant) gr).getPrivilege())
+          .containsExactlyInAnyOrderElementsOf(policyPrivilegesOnNamespace);
+    } finally {
+      policyApi.purge(currentCatalogName, NS1);
+    }
+  }
+
+  @Test
+  public void testGrantsOnCatalog() {
+    managementApi.createCatalogRole(currentCatalogName, CATALOG_ROLE_2);
+    List<CatalogPrivilege> policyPrivilegesOnCatalog =
+        List.of(
+            CatalogPrivilege.POLICY_LIST,
+            CatalogPrivilege.POLICY_CREATE,
+            CatalogPrivilege.POLICY_DROP,
+            CatalogPrivilege.POLICY_WRITE,
+            CatalogPrivilege.POLICY_READ,
+            CatalogPrivilege.POLICY_FULL_METADATA,
+            CatalogPrivilege.CATALOG_ATTACH_POLICY,
+            CatalogPrivilege.CATALOG_DETACH_POLICY);
+    Stream<CatalogGrant> catalogGrants =
+        policyPrivilegesOnCatalog.stream()
+            .map(p -> new CatalogGrant(p, GrantResource.TypeEnum.CATALOG));
+    catalogGrants.forEach(g -> managementApi.addGrant(currentCatalogName, CATALOG_ROLE_2, g));
+
+    Assertions.assertThat(managementApi.listGrants(currentCatalogName, CATALOG_ROLE_2))
+        .extracting(GrantResources::getGrants)
+        .asInstanceOf(InstanceOfAssertFactories.list(GrantResource.class))
+        .map(gr -> ((CatalogGrant) gr).getPrivilege())
+        .containsExactlyInAnyOrderElementsOf(policyPrivilegesOnCatalog);
+  }
+
+  @Test
+  public void testGrantsOnTable() {
+    restCatalog.createNamespace(NS2);
+    try {
+      managementApi.createCatalogRole(currentCatalogName, CATALOG_ROLE_2);
+      restCatalog
+          .buildTable(
+              NS2_T1, new Schema(Types.NestedField.optional(1, "string", Types.StringType.get())))
+          .create();
+
+      List<TablePrivilege> policyPrivilegesOnTable =
+          List.of(TablePrivilege.TABLE_ATTACH_POLICY, TablePrivilege.TABLE_DETACH_POLICY);
+
+      Stream<TableGrant> tableGrants =
+          policyPrivilegesOnTable.stream()
+              .map(
+                  p ->
+                      new TableGrant(
+                          Arrays.asList(NS2.levels()),
+                          NS2_T1.name(),
+                          p,
+                          GrantResource.TypeEnum.TABLE));
+      tableGrants.forEach(g -> managementApi.addGrant(currentCatalogName, CATALOG_ROLE_2, g));
+
+      Assertions.assertThat(managementApi.listGrants(currentCatalogName, CATALOG_ROLE_2))
+          .extracting(GrantResources::getGrants)
+          .asInstanceOf(InstanceOfAssertFactories.list(GrantResource.class))
+          .map(gr -> ((TableGrant) gr).getPrivilege())
+          .containsExactlyInAnyOrderElementsOf(policyPrivilegesOnTable);
+    } finally {
+      policyApi.purge(currentCatalogName, NS2);
+    }
+  }
+
   private static ApplicablePolicy policyToApplicablePolicy(
       Policy policy, boolean inherited, Namespace parent) {
     return new ApplicablePolicy(

plugins/spark/README.md

@@ -31,7 +31,7 @@ and depends on iceberg-spark-runtime 1.9.0.
 # Build Plugin Jar
 A task createPolarisSparkJar is added to build a jar for the Polaris Spark plugin, the jar is named as:
 `polaris-iceberg-<icebergVersion>-spark-runtime-<sparkVersion>_<scalaVersion>-<polarisVersion>.jar`. For example:
-`polaris-iceberg-1.8.1-spark-runtime-3.5_2.12-0.10.0-beta-incubating-SNAPSHOT.jar`.
+`polaris-iceberg-1.9.0-spark-runtime-3.5_2.12-0.10.0-beta-incubating-SNAPSHOT.jar`.
 
 - `./gradlew :polaris-spark-3.5_2.12:createPolarisSparkJar` -- build jar for Spark 3.5 with Scala version 2.12.
 - `./gradlew :polaris-spark-3.5_2.13:createPolarisSparkJar` -- build jar for Spark 3.5 with Scala version 2.13.
@@ -53,7 +53,7 @@ jar, and to use the local Polaris server as a Catalog.
 ```shell
 bin/spark-shell \
 --jars <path-to-spark-client-jar> \
---packages org.apache.hadoop:hadoop-aws:3.4.0,io.delta:delta-spark_2.12:3.3.1 \
+--packages org.apache.iceberg:iceberg-aws-bundle:1.9.0,io.delta:delta-spark_2.12:3.3.1 \
 --conf spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions,io.delta.sql.DeltaSparkSessionExtension \
 --conf spark.sql.catalog.spark_catalog=org.apache.spark.sql.delta.catalog.DeltaCatalog \
 --conf spark.sql.catalog.<catalog-name>.warehouse=<catalog-name> \
@@ -67,13 +67,13 @@ bin/spark-shell \
 ```
 
 Assume the path to the built Spark client jar is
-`/polaris/plugins/spark/v3.5/spark/build/2.12/libs/polaris-iceberg-1.8.1-spark-runtime-3.5_2.12-0.10.0-beta-incubating-SNAPSHOT.jar`
+`/polaris/plugins/spark/v3.5/spark/build/2.12/libs/polaris-iceberg-1.9.0-spark-runtime-3.5_2.12-0.10.0-beta-incubating-SNAPSHOT.jar`
 and the name of the catalog is `polaris`. The cli command will look like following:
 
 ```shell
 bin/spark-shell \
---jars /polaris/plugins/spark/v3.5/spark/build/2.12/libs/polaris-iceberg-1.8.1-spark-runtime-3.5_2.12-0.10.0-beta-incubating-SNAPSHOT.jar \
---packages org.apache.hadoop:hadoop-aws:3.4.0,io.delta:delta-spark_2.12:3.3.1 \
+--jars /polaris/plugins/spark/v3.5/spark/build/2.12/libs/polaris-iceberg-1.9.0-spark-runtime-3.5_2.12-0.10.0-beta-incubating-SNAPSHOT.jar \
+--packages org.apache.iceberg:iceberg-aws-bundle:1.9.0,io.delta:delta-spark_2.12:3.3.1 \
 --conf spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions,io.delta.sql.DeltaSparkSessionExtension \
 --conf spark.sql.catalog.spark_catalog=org.apache.spark.sql.delta.catalog.DeltaCatalog \
 --conf spark.sql.catalog.polaris.warehouse=<catalog-name> \

plugins/spark/v3.5/getting-started/notebooks/SparkPolaris.ipynb

@@ -266,8 +266,8 @@
     "from pyspark.sql import SparkSession\n",
     "\n",
     "spark = (SparkSession.builder\n",
-    "  .config(\"spark.jars\", \"../polaris_libs/polaris-iceberg-1.8.1-spark-runtime-3.5_2.12-0.11.0-beta-incubating-SNAPSHOT.jar\")\n",
-    "  .config(\"spark.jars.packages\", \"org.apache.hadoop:hadoop-aws:3.3.4,io.delta:delta-spark_2.12:3.2.1\")\n",
+    "  .config(\"spark.jars\", \"../polaris_libs/polaris-iceberg-1.9.0-spark-runtime-3.5_2.12-0.11.0-beta-incubating-SNAPSHOT.jar\")\n",
+    "  .config(\"spark.jars.packages\", \"org.apache.iceberg:iceberg-aws-bundle:1.9.0,io.delta:delta-spark_2.12:3.2.1\")\n",
     "  .config(\"spark.sql.catalog.spark_catalog\", \"org.apache.spark.sql.delta.catalog.DeltaCatalog\")\n",
     "  .config('spark.sql.iceberg.vectorization.enabled', 'false')\n",
     "\n",