Skip to content

google_biglake_iceberg_table: cannot create a partitioned table, because the API now requires field-id and the provider treats it as output-only#29136

Description

@Ferdinanddb

Community Note

  • Please vote on this issue by adding a 馃憤 reaction to the original issue to help the community and maintainers prioritize this request.
  • Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment.
  • If an issue is assigned to a user, that user is claiming responsibility for the issue.
  • Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Terraform Version & Provider Version(s)

OpenTofu v1.11.5
provider registry.terraform.io/hashicorp/google v7.46.0

I also reproduced this against hashicorp/google v8.0.0. The resource schema is identical on this point.

Affected Resource(s)

google_biglake_iceberg_table

Terraform Configuration

resource "google_biglake_iceberg_catalog" "example" {
  name              = "example-catalog"
  catalog_type      = "CATALOG_TYPE_GCS_BUCKET"
  default_warehouse = "gs://example-bucket"
}

resource "google_biglake_iceberg_namespace" "example" {
  catalog      = google_biglake_iceberg_catalog.example.name
  namespace_id = "example_namespace"
}

resource "google_biglake_iceberg_table" "example" {
  catalog   = google_biglake_iceberg_catalog.example.name
  namespace = google_biglake_iceberg_namespace.example.namespace_id
  name      = "example_table"

  schema {
    fields {
      id       = 1
      name     = "id"
      type     = "long"
      required = false
    }
    fields {
      id       = 2
      name     = "event_date"
      type     = "date"
      required = false
    }
  }

  # This is the example shape from the registry documentation, copied as-is.
  partition_spec {
    fields {
      name      = "event_date"
      source_id = 2
      transform = "identity"
    }
  }
}

Debug Output

The create request fails at the API:

Error: Error creating IcebergTable: googleapi: Error 400: Failed to parse partition spec: field-id missing for partition field event_date

  with google_biglake_iceberg_table.example,
  on main.tf line 20, in resource "google_biglake_iceberg_table" "example":
  20: resource "google_biglake_iceberg_table" "example" {

The catalog and the namespace are created successfully in the same apply. Only the table fails.

Expected Behavior

The table is created. Either the provider assigns an id to each partition field (Iceberg's convention is 1000 + index, which is what other Iceberg clients do), or field_id becomes settable so that the configuration can supply it.

Actual Behavior

The API rejects the create because the partition spec carries no field-id, and there is no way to provide one from the configuration.

field_id is declared output-only in the generator, in mmv1/products/biglakeiceberg/IcebergTable.yaml:181-186:

- name: field_id
  api_name: field-id
  type: Integer
  output: true
  description: |
    The unique identifier of the partition field.

So the provider reads the value back, but never includes it in the create request body. The generated schema confirms there is no way around this. Here is terraform providers schema -json, identical in 7.46.0 and 8.0.0:

partition_spec attributes: {'spec_id': ['computed']}
  fields.field_id: number [computed]
  fields.name:      string [required]
  fields.source_id: number [required]
  fields.transform: string [required]

The sortOrder block just below in the same file already handles a create/read asymmetry for this API. The create body uses write-order, while the read response returns sort-orders plus default-sort-order-id, and the two are bridged in post_read. A create-side field-id would fit the way this resource is already modelled.

Steps to reproduce

  1. terraform apply the configuration above.

Important Factoids

This looks like a recent server-side change rather than a provider regression. The provider's BigLake Iceberg code has not changed between 7.41 and 8.0.0 (there are no changelog entries in that range), yet this same configuration shape used to work.

I gathered the following from a catalog that holds 188 tables, 136 of them partitioned:

  • Every existing partitioned table carries a field id that the API assigned on its own: field_id=1000 where there is one partition field, [1000, 1001] where there are two.
  • The most recent partitioned table created through Terraform in that catalog dates from 2026-08-17. Every attempt after that date fails with the error above.
  • Terraform still creates unpartitioned tables in the same catalog without trouble. The most recent one is from 2026-08-24. With no partition_spec there is nothing for the API to validate.
  • Partitioned tables created by other Iceberg clients (Spark, pyiceberg) still succeed. The most recent one is from 2026-08-27. Those clients assign the field ids themselves, starting at 1000.

Putting that together: the API used to assign field-id when the client left it out, and it now expects the client to send one. Any partitioned google_biglake_iceberg_table is currently impossible to create through Terraform. Tables that already exist in state are unaffected.

References

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions