Column-level lineage for struct data not visible in new UI — works in legacy UI

[kzajaczkowski]

Describe the bug

After upgrading DataHub server from v1.3.1 to v1.4.0.3, column-level lineage (CLL) for complex data types in Trino is no longer visible in the new UI. The data is correctly ingested and present in the system — disabling the new UI in Settings immediately restores column-level lineage visibility without re-running ingestion. This indicates a frontend rendering issue in the new UI, not an ingestion problem.

To Reproduce

1. Ingest dbt metadata with include_column_lineage: true and target_platform: trino
2. Ingestion completes successfully: fineGrainedLineages: 8, sql_parser_column_errors: 0
3. Open any dataset in DataHub UI -> Lineage tab
4. Column-level lineage is not shown in the new UI
5. Go to Settings -> disable new UI -> column-level lineage is visible

Ingestion recipe (simplified):

  source:
    type: dbt
    config:
      manifest_path: target/manifest.json
      catalog_path: target/catalog.json
      run_results_paths:
        - target/run_results.json
      target_platform: trino
      include_column_lineage: true
      skip_sources_in_lineage: true
      entities_enabled:
        sources: No
  sink:
    type: datahub-rest
    config:
      server: ${DATAHUB_GMS_URL}
      token: ${DATAHUB_GMS_TOKEN}

Ingestion log summary:

  'upstreamLineage': 8,
  'fineGrainedLineages': 8,
  'sql_parser_table_errors': 0,
  'sql_parser_column_errors': 0,
  'has_errors': False,
  'has_warnings': False,

Expected behavior

Column-level lineage (fineGrainedLineages) should be visible in the new UI lineage view, the same way it is displayed when the new UI is disabled via Settings. The ingestion pipeline emits fineGrainedLineages successfully (8 aspects, 0 parser errors), and the data is present in the system — the legacy UI renders it correctly.

Screenshots

new UI — lineage tab, column-level lineage not visible

legacy UI (new UI disabled in Settings)

Desktop

• OS: macOS 15.7.3 (arm64)
• Browser: Chrome
• DataHub server: v1.4.0.3 (Acryl)
• DataHub CLI: v1.4.0.3 (also tested with v1.4.0.2)
• Previous working server: v1.3.1

Additional context

• Ingestion source: dbt (dbt-core with Trino adapter)
• The issue appeared after upgrading the server from v1.3.1 to v1.4.0.3. On v1.3.1 with the same CLI (v1.4.0.2) and the same recipe, column-level lineage was fully visible.
• Setting patch_lineage: false in the sink config does not change the behavior.
• The problem is purely frontend — toggling "Disable new UI" in Settings immediately restores column-level lineage without re-running ingestion. No server restart or re-ingestion needed.
• Workaround: disable new UI in DataHub Settings.

[manuschillerdev]

@kzajaczkowski cc: #16344