KMS-705: Correct publisher keyword-event mapping so published and draft keyword CSVs produce complete, CSV-shaped correction objects for every supported scheme.

[cgokey]

Overview

What is the feature?

Correct publisher keyword-event mapping so published and draft keyword CSVs produce complete, CSV-shaped correction objects for every supported scheme.

This work began with the short-name scheme defect described in KMS-705. The shared CSV comparison treated every column before UUID as a hierarchy position. For schemes with auxiliary columns such as Long_Name or Data_Center_URL, the last auxiliary value could be emitted as ShortName while the actual Short_Name was discarded. This potentially affected platforms, instruments, projects, providers, IDN nodes, data formats, and granule data formats.

Since this was IP sprint, my goal was to test the complete workflow from published CSVs through generated native metadata rather than limiting validation to the initial publisher fix. This uncovered a broader contract issue: correction objects sometimes relied on positional assumptions or native metadata field names instead of the actual production CSV columns. The scope therefore expanded to formalize that contract and protect the full workflow with end-to-end regression tests.

For example, a GOSAT short-name update previously produced an unchanged event because Long_Name was incorrectly treated as ShortName:

{
  "Category": "Platforms",
  "Class": "Space-based Platforms",
  "Type": "Earth Observation Satellites",
  "ShortName": "Greenhouse Gases Observing Satellite"
}

The corrected event follows the platform CSV headers and preserves both values independently:

{
  "Basis": "Space-based Platforms",
  "Category": "Earth Observation Satellites",
  "SubCategory": "",
  "ShortName": "GOSAT",
  "LongName": "Greenhouse Gases Observing Satellite"
}

The draft object is identical except that ShortName is correctly set to GOSAT - Test1.

The same positional behavior could lose intermediate hierarchy values. An Amazonia platform update was recorded with a new path similar to Space-based Platforms > > Amazonia-1-Chris-Test, dropping Earth Observation Satellites. Reading the named Category column now preserves that value, while also retaining the intentionally blank Sub_Category slot.

What is the Solution?

• Parse exported keyword CSVs by normalized header name instead of treating every pre-UUID column as an indexed hierarchy value.
• Define the exact CSV fields and canonical order for each scheme, including production names such as Basis, Sub_Category, Short_Name, Long_Name, and Data_Center_URL.
• Preserve blank hierarchy positions so later values do not shift into the wrong semantic field.
• Keep auxiliary values such as LongName and DataCenterURL in correction objects without treating them as hierarchy levels.
• Use one CSV field contract for publisher events, historical lookups, and correction resolution.
• Separate CSV correction-object fields from native metadata fields. The UMM-C, ECHO10, DIF10, and ISO editors now explicitly translate CSV-shaped correction objects into their format-specific JSON or XML fields.
• Preserve long-name-only changes even when the canonical hierarchy path itself does not change.
• Add production-shaped published.csv and draft.csv fixtures for every supported keyword scheme.
• Remove redundant tests while retaining focused unit coverage and the complete end-to-end regression suite.

The end-to-end tests now verify two independent boundaries:

1. The production-shaped published and draft CSV fixtures are compared, publisher events are created, and the resulting old/new keyword objects are resolved into corrections. Those generated corrections must match the checked-in correction JSON for each native format.
2. The checked-in correction JSON is independently applied to complete UMM-C, DIF10, ECHO10, ISO19115, and ISO-SMAP records. The full serialized result must exactly match the checked-in after record for both updates and deletions.

The generated corrections are deliberately not passed directly into the native transformation assertion. Keeping the two stages independent prevents a broken publisher from defining its own expected native result. It also gives us a repeatable pattern for future mapping defects: add the production-shaped CSV case, the expected correction object, and the affected native metadata field, then the test locks the reported behavior from publication through transformation.

What areas of the application does this impact?

• Published-versus-draft keyword CSV comparison and event generation
• Historical and published keyword cache lookups
• Keyword path and correction-object construction
• UMM-C, DIF10, ECHO10, ISO19115, and ISO-SMAP metadata editors
• Native metadata correction fixtures and regression tests
• Local publisher mapping smoke coverage

Testing

1. Run npm test -- --run serverless/src/shared/__tests__/applyNativeMetadataCorrectionMappings.test.js.
2. Confirm each native format generates the checked-in update and deletion corrections from the published/draft CSV fixtures.
3. Confirm those checked-in corrections produce an exact match with each complete expected native metadata record.
4. Run npm test -- --run and confirm the full suite passes. The current result is 226 test files and 2,296 tests passing.
5. Run npm run lint and confirm ESLint passes.
6. In VS Code, select a before native metadata fixture for comparison, then compare it with the corresponding after fixture under serverless/src/shared/__mocks__/native_metadata_correction_mappings/end_to_end.
7. Verify _CHANGED values appear only in intended mapped fields, _SHOULD_DELETE values are absent from deletion results, and unrelated metadata is unchanged.
8. Review generated/short-name-events.json and confirm short-name schemes preserve the true short name plus auxiliary long-name or provider URL values.

Attachments

The production-shaped CSV inputs, expected publisher events, correction objects, and complete before/after native records are included under serverless/src/shared/__mocks__/native_metadata_correction_mappings/end_to_end.

Checklist

• I have added automated tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 9c1f43e1-cc73-4dcd-a7c8-c68184b31e9a

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[codecov-commenter]

Codecov Report

❌ Patch coverage is 98.87006% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 99.63%. Comparing base (28d0069) to head (42db0bc).

Files with missing lines	Patch %	Lines
serverless/src/shared/Iso19115DomEditor.js	97.36%	1 Missing ⚠️
.../src/shared/redis-path-store/helpers/keywordCsv.js	98.82%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #134      +/-   ##
==========================================
- Coverage   99.73%   99.63%   -0.10%     
==========================================
  Files         234      232       -2     
  Lines        6391     6341      -50     
  Branches     1907     1864      -43     
==========================================
- Hits         6374     6318      -56     
- Misses         16       21       +5     
- Partials        1        2       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.