Bug details:
It appears that StarshipDagHistoryMigrationOperator may have a limitation where it does not migrate DAG run run_config fields. The source (AWS MWAA) DAG runs have non-null run_config values, but after the migration to the target environment (Astro), the DAG runs have a run_config value of None.
Source: MWAA, local instance using Astro CLI or any Airflow instance
Destination: Astro
Airflow Version: Replicated the issue on 2.9.2 but applicable for all the Airflow versions.
Starship Version: Latest version 2.1.0
Desktop (please complete the following information):
- OS: iOS
- Browser: ARC
-
-
- Chromium Engine Version 131.0.6778.86
Screenshots:
- Source dag_run run config:
- Target dag_run run config:
Logs:
No errors in the logs
Additional context
-
Expected behavior:
Given a source DAG run with a non-null run_config, after running the migration operator logic, the target DAG run should include the same run_config.
-
Observed behavior:
Given a source DAG run with a non-null run_config, after the migration operator logic, the target DAG run’s run_config is None.
The migration logic appears to omit run_config during the creation of new DAG runs. Please advise if run_config fields are expected to be migrated or if there's an additional configuration required.
Bug details:
It appears that StarshipDagHistoryMigrationOperator may have a limitation where it does not migrate DAG run run_config fields. The source (AWS MWAA) DAG runs have non-null run_config values, but after the migration to the target environment (Astro), the DAG runs have a run_config value of None.
Source: MWAA, local instance using Astro CLI or any Airflow instance
Destination: Astro
Airflow Version: Replicated the issue on 2.9.2 but applicable for all the Airflow versions.
Starship Version: Latest version 2.1.0
Desktop (please complete the following information):
Screenshots:
Logs:
No errors in the logs
Additional context
Expected behavior:
Given a source DAG run with a non-null run_config, after running the migration operator logic, the target DAG run should include the same run_config.
Observed behavior:
Given a source DAG run with a non-null run_config, after the migration operator logic, the target DAG run’s run_config is None.
The migration logic appears to omit run_config during the creation of new DAG runs. Please advise if run_config fields are expected to be migrated or if there's an additional configuration required.