Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(powerbi): Report to Dashboard lineage #12451

Conversation

sgomezvillamor
Copy link
Contributor

@sgomezvillamor sgomezvillamor commented Jan 23, 2025

PowerBI Report objects (mapped as DataHub Dashboards) can contain PowerBI Dashboard objects (also DataHub Dashboards).

Important

Depends on PR #12433

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@github-actions github-actions bot added the ingestion PR or Issue related to the ingestion of metadata label Jan 23, 2025
Copy link

codecov bot commented Jan 23, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

✅ All tests successful. No failed tests found.

Files with missing lines Coverage Δ
...ion/src/datahub/ingestion/source/powerbi/config.py 98.45% <100.00%> (+<0.01%) ⬆️
...on/src/datahub/ingestion/source/powerbi/powerbi.py 94.64% <100.00%> (+0.28%) ⬆️
...on/source/powerbi/rest_api_wrapper/data_classes.py 92.99% <100.00%> (+0.03%) ⬆️
...n/source/powerbi/rest_api_wrapper/data_resolver.py 83.60% <ø> (+<0.01%) ⬆️
...ion/source/powerbi/rest_api_wrapper/powerbi_api.py 90.72% <100.00%> (+0.12%) ⬆️

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ac13f25...bf21f7d. Read the comment docs.

@sgomezvillamor sgomezvillamor changed the base branch from master to feature/cus-3571-fix-model-DashboardInfo-DashboardContainsDashboard January 27, 2025 06:39
…insDashboard' into feature/cus-3571/feat-powerbi-report-to-dashboard-lineage
@sgomezvillamor
Copy link
Contributor Author

Tested locally with the generated golden file

datahub ingest mcps /Users/sergio/workspace/github/datahub-project/datahub/metadata-ingestion/tests/integration/powerbi/golden_test_container.json
Screenshot 2025-01-27 at 12 17 56

Base automatically changed from feature/cus-3571-fix-model-DashboardInfo-DashboardContainsDashboard to master February 3, 2025 10:33
dataset=None,
**new_dataset_or_report(instance),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for cleaning this up

are there any entries in Constant that can be removed now?

createdFrom: CreatedFrom

dataset: Optional["PowerBIDataset"]
report: Optional[Report]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are report_id and report set at the same time? if not, it might be worth putting a comment that says the latter is filled in at a later time

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ids are set on a first traversal and objects in a later one
I will add a comment

downstream_dashboards_edges = []
for d in workspace.dashboards.values():
if any(t.report_id == report.id for t in d.tiles):
dashboard_urn = builder.make_dashboard_urn(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems worrying that these lines weren't covered by the tests? as per the codecov report

also, to make sure I understand - the lineage is tile -> dashboard -> report?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems worrying that these lines weren't covered by the tests? as per the codecov report

this may be a sort of false positive, because my update on default_mock_response.json was specifically tailored to force this code path

also, to make sure I understand - the lineage is tile -> dashboard -> report?

Lineage is represented with a contains relationship. Any PowerBI Dashboard having a PowerBI Tile which references a parent PowerBI Report is modelled as follows:

PowerBI Report (DataHub Dashboard) -- contains --> PowerBI Dashboard (DataHub Dashboard) // this is the new addtion
PowerBI Dashboard (DataHub Dashboard) --contains --> PowerBI Tile (DataHub Chart) // this one was already supported

This was confirmed with users to match their expectations; this is how is modelled in PowerBI Lineage.

@datahub-cyborg datahub-cyborg bot added pending-submitter-merge and removed needs-review Label for PRs that need review from a maintainer. labels Feb 5, 2025
@sgomezvillamor
Copy link
Contributor Author

merging; CI error is unrelated

@sgomezvillamor sgomezvillamor merged commit 32b654c into master Feb 5, 2025
189 of 190 checks passed
@sgomezvillamor sgomezvillamor deleted the feature/cus-3571/feat-powerbi-report-to-dashboard-lineage branch February 5, 2025 10:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ingestion PR or Issue related to the ingestion of metadata pending-submitter-merge
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants