Skip to content

GitHub App: track extra metadata - #12829

Open
stsewd wants to merge 3 commits into
mainfrom
gh-installation-metadata
Open

GitHub App: track extra metadata#12829
stsewd wants to merge 3 commits into
mainfrom
gh-installation-metadata

Conversation

@stsewd

@stsewd stsewd commented Mar 4, 2026

Copy link
Copy Markdown
Member

We could also make these extra properties that dynamically get the information from extra_data as well (they are present in all objects). Since we weren't updating the extra data field after the integration was created we will need to update all of them (we have less than 2K installations).

from readthedocs.oauth.clients import get_gh_app_client
client = get_gh_app_client()
for installation in GitHubAppInstallation.objects.iterator():
    gh_installation = client.get_app_installation(installation.installation_id)
    gh_installation.extra_data = {"installation": gh_installation.raw_data}
    gh_installation.save()

Closes https://github.com/readthedocs/readthedocs-corporate/issues/2046

@stsewd
stsewd requested a review from a team as a code owner March 4, 2026 23:23
@stsewd
stsewd requested a review from humitos March 4, 2026 23:23

@humitos humitos left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

    gh_installation.extra_data = {"installation": gh_installation.raw_data}

Why are we saving this data under a installation key, the object is the installation already, there is no need to have a nested field inside it. We should just do gh_installation.extra_data = gh_installation.raw_data for simplification.

Comment thread readthedocs/oauth/models.py
Comment thread readthedocs/oauth/models.py Outdated
Comment on lines +95 to +96
default="",
null=True,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
default="",
null=True,
null=True,
blank=True,

Remove the default="" so it uses None by default. Also add blank so we can define it from the admin.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All objects in production will have this field populated after the data migration.

@stsewd

stsewd commented Mar 5, 2026

Copy link
Copy Markdown
Member Author

Why are we saving this data under a installation key, the object is the installation already, there is no need to have a nested field inside it. We should just do gh_installation.extra_data = gh_installation.raw_data for simplification.

Extra data can contain more info related to event when the installation was created.

@humitos

humitos commented Mar 9, 2026

Copy link
Copy Markdown
Member

Extra data can contain more info related to event when the installation was created.

In that case, we should call gh_installation.extra_data.update(extra_data) to avoid overriding other fields in the JSON, shouldn't we?

@stsewd

stsewd commented Mar 10, 2026

Copy link
Copy Markdown
Member Author

In that case, we should call gh_installation.extra_data.update(extra_data) to avoid overriding other fields in the JSON, shouldn't we?

During the migration yeah, after that it's fine to override.

@humitos

humitos commented Mar 10, 2026

Copy link
Copy Markdown
Member

In that case, after the migration we won't have other fields inside the JSON... that goes back to my original question? Why are we saving it as = {"installation": data} instead of just = data?

@stsewd

stsewd commented Mar 10, 2026

Copy link
Copy Markdown
Member Author

In that case, after the migration we won't have other fields inside the JSON.

After the migration we will still have extra fields in the data field, that's from when the installation was updated

data = self.data.copy()

At the end, this is just extra metadata to debug, or make it easy to inspect some fields without re-fetching the installation.

@github-project-automation github-project-automation Bot moved this to Planned in 📍Roadmap Apr 8, 2026
@stsewd stsewd moved this from Planned to Needs review in 📍Roadmap Apr 8, 2026
@agjohnson

Copy link
Copy Markdown
Contributor

After the migration we will still have extra fields in the data field, that's from when the installation was updated

It is confusing that this data structure switches purposes. It's unclear to me what this data structure will look like now compared to after the migration.

If we can't replace the extra data completely as @humitos is suggesting, this would be clearer as either a separate field for the installation data or both sets of data using special keys under extra_data. I think mixing the two is what makes this more confusing.

I don't have strong opinions here either way though.

@ericholscher

Copy link
Copy Markdown
Member

We should just pick a path here and move forward. This seems useful, and it's been blocked on a minor design decision for a while.

@stsewd

stsewd commented Apr 13, 2026

Copy link
Copy Markdown
Member Author

It is confusing that this data structure switches purposes. It's unclear to me what this data structure will look like now compared to after the migration.

It would be the same.

this would be clearer as either a separate field for the installation data or both sets of data using special keys under extra_data. I think mixing the two is what makes this more confusing.

Extra data doesn't have a strict structure, as the name says it's just extra metadata from when the installation was created. We can fetch anything we want from the installation already using the API.

@ericholscher
ericholscher requested a balanced review from Copilot August 19, 2026 23:25
@ericholscher

Copy link
Copy Markdown
Member

Can we just remove the denormalized fields, etc. and just store extra_data as an update? It feels like we debated on this PR longer than it would take to just do a simple fix.

Going to let the AI decide if it's a real issue :D

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Tracks GitHub App account login and repository-selection metadata on installations.

Changes:

  • Adds persisted metadata fields and derives them from extra_data.
  • Refreshes existing installation metadata.
  • Adds webhook coverage for metadata persistence.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
readthedocs/oauth/models.py Adds fields and synchronization logic.
readthedocs/oauth/migrations/0019_githubapp_add_extra_fields.py Adds database columns.
readthedocs/oauth/tests/test_githubapp_webhook.py Tests metadata extraction.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +15 to +19
migrations.AddField(
model_name="githubappinstallation",
name="all_repositories_selected",
field=models.BooleanField(
db_default=False,
@ericholscher

Copy link
Copy Markdown
Member

hahaha - it didn't catch anything :D

@ericholscher ericholscher left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like we spent way too much time on this review -- the current code is better than what we had before. We should just merge it? Though we should also probably expose it in the admin? Otherwise not really sure how useful it will be.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Needs review

Development

Successfully merging this pull request may close these issues.

5 participants