Skip to content

feat(integrations): Expand platform detection to 98% picker coverage#109701

Merged
jaydgoss merged 20 commits intomasterfrom
jaygoss/vdy-15-poc-platform-detection-from-github-repository-languages
Mar 12, 2026
Merged

feat(integrations): Expand platform detection to 98% picker coverage#109701
jaydgoss merged 20 commits intomasterfrom
jaygoss/vdy-15-poc-platform-detection-from-github-repository-languages

Conversation

@jaydgoss
Copy link
Member

@jaydgoss jaydgoss commented Mar 2, 2026

Expand the composable framework detection system to cover 97/100 (97%) of selectable platforms in the picker, up from the ~36 platforms in PR 2.

Infrastructure additions:

  • match_ext + match_content combo rules -- find files by extension, then search content (needed for .csproj inspection where filenames vary)
  • _NON_SELECTABLE_PLATFORMS filter -- platforms detected internally for ranking (e.g. preventing WordPress from being misidentified as Symfony) but not shown to users because they lack onboarding docs or map to other platforms (perl, php-wordpress, swift)
  • Dual base_platform entries for Android (java + kotlin) so Kotlin-first projects are correctly detected

61 new framework definitions:

  • JavaScript (22): astro, gatsby, sveltekit, solidstart, solid, ember, tanstackstart-react, react-router, react-native, electron, capacitor, ionic, cordova, node, nestjs, fastify, connect, hapi, awslambda, gcpfunctions, azurefunctions, cloudflare-workers, cloudflare-pages
  • Python (13): aiohttp, bottle, falcon, pyramid, quart, sanic, tryton, chalice, asgi, wsgi, awslambda, gcpfunctions, rq
  • Go (4): fasthttp, iris, negroni
  • Java (2): log4j2, logback
  • Ruby (1): rack
  • PHP (2): wordpress (non-selectable), symfony
  • Dart (1): flutter
  • Swift (1): apple-macos
  • Native (1): native-qt
  • .NET (7): maui, wpf, winforms, xamarin, aspnet, awslambda, gcpfunctions
  • Mobile/Gaming (5): unity, android, dotnet-aspnetcore, unreal, godot
  • Other (3): bun, deno, PowerShell (base platform)

Only 3 platforms remain undetectable: go-http (stdlib net/http, no manifest signal), minidump (crash dump format, not a project type), and python-serverless (too generic, overlaps with awslambda/gcpfunctions).

Note on go-http: Plain Go repos now intentionally return go as the base platform instead of go-http. There is no reliable way to distinguish a net/http project from any other Go project without a framework dependency, so the generic go platform is the correct fallback.

Stack:

  • PR 1: Core detection + API endpoint
  • PR 2: Composable framework definitions refactor
  • PR 3 (this): Expanded coverage (97% of picker platforms)

@jaydgoss jaydgoss requested a review from a team as a code owner March 2, 2026 19:40
@linear
Copy link

linear bot commented Mar 2, 2026

@github-actions github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Mar 2, 2026
@jaydgoss jaydgoss force-pushed the jaygoss/vdy-15-poc-platform-detection-from-github-repository-languages branch from a6bfbc8 to 82d4875 Compare March 2, 2026 19:44
@jaydgoss jaydgoss force-pushed the jaygoss/vdy-15-platform-detection-composable branch from 61cacd1 to 0815067 Compare March 2, 2026 19:44
@jaydgoss jaydgoss marked this pull request as draft March 2, 2026 20:14
@jaydgoss jaydgoss force-pushed the jaygoss/vdy-15-platform-detection-composable branch from 0815067 to efa46fe Compare March 2, 2026 23:11
@jaydgoss jaydgoss force-pushed the jaygoss/vdy-15-poc-platform-detection-from-github-repository-languages branch from 82d4875 to 1669722 Compare March 2, 2026 23:11
@jaydgoss jaydgoss force-pushed the jaygoss/vdy-15-poc-platform-detection-from-github-repository-languages branch from 1669722 to 67771da Compare March 2, 2026 23:15
@jaydgoss jaydgoss force-pushed the jaygoss/vdy-15-poc-platform-detection-from-github-repository-languages branch from 67771da to 0ccfbe1 Compare March 2, 2026 23:19
@jaydgoss jaydgoss force-pushed the jaygoss/vdy-15-poc-platform-detection-from-github-repository-languages branch from 0ccfbe1 to da01416 Compare March 3, 2026 16:07
@jaydgoss jaydgoss force-pushed the jaygoss/vdy-15-poc-platform-detection-from-github-repository-languages branch from 9c67c75 to 5f350ee Compare March 3, 2026 16:45
@jaydgoss jaydgoss force-pushed the jaygoss/vdy-15-poc-platform-detection-from-github-repository-languages branch from 5f350ee to 88af1ae Compare March 3, 2026 17:02
@jaydgoss jaydgoss force-pushed the jaygoss/vdy-15-poc-platform-detection-from-github-repository-languages branch from 88af1ae to 99bb0f7 Compare March 3, 2026 17:27
@jaydgoss jaydgoss force-pushed the jaygoss/vdy-15-platform-detection-composable branch from 782090f to 40dfe37 Compare March 3, 2026 17:48
@github-actions
Copy link
Contributor

github-actions bot commented Mar 9, 2026

Backend Test Failures

Failures on d22eaf8 in this run:

tests/sentry/integrations/github/test_platform_detection.py::TestDetectPlatforms::test_godot_detected_from_project_filelog
tests/sentry/integrations/github/test_platform_detection.py:1435: in test_godot_detected_from_project_file
    assert "godot" in platforms
E   AssertionError: assert 'godot' in ['native']

@github-actions
Copy link
Contributor

github-actions bot commented Mar 9, 2026

Backend Test Failures

Failures on 3087ce4 in this run:

tests/sentry/integrations/github/test_platform_detection.py::TestDetectPlatforms::test_godot_detected_from_project_filelog
tests/sentry/integrations/github/test_platform_detection.py:1435: in test_godot_detected_from_project_file
    assert "godot" in platforms
E   AssertionError: assert 'godot' in ['native']

Copy link
Contributor

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 9, 2026

Backend Test Failures

Failures on 415a006 in this run:

tests/sentry/integrations/github/test_platform_detection.py::TestDetectPlatforms::test_godot_detected_from_project_filelog
tests/sentry/integrations/github/test_platform_detection.py:1435: in test_godot_detected_from_project_file
    assert "godot" in platforms
E   AssertionError: assert 'godot' in ['native']

Copy link
Contributor

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 9, 2026

Backend Test Failures

Failures on aadef81 in this run:

tests/sentry/integrations/github/test_platform_detection.py::TestDetectPlatforms::test_wordpress_does_not_supersede_symfonylog
tests/sentry/integrations/github/test_platform_detection.py:1482: in test_wordpress_does_not_supersede_symfony
    assert "php-symfony" in platforms
E   AssertionError: assert 'php-symfony' in ['php']

@github-actions
Copy link
Contributor

github-actions bot commented Mar 10, 2026

Backend Test Failures

Failures on cbbaffb in this run:

tests/sentry/profiles/test_task.py::DeobfuscationViaSymbolicator::test_basic_resolvinglog
tests/sentry/profiles/test_task.py:627: in test_basic_resolving
    assert android_profile["profile"]["methods"] == [
E   AssertionError: assert [{'class_name...oolean', ...}] == [{'class_name...oolean', ...}]
E     
E     At index 0 diff: {'class_name': 'org.slf4j.helpers.Util$ClassContextSecurityManager', 'name': 'getClassContext', 'signature': '()', 'source_file': 'Util.java', 'source_line': 67, 'data': {'deobfuscation_status': 'deobfuscated'}} != {'data': {'deobfuscation_status': 'deobfuscated'}, 'name': 'getClassContext', 'class_name': 'org.slf4j.helpers.Util$ClassContextSecurityManager', 'signature': '()', 'source_file': 'Something.java', 'source_line': 67}
E     
E     Full diff:
E       [
E           {
E               'class_name': 'org.slf4j.helpers.Util$ClassContextSecurityManager',
E               'data': {
E                   'deobfuscation_status': 'deobfuscated',
E               },
E               'name': 'getClassContext',
E               'signature': '()',
E     -         'source_file': 'Something.java',
E     ?                         ^^^^ - ^^
E     +         'source_file': 'Util.java',
E     ?                         ^  ^
E               'source_line': 67,
E           },
E           {
E               'class_name': 'org.slf4j.helpers.Util$ClassContextSecurityManager',
E               'data': {
E                   'deobfuscation_status': 'deobfuscated',
E               },
E               'name': 'getExtraClassContext',
E               'signature': '(): boolean',
E     -         'source_file': 'Else.java',
E     ?                         ^ --
E     +         'source_file': 'Util.java',
E     ?                         ^^^
E               'source_line': 69,
E           },
E       ]
tests/sentry/integrations/api/endpoints/test_organization_repository_platforms.py::OrganizationRepositoryPlatformsGetTest::test_detects_platformslog
tests/sentry/integrations/api/endpoints/test_organization_repository_platforms.py:69: in test_detects_platforms
    assert response.data == {
E   AssertionError: assert {'platforms':...ython', ...}]} == {'platforms':...cript', ...}]}
E     
E     Differing items:
E     {'platforms': [{'bytes': 50000, 'confidence': 'medium', 'language': 'Python', 'platform': 'python', ...}]} != {'platforms': [{'bytes': 50000, 'confidence': 'medium', 'language': 'Python', 'platform': 'python', ...}, {'bytes': 30000, 'confidence': 'medium', 'language': 'JavaScript', 'platform': 'javascript', ...}]}
E     
E     Full diff:
E       {
E           'platforms': [
E               {
E                   'bytes': 50000,
E                   'confidence': 'medium',
E                   'language': 'Python',
E                   'platform': 'python',
E                   'priority': 1,
E               },
E     -         {
E     -             'bytes': 30000,
E     -             'confidence': 'medium',
E     -             'language': 'JavaScript',
E     -             'platform': 'javascript',
E     -             'priority': 1,
E     -         },
E           ],
E       }
tests/sentry/profiles/test_task.py::DeobfuscationViaSymbolicator::test_inline_resolvinglog
tests/sentry/profiles/test_task.py:683: in test_inline_resolving
    assert android_profile["profile"]["methods"] == [
E   AssertionError: assert [{'class_name...andler', ...}] == [{'class_name...andler', ...}]
E     
E     At index 0 diff: {'class_name': 'io.sentry.sample.-$$Lambda$r3Avcbztes2hicEObh02jjhQqd4', 'name': 'onClick', 'signature': '()', 'source_file': '-.java', 'source_line': 2, 'data': {'deobfuscation_status': 'deobfuscated'}} != {'class_name': 'io.sentry.sample.-$$Lambda$r3Avcbztes2hicEObh02jjhQqd4', 'data': {'deobfuscation_status': 'deobfuscated'}, 'name': 'onClick', 'signature': '()', 'source_file': None, 'source_line': 2}
E     
E     Full diff:
E       [
E           {
E               'class_name': 'io.sentry.sample.-$$Lambda$r3Avcbztes2hicEObh02jjhQqd4',
E               'data': {
E                   'deobfuscation_status': 'deobfuscated',
E               },
E               'name': 'onClick',
E               'signature': '()',
E     -         'source_file': None,
E     ?                        ^^^^
E     +         'source_file': '-.java',
E     ?                        ^^^^^^^^
E               'source_line': 2,
E           },
E           {
E               'class_name': 'io.sentry.sample.MainActivity',
E               'data': {
E                   'deobfuscation_status': 'deobfuscated',
E               },
E               'inline_frames': [
E                   {
E                       'class_name': 'io.sentry.sample.MainActivity',
E                       'data': {
E                           'deobfuscation_status': 'deobfuscated',
E                       },
E                       'name': 'onClickHandler',
E                       'signature': '()',
E                       'source_file': 'MainActivity.java',
E                       'source_line': 40,
E                   },
E                   {
E                       'class_name': 'io.sentry.sample.MainActivity',
E                       'data': {
E                           'deobfuscation_status': 'deobfuscated',
E                       },
E                       'name': 'foo',
E                       'signature': '()',
E                       'source_file': 'MainActivity.java',
E                       'source_line': 44,
E                   },
E                   {
E                       'class_name': 'io.sentry.sample.MainActivity',
E                       'data': {
... (14 more lines)

@github-actions
Copy link
Contributor

Backend Test Failures

Failures on 3207305 in this run:

tests/sentry/integrations/api/endpoints/test_organization_repository_platforms.py::OrganizationRepositoryPlatformsGetTest::test_detects_platformslog
tests/sentry/integrations/api/endpoints/test_organization_repository_platforms.py:69: in test_detects_platforms
    assert response.data == {
E   AssertionError: assert {'platforms':...ython', ...}]} == {'platforms':...cript', ...}]}
E     
E     Differing items:
E     {'platforms': [{'bytes': 50000, 'confidence': 'medium', 'language': 'Python', 'platform': 'python', ...}]} != {'platforms': [{'bytes': 50000, 'confidence': 'medium', 'language': 'Python', 'platform': 'python', ...}, {'bytes': 30000, 'confidence': 'medium', 'language': 'JavaScript', 'platform': 'javascript', ...}]}
E     
E     Full diff:
E       {
E           'platforms': [
E               {
E                   'bytes': 50000,
E                   'confidence': 'medium',
E                   'language': 'Python',
E                   'platform': 'python',
E                   'priority': 1,
E               },
E     -         {
E     -             'bytes': 30000,
E     -             'confidence': 'medium',
E     -             'language': 'JavaScript',
E     -             'platform': 'javascript',
E     -             'priority': 1,
E     -         },
E           ],
E       }

Copy link
Contributor

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

"php": "composer.json",
"dart": "pubspec.yaml",
"ruby": "Gemfile",
"go": "go.mod",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Go manifest parsing infrastructure is unused dead code

Low Severity

_PACKAGE_MANIFEST_FILES["go"] = "go.mod" causes go.mod to be parsed via _parse_go_mod during detection, and Go module version matching was added to _package_in_manifest. However, every Go framework definition uses match_content regex rules on go.mod — none uses match_package. The parsed Go manifest and version-path matching logic are never exercised in production, making this dead infrastructure.

Additional Locations (2)
Fix in Cursor Fix in Web

jaydgoss and others added 20 commits March 12, 2026 13:04
Expand the composable framework detection system to cover 97/100 (97%)
of selectable platforms in the picker.

Infrastructure additions:
- match_ext + match_content combo rules for .csproj inspection
- _NON_SELECTABLE_PLATFORMS filter for internally-used platforms
- Dual base_platform entries for Android (java + kotlin)
- Cross-base-platform dedup keeps higher byte count

61 new framework definitions across JavaScript, Python, Go, Java, Ruby,
PHP, Dart, Swift, Native, .NET, Mobile/Gaming, and other runtimes.
Replace the custom line-based pubspec.yaml parser with
yaml.safe_load() via sentry.utils.yaml, which is already used
elsewhere in the codebase. More robust against comments, flow
mappings, and non-standard indentation.
Map GDScript directly to godot in the language map. GDScript is
exclusively used for Godot, so any repo with GDScript as its primary
language is a Godot project regardless of whether project.godot is
present at root.
Perl was removed from GITHUB_LANGUAGE_TO_SENTRY_PLATFORM, so it can
never appear as a detected platform. The _NON_SELECTABLE_PLATFORMS
entry was dead code.
When the same platform is detected via different base_platform entries,
also update confidence and priority so the values stay consistent if
entries ever diverge in sort order.
The engines.node field is too ubiquitous in the JS ecosystem to be a
reliable Node.js signal — even browser-only libraries set it for CI
compatibility. This caused pure JS tools like prettier to be detected
as Node instead of JavaScript.

Node detection now relies on stronger signals: .nvmrc, .node-version,
nodemon.json, and Procfile with "node".
…match

When the same platform ID is detected via different base_platforms
(e.g. apple-ios from both Objective-C and Swift), always upgrade
confidence and priority when a framework rule matches, even if the
new base_platform has fewer bytes. Previously, confidence stayed at
"medium" if the language-only entry had more bytes.
Only one platform suggestion is shown to the user, so processing all
languages wastes API calls. Now detect_platforms finds the top language
by bytes and only evaluates frameworks for its base platform.
…tection

The else branch that merged duplicate platform IDs across different
base platforms became unreachable after limiting detection to a single
top-language platform. With only one base_platform in active_platforms,
and no duplicate platform IDs within a single base_platform, the
dedup path can never execute.

Co-Authored-By: Claude <noreply@anthropic.com>
Both Laravel and WordPress bundle Symfony components, so detecting
symfony/ in composer.json would incorrectly suggest Symfony onboarding.
For WordPress this is critical since it gets removed by
_NON_SELECTABLE_PLATFORMS, leaving only the wrong Symfony result.

Co-Authored-By: Claude <noreply@anthropic.com>
…apping

GDScript maps to base platform "godot" but the framework definition
used "native", so the project.godot file check was unreachable for
GDScript-dominant repos, always falling back to medium confidence.

Co-Authored-By: Claude <noreply@anthropic.com>
…t space

Replace broad `except Exception` in manifest parsing with specific
exceptions (JSONDecodeError, YAMLError, ValueError, KeyError,
TypeError, AttributeError) per repo guidelines. Also handle
`require(` without space in go.mod for hand-edited files.

Co-Authored-By: Claude <noreply@anthropic.com>
A C++ repo without GDScript is not a Godot project. Now that Godot
has base_platform "godot" aligned with GDScript, the test should use
GDScript as the top language to correctly represent a Godot repo.

Co-Authored-By: Claude <noreply@anthropic.com>
Non-selectable platforms like php-wordpress were superseding selectable
ones like php-symfony, then getting filtered out themselves, dropping
valid detections. Only selectable platforms should participate in
supersession.

Co-Authored-By: Claude <noreply@anthropic.com>
The test mock's root listing was missing composer.json, so Symfony
was never detected and the supersession scenario was not exercised.

Co-Authored-By: Claude <noreply@anthropic.com>
- Remove local.settings.json from Azure Functions detection rule since
  it is commonly gitignored, causing false negatives
- Remove dead match_ext rule for .xcodeproj (it's a directory, not a
  file, so only match_dir applies)
- Restore platform dedup guard in framework matching loop to prevent
  duplicates when multi-language support is re-enabled
- Clean up module-scope _fw variable leak after index-building loops
- Handle single-line go.mod require blocks like require (pkg v1.0)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
No Go framework definitions use match_package rules -- they all use
match_content with regex patterns on go.mod directly. Remove the
orphaned _parse_go_mod function, go.mod manifest entry, and Go module
version path matching in _package_in_manifest.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components Scope: Frontend Automatically applied to PRs that change frontend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants