Skip to content

Add comprehensive LLM tool detection support (18 tools)#52878

Open
Copilot wants to merge 4 commits intomainfrom
copilot/add-llm-tool-detections
Open

Add comprehensive LLM tool detection support (18 tools)#52878
Copilot wants to merge 4 commits intomainfrom
copilot/add-llm-tool-detections

Conversation

Copy link
Contributor

Copilot AI commented Feb 6, 2026

Plan: Add More LLM Tool Detections

Based on the am-i-ai library (https://github.com/trieloff/am-i-ai), we need to add the following LLM tool detections:

  • Review current implementation (5 tools: Claude, Cursor, Gemini, Copilot, generic_agent)
  • Add new EnvironmentVariableValueRule for value-based detection
  • Add Codex CLI detection (CODEX_CLI, CODEX_SANDBOX)
  • Add Aider detection (OR_APP_NAME matching "Aider")
  • Add Amp detection (AMP_HOME)
  • Add Qwen Code detection (QWEN_CODE)
  • Add Droid detection (DROID_CLI)
  • Add OpenCode detection (OPENCODE_AI)
  • Add Zed AI detection (ZED_ENVIRONMENT, ZED_TERM)
  • Add Kimi CLI detection (KIMI_CLI)
  • Add OpenHands detection (OR_APP_NAME matching "OpenHands")
  • Add Goose detection (GOOSE_TERMINAL)
  • Add Cline detection (CLINE_TASK_ID)
  • Add Roo Code detection (ROO_CODE_TASK_ID)
  • Add Windsurf detection (WINDSURF_SESSION)
  • Add appropriate test cases for all new detections
  • Add test cases for combinations of newer tools
  • Run tests to validate changes (38/38 tests passing)
  • Run formatting (no changes needed)
  • Code review (addressed feedback)
  • Security scan with CodeQL (no issues found)
  • Update telemetry documentation

Summary

Successfully added detection support for 13 additional LLM tools, bringing the total from 5 to 18 supported tools. The implementation includes:

  • New EnvironmentVariableValueRule for value-based matching
  • Enhanced existing Claude and Cursor detections with additional environment variables
  • Comprehensive test coverage (38 tests, all passing)
  • Updated telemetry documentation
  • No security vulnerabilities detected
Original prompt

This section details on the original issue you should resolve

<issue_title>Add more LLM tool detections</issue_title>
<issue_description>### Is your feature request related to a problem? Please describe.
There are libraries/ecosystems that have a more comprehensive set of detections for LLM tools, like https://github.com/trieloff/am-i-ai?tab=readme-ov-file#supported-ai-tools.

We should have parity in our detections, like adding Codex support in #52861</issue_description>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 2 commits February 6, 2026 18:55
Co-authored-by: baronfel <573979+baronfel@users.noreply.github.com>
Co-authored-by: baronfel <573979+baronfel@users.noreply.github.com>
Copilot AI changed the title [WIP] Add support for additional LLM tool detections Add comprehensive LLM tool detection support (18 tools) Feb 6, 2026
Copilot AI requested a review from baronfel February 6, 2026 19:00
@baronfel baronfel marked this pull request as ready for review February 6, 2026 19:19
@baronfel baronfel requested review from a team and Copilot February 6, 2026 19:19
@dotnet-policy-service
Copy link
Contributor

This PR has been labeled with Area-Telemetry. Please ensure that any telemetry changes in this PR

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Expands dotnet CLI telemetry’s LLM environment detection to cover a broader set of AI/agent tools (18 total), aiming for parity with the referenced “am-i-ai” detection list.

Changes:

  • Added a new detection rule type to match specific environment variable values case-insensitively (EnvironmentVariableValueRule).
  • Expanded LLMEnvironmentDetectorForTelemetry from a small set of tools to 18 tools, supporting multiple simultaneous detections (comma-separated).
  • Extended unit test coverage to include the new tools and some multi-tool combinations.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File Description
test/dotnet.Tests/TelemetryCommonPropertiesTests.cs Adds test cases for the newly detected tools and multi-tool combinations.
src/Cli/dotnet/Telemetry/LLMEnvironmentDetectorForTelemetry.cs Extends LLM tool detection rules from 5 to 18 tools and returns comma-separated results.
src/Cli/dotnet/Telemetry/EnvironmentDetectionRule.cs Introduces EnvironmentVariableValueRule for case-insensitive value-based env var matching.

@baronfel
Copy link
Member

baronfel commented Feb 6, 2026

Co-authored-by: baronfel <573979+baronfel@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Feb 6, 2026

@copilot update the in-repo telemetry docs at https://github.com/dotnet/sdk/pull/documentation/project-docs/telemetry.md

Updated the telemetry documentation with the complete list of 18 supported LLM tools. (4adfd2a)

Copilot AI requested a review from baronfel February 7, 2026 00:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add more LLM tool detections

3 participants