doc: manual test plan [IDE-1367] #680
Conversation
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
Summary of ChangesHello @acke, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request adds a detailed manual test plan for the new feature that enables users to select project organizations within the VSCode extension. The document serves as a guide for thoroughly testing the functionality, covering its integration with the Language Server, user interface, configuration persistence, and various operational scenarios to ensure robustness and correctness. Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds a comprehensive manual test plan for the new organization selection feature. The document is well-structured and detailed, covering various scenarios from basic functionality to edge cases and regression testing. I've found a minor numbering inconsistency in one of the test cases and provided a suggestion to fix it. Overall, this is an excellent test plan that will be very helpful for quality assurance.
| #### 7.1 Test Adding/Changing Organization at Folder Level | ||
| 1. Ensure workspace folder has migrated folder config | ||
| 2. Open Settings | ||
| 3. Set **"Snyk: Advanced: Organization"** at folder level (in `.vscode/settings.json`) | ||
| 4. Settings are saved automatically | ||
| 5. Verify: | ||
| - Folder-level organization is saved in `.vscode/settings.json` | ||
| - **Folder configs are updated** (folder-specific settings take precedence) | ||
| - Language Server receives updated folder config with `preferredOrg` | ||
| - If folder has no preferred org, fallback to folder-level setting works | ||
|
|
||
| #### 7.2 Test Deleting/Blanking Organization at Folder Level | ||
| 1. Ensure workspace folder has migrated folder config | ||
| 2. Open Settings | ||
| 3. Clear **"Snyk: Advanced: Organization"** at folder level (set to empty) | ||
| 4. Settings are saved automatically | ||
| 5. Verify: | ||
| - Folder-level organization is cleared in `.vscode/settings.json` | ||
| - **Folder configs are updated** (`preferredOrg` may be cleared) | ||
| - Language Server receives updated folder config | ||
| - If folder has no preferred org, fallback to workspace/user level works | ||
|
|
||
| #### 7.3 Test Organization Change at Workspace Level | ||
| 1. Set organization at workspace level (in workspace root's `.vscode/settings.json`) | ||
| 2. Verify folder config uses folder-level org if set (not workspace-level) | ||
| 3. Clear folder-level organization | ||
| 4. Verify fallback to workspace-level organization works | ||
| 5. Verify Language Server receives correct organization | ||
|
|
||
| #### 7.4 Test Organization Change at User Level | ||
| 1. Set organization at user level (in user `settings.json`) | ||
| 2. Clear folder and workspace-level organizations | ||
| 3. Verify fallback to user-level organization works | ||
| 4. Verify Language Server receives correct organization | ||
|
|
||
| #### 7.5 Test Organization Change with Auto-Select Enabled | ||
| 1. Enable auto-select at folder level | ||
| 2. Change organization at workspace or user level | ||
| 3. Settings are saved automatically | ||
| 4. Verify: | ||
| - Folder uses `autoDeterminedOrg` (not workspace/user org) | ||
| - Workspace/user org change does not affect folder config | ||
| - Scans use `autoDeterminedOrg` | ||
|
|
||
| #### 7.6 Test Configuration Hierarchy Precedence | ||
| 1. Set different organizations at different levels: | ||
| - Folder level: "org-folder" | ||
| - Workspace level: "org-workspace" | ||
| - User level: "org-user" | ||
| 2. Verify folder-level takes precedence | ||
| 3. Clear folder-level, verify workspace-level is used | ||
| 4. Clear workspace-level, verify user-level is used |
There was a problem hiding this comment.
There's a numbering inconsistency in the sub-sections of 'Test Case 8'. They are numbered as 7.x instead of 8.x. This seems to be a copy-paste error from 'Test Case 7'. Please correct the numbering for clarity.
#### 8.1 Test Adding/Changing Organization at Folder Level
1. Ensure workspace folder has migrated folder config
2. Open Settings
3. Set **"Snyk: Advanced: Organization"** at folder level (in `.vscode/settings.json`)
4. Settings are saved automatically
5. Verify:
- Folder-level organization is saved in `.vscode/settings.json`
- **Folder configs are updated** (folder-specific settings take precedence)
- Language Server receives updated folder config with `preferredOrg`
- If folder has no preferred org, fallback to folder-level setting works
#### 8.2 Test Deleting/Blanking Organization at Folder Level
1. Ensure workspace folder has migrated folder config
2. Open Settings
3. Clear **"Snyk: Advanced: Organization"** at folder level (set to empty)
4. Settings are saved automatically
5. Verify:
- Folder-level organization is cleared in `.vscode/settings.json`
- **Folder configs are updated** (`preferredOrg` may be cleared)
- Language Server receives updated folder config
- If folder has no preferred org, fallback to workspace/user level works
#### 8.3 Test Organization Change at Workspace Level
1. Set organization at workspace level (in workspace root's `.vscode/settings.json`)
2. Verify folder config uses folder-level org if set (not workspace-level)
3. Clear folder-level organization
4. Verify fallback to workspace-level organization works
5. Verify Language Server receives correct organization
#### 8.4 Test Organization Change at User Level
1. Set organization at user level (in user `settings.json`)
2. Clear folder and workspace-level organizations
3. Verify fallback to user-level organization works
4. Verify Language Server receives correct organization
#### 8.5 Test Organization Change with Auto-Select Enabled
1. Enable auto-select at folder level
2. Change organization at workspace or user level
3. Settings are saved automatically
4. Verify:
- Folder uses `autoDeterminedOrg` (not workspace/user org)
- Workspace/user org change does not affect folder config
- Scans use `autoDeterminedOrg`
#### 8.6 Test Configuration Hierarchy Precedence
1. Set different organizations at different levels:
- Folder level: "org-folder"
- Workspace level: "org-workspace"
- User level: "org-user"
2. Verify folder-level takes precedence
3. Clear folder-level, verify workspace-level is used
4. Clear workspace-level, verify user-level is used
acke
changed the title
Description
Provide description of this PR and changes, if linked Jira ticket doesn't cover it in full.
Checklist
Screenshots / GIFs
Visuals that may help the reviewer. Please add screenshots for any UI change. GIFs are most welcome!