Merge pull request #49273 from github/update-autofix-docs-python-support

Update code scanning autofix documentation to include Python support

Author: saritai

content/code-security/code-scanning/managing-code-scanning-alerts/about-autofix-for-codeql-code-scanning.md

@@ -4,7 +4,7 @@ shortTitle: Autofix for code scanning
 intro: Learn how GitHub uses AI to suggest potential fixes for {% data variables.product.prodname_code_scanning %} alerts found by {% data variables.product.prodname_codeql %} in your pull request.
 product: '{% data reusables.rai.code-scanning.gated-feature-autofix %}'
 versions:
-  feature: code-scanning-autofix-js-ts
+  feature: code-scanning-autofix
   fpt: '*'
 type: rai
 topics:
@@ -15,7 +15,7 @@ topics:
 ---
 <!--Note on the versioning above ^. This article is visible to free, pro, team users for transparency. They cannot use the feature so `fpt` is not included in the feature definition.-->
 
-{% data reusables.rai.code-scanning.beta-autofix-js-ts %}
+{% data reusables.rai.code-scanning.beta-autofix %}
 
 ## About autofix for {% data variables.product.prodname_codeql %} {% data variables.product.prodname_code_scanning %}
 
@@ -31,6 +31,10 @@ Autofix is an AI-powered expansion of {% data variables.product.prodname_code_sc
 
 After committing a suggested fix or modified fix, the developer should always verify that continuous integration testing (CI) for the codebase continues to pass and that the alert is shown as resolved before they merge their pull request.
 
+## Supported languages
+
+{% data variables.product.prodname_code_scanning_caps %} autofix supports fix generation for a subset of queries included in the default query suite for JavaScript, TypeScript, and Python. For more information on the default query suite, see "[AUTOTITLE](/code-security/code-scanning/managing-your-code-scanning-configuration/codeql-query-suites)."
+
 ## Autofix generation process
 
 When autofix is enabled for a repository, {% data variables.product.prodname_code_scanning %} alerts that are identified in a pull request by supported {% data variables.product.prodname_codeql %} queries send input to the LLM. If the LLM can generate a potential fix, the fix is shown in the pull request as a suggestion comment.
@@ -45,11 +49,13 @@ When autofix is enabled for a repository, {% data variables.product.prodname_cod
 
 Any autofix suggestions are generated and stored within the {% data variables.product.prodname_code_scanning %} backend. They are displayed as suggestion comments in the pull request. No user interaction is needed beyond enabling {% data variables.product.prodname_code_scanning %} on the codebase and creating the pull request.
 
+The process of generating fixes does not gather or utilize any customer data beyond the scope outlined above. Therefore, the use of this feature is governed by the existing terms and conditions associated with {% data variables.product.prodname_GH_advanced_security %}. Moreover, data handled by {% data variables.product.prodname_code_scanning %} autofix is strictly not employed for LLM training purposes. For more information on  {% data variables.product.prodname_GH_advanced_security %} terms and conditions, see "[AUTOTITLE](/free-pro-team@latest/site-policy/github-terms/github-terms-for-additional-products-and-features#advanced-security){% ifversion fpt %}."{% else %} in the Free, Pro, & Team documentation.{% endif %}
+
 ## Quality of autofix suggestions
 
 {% data variables.product.prodname_dotcom %} uses an automated test harness to continuously monitor the quality of autofix suggestions. This allows us to understand how the autofix suggestions generated by the LLM change as the model develops.
 
-The test harness includes a set of over 700 JavaScript/TypeScript alerts from a diverse set of public repositories where the highlighted code has test coverage. Autofix suggestions for these alerts are tested to see how good they are, that is, how much a developer would need to edit them before committing them to the codebase. For many of the test alerts, autofixes generated by the LLM could be committed as-is to fix the alert while continuing to successfully pass all the existing CI tests.
+The test harness includes a set of over 1,000 alerts from a diverse set of public repositories where the highlighted code has test coverage. Autofix suggestions for these alerts are tested to see how good they are, that is, how much a developer would need to edit them before committing them to the codebase. For many of the test alerts, autofixes generated by the LLM could be committed as-is to fix the alert while continuing to successfully pass all the existing CI tests.
 
 In addition, the system is stress-tested to check for any potential harm (often referred to as red teaming), and a filtering system on the LLM helps prevent potentially harmful suggestions being displayed to users.
 
@@ -64,19 +70,19 @@ We test the effectiveness of autofix suggestions by merging all suggested change
 
 In addition, we spot check many of the successful suggestions and verify that they fix the alert without introducing new problems. When one or more of these checks failed, our manual triage showed that in many cases the proposed fix was nearly correct but needed some minor modifications that a user could identify and manually perform.
 
-### Effectiveness on other JavaScript/TypeScript projects
+### Effectiveness on other projects
 
-The test set contains a broad range of different types of projects and alerts. We predict that autofixes for other JavaScript/TypeScript projects should follow a similar pattern.
+The test set contains a broad range of different types of projects and alerts. We predict that autofixes for other projects using languages supported by autofix should follow a similar pattern.
 
-- Autofix is likely to add a code suggestion to the majority of alerts for JavaScript/TypeScript projects.
+- Autofix is likely to add a code suggestion to the majority of alerts.
 - When developers evaluate the autofix suggestions we expect that the majority of fixes can be committed without editing or with minor updates to reflect the wider context of the code.
 - A small percentage of suggested fixes will reflect a significant misunderstanding of the codebase or the vulnerability.
 
 However, each project and codebase is unique, so developers may need to edit a larger percentage of suggested fixes before committing them. Autofix provides valuable information to help you resolve {% data variables.product.prodname_code_scanning %} alerts, but ultimately it remains your responsibility to evaluate the proposed change and ensure the security and accuracy of your code.
 
 {% note %}
 
-**Note:** The system does not suggest fixes for all types of {% data variables.product.prodname_code_scanning %} alerts identified by {% data variables.product.prodname_codeql %}. Autofix is supported for a subset of the default {% data variables.product.prodname_codeql %} JavaScript/TypeScript queries and the LLM is limited in its operational capacity. In addition, each suggested fix is tested before it is added to a pull request. If no suggestion is available, or if the suggested fix fails internal testing, then no autofix suggestion is displayed.
+**Note:** Fix generation for supported languages is subject to LLM operational capacity. In addition, each suggested fix is tested before it is added to a pull request. If no suggestion is available, or if the suggested fix fails internal testing, then no autofix suggestion is displayed.
 
 {% endnote %}
 
@@ -86,7 +92,7 @@ When you review an autofix suggestion, you must always consider the limitations
 
 ### Limitations of autofix code suggestions
 
-- _Programming languages:_ A subset of programming languages is supported, initially only JavaScript and TypeScript. Support for additional languages will be added, but there is no intention to provide support for all {% data variables.product.prodname_codeql %} languages.
+- _Programming languages:_ A subset of programming languages is supported. Support for additional languages will be added, but there is no intention to provide support for all {% data variables.product.prodname_codeql %} languages.
 - _Human languages:_ The system primarily uses English data, including the prompts sent to the system, the code seen by the LLMs in their datasets, and the test cases used for internal evaluation. Suggestions generated by the LLM may have a lower success rate for source code and comments written in other languages and using other character sets.
 - _Syntax errors:_ The system may suggest fixes that are not syntactically correct code changes, so it is important to run syntax checks on pull requests.
 - _Location errors:_ The system may suggest fixes that are syntactically correct code but are suggested at the incorrect location, which means that if a user accepts a fix without editing the location they will introduce a syntax error.
@@ -110,7 +116,7 @@ It is important to remember that the author of a pull request retains responsibi
 
 ## Next steps
 
-{% ifversion code-scanning-autofix-js-ts %}
+{% ifversion code-scanning-autofix %}
 
 - "[AUTOTITLE](/code-security/code-scanning/managing-code-scanning-alerts/about-code-scanning-alerts)"
 - "[AUTOTITLE](/code-security/code-scanning/managing-code-scanning-alerts/triaging-code-scanning-alerts-in-pull-requests#working-with-autofix-suggestions-for-alerts)"

content/code-security/code-scanning/managing-code-scanning-alerts/triaging-code-scanning-alerts-in-pull-requests.md

@@ -110,11 +110,11 @@ You can choose to require all conversations in a pull request, including those o
 
 Anyone with push access to a pull request can fix a {% data variables.product.prodname_code_scanning %} alert that's identified on that pull request. If you commit changes to the pull request this triggers a new run of the pull request checks. If your changes fix the problem, the alert is closed and the annotation removed.
 
-{% ifversion code-scanning-autofix-js-ts %}
+{% ifversion code-scanning-autofix %}
 
 ## Working with autofix suggestions for alerts
 
-{% data reusables.rai.code-scanning.beta-autofix-js-ts %}
+{% data reusables.rai.code-scanning.beta-autofix %}
 
 Autofix is an AI-powered expansion of {% data variables.product.prodname_code_scanning %} that provides you with targeted recommendations to help you fix {% data variables.product.prodname_code_scanning %} alerts in pull requests. The potential fixes are generated automatically by large language models (LLMs) using data from the codebase, the pull request, and from {% data variables.product.prodname_codeql %} analysis.
 

data/features/code-scanning-autofix-js-ts.yml renamed to data/features/code-scanning-autofix.yml

@@ -0,0 +1,9 @@
+{% ifversion code-scanning-autofix or fpt %}
+
+{% note %}
+
+**Note:** {% data variables.product.prodname_dotcom %} autofix for {% data variables.product.prodname_code_scanning %} is in beta. Functionality and documentation are subject to change. During this phase, the feature is restricted to JavaScript, TypeScript, and Python alerts identified by {% data variables.product.prodname_codeql %}. If you have an enterprise account and use {% data variables.product.prodname_GH_advanced_security %}, you can join the [waitlist for {% data variables.product.prodname_GH_advanced_security %} AI features](https://github.com/features/preview/security).
+
+{% endnote %}
+
+{% endif %}