-
Notifications
You must be signed in to change notification settings - Fork 3k
SDK 1DP Integration #40656
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SDK 1DP Integration #40656
Conversation
API Change CheckAPIView identified API level changes in this PR and created the following API reviews |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR implements the "SDK 1DP Integration" by updating parameterization in tests and modifying the evaluator and simulation code to support OneDP integration. Key changes include:
- Updating test files to replace the generic evaluator_class with explicit evaluator instantiations and new parameter tuples for "project_scope" and "azure_cred".
- Introducing new OneDP fixtures and credentials in conftest.py and adjusting functions to accept a union of types (str or dict) for azure_ai_project and different session types (AsyncHttpPipeline or AIProjectClient).
- Modifying simulator, model tools, and operations code to integrate with the new OneDP client (AIProjectClient) and updating parameter names and return types accordingly.
Reviewed Changes
Copilot reviewed 18 out of 19 changed files in this pull request and generated 1 comment.
Show a summary per file
File | Description |
---|---|
sdk/evaluation/azure-ai-evaluation/tests/e2etests/test_sim_and_eval.py | Updated parameterization and evaluator usage for simulation tests. |
sdk/evaluation/azure-ai-evaluation/tests/e2etests/test_mass_evaluate.py | Added new parameter decorators for OneDP project scopes and credentials. |
sdk/evaluation/azure-ai-evaluation/tests/conftest.py | Introduced new fixtures for OneDP project scope and credential. |
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/simulator/_model_tools/models.py | Adjusted session type annotations to support AIProjectClient. |
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/simulator/_model_tools/_proxy_completion_model.py | Integrated OneDP submission flow and updated session handling. |
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/rai_service.py | Added OneDP request submission functions and updated common header/token handling. |
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/onedp/operations/_operations.py | Updated parameter names and return types to align with the new API. |
Files not reviewed (1)
- sdk/evaluation/azure-ai-evaluation/cspell.json: Language not supported
Comments suppressed due to low confidence (1)
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/simulator/_model_tools/_proxy_completion_model.py:504
- The variable 'sim_request_dto' is defined only in the if branch (when session is an instance of AIProjectClient) but is also used in the else branch. Consider defining 'sim_request_dto' appropriately or using 'request_data' directly in the else branch to avoid a runtime error.
response = await session.post(url=self.endpoint_url, headers=proxy_headers, json=sim_request_dto.to_dict())
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/rai_service.py
Outdated
Show resolved
Hide resolved
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/onedp/operations/_operations.py
Outdated
Show resolved
Hide resolved
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/onedp/operations/_operations.py
Outdated
Show resolved
Hide resolved
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/rai_service.py
Show resolved
Hide resolved
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/rai_service.py
Show resolved
Hide resolved
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/rai_service.py
Show resolved
Hide resolved
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/rai_service.py
Outdated
Show resolved
Hide resolved
...evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_common/_base_rai_svc_eval.py
Outdated
Show resolved
Hide resolved
eng/common/pipelines/templates/steps/save-package-properties.yml
Outdated
Show resolved
Hide resolved
/azp run python - pullrequest |
Azure Pipelines successfully started running 1 pipeline(s). |
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/simulator/_adversarial_simulator.py
Show resolved
Hide resolved
* initial changes * fix * fix * fix * fix * rearranged * fix * Fix tests * Fix tests * Fix tests * Fix tests * Copilot fix Co-authored-by: Copilot <[email protected]> * Fix tests * Fix tests * Fix tests - assets * Fix tests - assets * Fix tests - assets * assets * assets * fix-assets * adding service check * revert one test to avoid large size recording * unit test fix * unit test fix * last asset * red-team * Enabling Red Teaming for 1dp * assset change * fix for one test * typo fix * fix * dep * asset * CI Fix * CI Fix * CI Fix * revert again * revert * updated asset for one test * rollback changes * rollback changes --------- Co-authored-by: Copilot <[email protected]>
* initial changes * fix * fix * fix * fix * rearranged * fix * Fix tests * Fix tests * Fix tests * Fix tests * Copilot fix Co-authored-by: Copilot <[email protected]> * Fix tests * Fix tests * Fix tests - assets * Fix tests - assets * Fix tests - assets * assets * assets * fix-assets * adding service check * revert one test to avoid large size recording * unit test fix * unit test fix * last asset * red-team * Enabling Red Teaming for 1dp * assset change * fix for one test * typo fix * fix * dep * asset * CI Fix * CI Fix * CI Fix * revert again * revert * updated asset for one test * rollback changes * rollback changes --------- Co-authored-by: Copilot <[email protected]>
No description provided.