Skip to content

SDK 1DP Integration #40656

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 50 commits into from
Apr 29, 2025
Merged

SDK 1DP Integration #40656

merged 50 commits into from
Apr 29, 2025

Conversation

w-javed
Copy link
Contributor

@w-javed w-javed commented Apr 22, 2025

No description provided.

@github-actions github-actions bot added the Evaluation Issues related to the client library for Azure AI Evaluation label Apr 22, 2025
@azure-sdk
Copy link
Collaborator

azure-sdk commented Apr 22, 2025

API Change Check

APIView identified API level changes in this PR and created the following API reviews

azure-ai-evaluation

@w-javed w-javed marked this pull request as ready for review April 23, 2025 16:59
@Copilot Copilot AI review requested due to automatic review settings April 23, 2025 16:59
@w-javed w-javed requested a review from a team as a code owner April 23, 2025 16:59
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements the "SDK 1DP Integration" by updating parameterization in tests and modifying the evaluator and simulation code to support OneDP integration. Key changes include:

  • Updating test files to replace the generic evaluator_class with explicit evaluator instantiations and new parameter tuples for "project_scope" and "azure_cred".
  • Introducing new OneDP fixtures and credentials in conftest.py and adjusting functions to accept a union of types (str or dict) for azure_ai_project and different session types (AsyncHttpPipeline or AIProjectClient).
  • Modifying simulator, model tools, and operations code to integrate with the new OneDP client (AIProjectClient) and updating parameter names and return types accordingly.

Reviewed Changes

Copilot reviewed 18 out of 19 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
sdk/evaluation/azure-ai-evaluation/tests/e2etests/test_sim_and_eval.py Updated parameterization and evaluator usage for simulation tests.
sdk/evaluation/azure-ai-evaluation/tests/e2etests/test_mass_evaluate.py Added new parameter decorators for OneDP project scopes and credentials.
sdk/evaluation/azure-ai-evaluation/tests/conftest.py Introduced new fixtures for OneDP project scope and credential.
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/simulator/_model_tools/models.py Adjusted session type annotations to support AIProjectClient.
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/simulator/_model_tools/_proxy_completion_model.py Integrated OneDP submission flow and updated session handling.
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/rai_service.py Added OneDP request submission functions and updated common header/token handling.
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_common/onedp/operations/_operations.py Updated parameter names and return types to align with the new API.
Files not reviewed (1)
  • sdk/evaluation/azure-ai-evaluation/cspell.json: Language not supported
Comments suppressed due to low confidence (1)

sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/simulator/_model_tools/_proxy_completion_model.py:504

  • The variable 'sim_request_dto' is defined only in the if branch (when session is an instance of AIProjectClient) but is also used in the else branch. Consider defining 'sim_request_dto' appropriately or using 'request_data' directly in the else branch to avoid a runtime error.
response = await session.post(url=self.endpoint_url, headers=proxy_headers, json=sim_request_dto.to_dict())

@w-javed w-javed requested a review from a team as a code owner April 29, 2025 05:48
@weshaggard
Copy link
Member

/azp run python - pullrequest

Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@singankit singankit enabled auto-merge (squash) April 29, 2025 19:08
@singankit singankit merged commit 6ac6107 into main Apr 29, 2025
20 checks passed
@singankit singankit deleted the sdk_1dp_integration branch April 29, 2025 20:52
bambriz pushed a commit to bambriz/azure-sdk-for-python that referenced this pull request May 5, 2025
* initial changes

* fix

* fix

* fix

* fix

* rearranged

* fix

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Copilot fix

Co-authored-by: Copilot <[email protected]>

* Fix tests

* Fix tests

* Fix tests - assets

* Fix tests - assets

* Fix tests - assets

* assets

* assets

* fix-assets

* adding service check

* revert one test to avoid large size recording

* unit test fix

* unit test fix

* last asset

* red-team

* Enabling Red Teaming for 1dp

* assset change

* fix for one test

* typo fix

* fix

* dep

* asset

* CI Fix

* CI Fix

* CI Fix

* revert again

* revert

* updated asset for one test

* rollback changes

* rollback changes

---------

Co-authored-by: Copilot <[email protected]>
cRui861 pushed a commit that referenced this pull request May 14, 2025
* initial changes

* fix

* fix

* fix

* fix

* rearranged

* fix

* Fix tests

* Fix tests

* Fix tests

* Fix tests

* Copilot fix

Co-authored-by: Copilot <[email protected]>

* Fix tests

* Fix tests

* Fix tests - assets

* Fix tests - assets

* Fix tests - assets

* assets

* assets

* fix-assets

* adding service check

* revert one test to avoid large size recording

* unit test fix

* unit test fix

* last asset

* red-team

* Enabling Red Teaming for 1dp

* assset change

* fix for one test

* typo fix

* fix

* dep

* asset

* CI Fix

* CI Fix

* CI Fix

* revert again

* revert

* updated asset for one test

* rollback changes

* rollback changes

---------

Co-authored-by: Copilot <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Evaluation Issues related to the client library for Azure AI Evaluation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants