AIDM-529: Ensuring latest version of composer package are tested #3041

PROFeNoM · 2025-01-16T15:06:45Z

Description

Because of the amount of changes associated with this PR, I’ll describe below as exhaustively as I can all the changes worth considering:

MAIN CHANGES

tests/Common/IntegrationTestCase.php

A new method has been introduced: getTestedLibrary.
By default, this method looks at the location of the test file for a composer.json. This is the case for integration tests. It will use the first required package as the tested library if it finds one.
This method is meant to be overridden by test cases in other cases where the tested library is not as easily found. For instance, for web tests, and where applicable, this method has been overridden to return the tested library (e.g., laravel/framework).

In the future, when creating a new test case:

If this is an integration test for a composer package, then nothing more than what we currently do will have to be done
If this is a web test for a composer-based framework, then getTestedLibrary will have to be overridden with the relevant library.
Nota Bene: Non-composer-based frameworks (e.g., WordPress) or extensions will be addressed in later PRs.

tests/PackageUpdater.php

This file is responsible for:

Finding relevant composer.json files and libraries to be updated
Retrieving the latest, stable release of the composer package
Update the value of the version in the composer.json if it is different

This file is used in the Github Action job (see below - update_latest_versions.yml) to create the PR and update the pins.

Sample Script Logs

Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/AMQP/Latest/AMQPTest.php
[9](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:10)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/CLI/CakePHP/Latest/CommonScenariosTest.php
[10](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:11)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/CLI/Laravel/Latest/CommonScenariosTest.php
[11](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:12)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/CLI/Symfony/Latest/CommonScenariosTest.php
[12](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:13)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/CakePHP/Latest/CommonScenariosTest.php
[13](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:14)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Elasticsearch/Latest/ElasticSearchIntegrationTest.php
[14](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:15)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/GoogleSpanner/Latest/GoogleSpannerIntegrationTest.php
[15](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:16)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Guzzle/Latest/GuzzleIntegrationTest.php
[16](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:17)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Laminas/ApiTools/Latest/RESTTest.php
[17](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:18)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Laminas/Mvc/Latest/CommonScenariosTest.php
[18](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:19)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Laravel/Latest/CommonScenariosTest.php
[19](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:20)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Laravel/Octane/Latest/CommonScenariosTest.php
[20](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:21)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Logs/MonologLatest/MonologLatestTest.php
[21](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:22)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/MongoDB/Latest/MongoDBTest.php
[22](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:23)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Nette/Latest/NetteTest.php
[23](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:24)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/OpenAI/Latest/OpenAITest.php
[24](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:25)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Predis/Latest/PredisTest.php
[25](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:26)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Slim/Latest/CommonScenariosTest.php
[26](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:27)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Symfony/Latest/AutomatedLoginEventsTest.php
[27](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:28)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Symfony/Latest/CommonScenariosTest.php
[28](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:29)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Symfony/Latest/ConsoleCommandTest.php
[29](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:30)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Symfony/Latest/MessengerTest.php
[30](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:31)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Symfony/Latest/TraceSearchConfigTest.php
[31](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:32)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Yii/Latest/CommonScenariosTest.php
[32](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:33)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Yii/Latest/LazyLoadingIntegrationsFromYiiTest.php
[33](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:34)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Yii/Latest/ModuleTest.php
[34](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:35)
Processing: /home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Yii/Latest/ParameterizedRouteTest.php
[35](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:36)

[36](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:37)
Packages updated:
[37](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:38)
- cakephp/cakephp: 5.1.4 → 5.1.5 (/home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/CakePHP/Latest/../../../Frameworks/CakePHP/Latest/composer.json)
[38](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:39)
- google/cloud-spanner: 1.80.0 → 1.92.0 (/home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/GoogleSpanner/Latest/composer.json)
[39](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:40)
- laminas-api-tools/api-tools: 1.6.0 → 1.7.0 (/home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Laminas/ApiTools/Latest/../../../../Frameworks/Laminas/ApiTools/Latest/composer.json)
[40](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:41)
- laminas/laminas-mvc: 3.6.0 → 3.8.0 (/home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Laminas/Mvc/Latest/../../../../Frameworks/Laminas/Mvc/Latest/composer.json)
[41](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:42)
- laravel/framework: 11.38.2 → 11.39.0 (/home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Laravel/Latest/../../../Frameworks/Laravel/Latest/composer.json)
[42](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:43)
- laravel/octane: 2.3.0 → 2.6.0 (/home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Laravel/Octane/Latest/../../../../Frameworks/Laravel/Octane/Latest/composer.json)
[43](https://github.com/DataDog/dd-trace-php/actions/runs/12911021407/job/36002454286?pr=3041#step:6:44)
- predis/predis: 1.1.10 → 2.3.0 (/home/runner/work/dd-trace-php/dd-trace-php/tests/Integrations/Predis/Latest/composer.json)

.github/workflows/update_latest_versions.yml

This file contains the Github Action job that will be run weekly on Sunday at Midnight (or manually).
This file will execute this aforementioned package updater script and create a PR in case of changes. This PR will effectively update the test suite to be tested against the latest version available.

Here is a sample job: link
Here is a sample PR created by the script: #3054

tests/**/Latest/**

The tests/ folder architecture has been slightly modified. All composer-based libraries/frameworks will now contain a Latest/ folder. Under the latter, the test suite will be run against the latest pinned version.

This means that the versions of the currently tested packages that were executed against the latest (e.g., tests/Frameworks/Laravel/Version_11_x) have been moved to a Latest/ folder (e.g., tests/Frameworks/Laravel/Latest). The same goes for the test suite.

Let’s suppose that tomorrow, Laravel V12 will be released. Because of composers’ version constraints, the test_web_laravel_latest will fail. At that point, it will be our duty to take manual action and:

Create a tests/Framework/Laravel/Version_11_x folder with the V11 of the framework;
Fix either the integration or the test framework to continue testing against V12;
Update the makefile with a new test_web_laravel_11x rule.

OTHER CHANGES

tests/Common/WebFrameworkTestCase.php

The visibility of getAppIndexScript has changed from protected to public. This change allows it to be used in tests/PackageUpdater.php to automatically retrieve the relevant composer.json file without further intervention.

Makefile

Now, there will be a test_.*_latest rule (e.g., test_web_laravel_latest). This rule will, therefore, test the latest pinned version.
Because 1. this version is pinned, and 2. not all libraries follow semantic versioning, some PHP runtime-breaking changes occurred in some libraries. When this happened, a new test case was added as a regression.

tests/snapshots/**

Because paths have been modified by being moved under Latest/, some snapshots needed to be modified accordingly.

Reviewer checklist

Test coverage seems ok.
Appropriate labels assigned.

github-actions · 2025-01-16T16:02:32Z

Snapshots difference summary

The following differences have been observed in committed snapshots. It is meant to help the reviewer.
The diff is simplistic, so please check some files anyway while we improve it.

If you need to update snapshots, please refer to CONTRIBUTING.md

1 occurrences of :

- "error.message": "Uncaught OpenAI\\Exceptions\\ErrorException: The server had an error while processing your request. Sorry about that! in /home/circleci/app/tests/Integrations/OpenAI/vendor/openai-php/client/src/Responses/StreamResponse.php:60"
- "error.stack": "#0 [internal function]: OpenAI\\Responses\\StreamResponse->getIterator()\n#1 /home/circleci/app/src/DDTrace/Integrations/OpenAI/OpenAIIntegration.php(1079): Generator->valid()\n#2 /home/circleci/app/src/DDTrace/Integrations/OpenAI/OpenAIIntegration.php(1033): DDTrace\\Integrations\\OpenAI\\OpenAIIntegration::readAndStoreStreamedResponse()\n#3 /home/circleci/app/src/DDTrace/Integrations/OpenAI/OpenAIIntegration.php(204): DDTrace\\Integrations\\OpenAI\\OpenAIIntegration::handleStreamedResponse()\n#4 /home/circleci/app/tests/Integrations/OpenAI/vendor/openai-php/client/src/Resources/Chat.php(54): OpenAI\\Resources\\Chat->DDTrace\\Integrations\\OpenAI\\{closure}()\n#5 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(100): OpenAI\\Resources\\Chat->createStreamed()\n#6 /home/circleci/app/tests/Common/SnapshotTestTrait.php(347): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->DDTrace\\Tests\\Integrations\\OpenAI\\{closure}()\n#7 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(93): DDTrace\\Tests\\Common\\IntegrationTestCase->isolateTracerSnapshot()\n#8 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(455): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->callStreamed()\n#9 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(1617): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->testCreateChatCompletionStreamWithError()\n#10 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(1223): PHPUnit\\Framework\\TestCase->runTest()\n#11 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestResult.php(729): PHPUnit\\Framework\\TestCase->runBare()\n#12 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(973): PHPUnit\\Framework\\TestResult->run()\n#13 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestSuite.php(685): PHPUnit\\Framework\\TestCase->run()\n#14 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestSuite.php(685): PHPUnit\\Framework\\TestSuite->run()\n#15 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/TestRunner.php(651): PHPUnit\\Framework\\TestSuite->run()\n#16 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/Command.php(146): PHPUnit\\TextUI\\TestRunner->run()\n#17 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/Command.php(99): PHPUnit\\TextUI\\Command->run()\n#18 /home/circleci/app/tests/vendor/phpunit/phpunit/phpunit(107): PHPUnit\\TextUI\\Command::main()\n#19 /home/circleci/app/tests/vendor/bin/phpunit(122): include()\n#20 {main}"
+ "error.message": "Uncaught OpenAI\\Exceptions\\ErrorException: The server had an error while processing your request. Sorry about that! in /home/circleci/app/tests/Integrations/OpenAI/Latest/vendor/openai-php/client/src/Responses/StreamResponse.php:60"
+ "error.stack": "#0 [internal function]: OpenAI\\Responses\\StreamResponse->getIterator()\n#1 /home/circleci/app/src/DDTrace/Integrations/OpenAI/OpenAIIntegration.php(1079): Generator->valid()\n#2 /home/circleci/app/src/DDTrace/Integrations/OpenAI/OpenAIIntegration.php(1033): DDTrace\\Integrations\\OpenAI\\OpenAIIntegration::readAndStoreStreamedResponse()\n#3 /home/circleci/app/src/DDTrace/Integrations/OpenAI/OpenAIIntegration.php(204): DDTrace\\Integrations\\OpenAI\\OpenAIIntegration::handleStreamedResponse()\n#4 /home/circleci/app/tests/Integrations/OpenAI/Latest/vendor/openai-php/client/src/Resources/Chat.php(54): OpenAI\\Resources\\Chat->DDTrace\\Integrations\\OpenAI\\{closure}()\n#5 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(100): OpenAI\\Resources\\Chat->createStreamed()\n#6 /home/circleci/app/tests/Common/SnapshotTestTrait.php(347): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->DDTrace\\Tests\\Integrations\\OpenAI\\{closure}()\n#7 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(93): DDTrace\\Tests\\Common\\IntegrationTestCase->isolateTracerSnapshot()\n#8 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(455): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->callStreamed()\n#9 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(1617): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->testCreateChatCompletionStreamWithError()\n#10 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(1223): PHPUnit\\Framework\\TestCase->runTest()\n#11 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestResult.php(729): PHPUnit\\Framework\\TestCase->runBare()\n#12 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(973): PHPUnit\\Framework\\TestResult->run()\n#13 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestSuite.php(685): PHPUnit\\Framework\\TestCase->run()\n#14 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestSuite.php(685): PHPUnit\\Framework\\TestSuite->run()\n#15 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/TestRunner.php(651): PHPUnit\\Framework\\TestSuite->run()\n#16 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/Command.php(146): PHPUnit\\TextUI\\TestRunner->run()\n#17 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/Command.php(99): PHPUnit\\TextUI\\Command->run()\n#18 /home/circleci/app/tests/vendor/phpunit/phpunit/phpunit(107): PHPUnit\\TextUI\\Command::main()\n#19 /home/circleci/app/tests/vendor/bin/phpunit(122): include()\n#20 {main}"

1 occurrences of :

- "error.message": "Uncaught OpenAI\\Exceptions\\ErrorException: Invalid schema for function 'get_current_weather':\nIn context=('properties', 'location'), array schema missing items in /home/circleci/app/tests/Integrations/OpenAI/vendor/openai-php/client/src/Transporters/HttpTransporter.php:133"
- "error.stack": "#0 /home/circleci/app/tests/Integrations/OpenAI/vendor/openai-php/client/src/Transporters/HttpTransporter.php(57): OpenAI\\Transporters\\HttpTransporter->throwIfJsonError()\n#1 /home/circleci/app/tests/Integrations/OpenAI/vendor/openai-php/client/src/Resources/Completions.php(33): OpenAI\\Transporters\\HttpTransporter->requestObject()\n#2 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(81): OpenAI\\Resources\\Completions->create()\n#3 /home/circleci/app/tests/Common/SnapshotTestTrait.php(347): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->DDTrace\\Tests\\Integrations\\OpenAI\\{closure}()\n#4 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(76): DDTrace\\Tests\\Common\\IntegrationTestCase->isolateTracerSnapshot()\n#5 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(468): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->call()\n#6 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(1617): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->testCreateCompletionsWithMultipleErrorMessages()\n#7 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(1223): PHPUnit\\Framework\\TestCase->runTest()\n#8 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestResult.php(729): PHPUnit\\Framework\\TestCase->runBare()\n#9 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(973): PHPUnit\\Framework\\TestResult->run()\n#10 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestSuite.php(685): PHPUnit\\Framework\\TestCase->run()\n#11 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestSuite.php(685): PHPUnit\\Framework\\TestSuite->run()\n#12 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/TestRunner.php(651): PHPUnit\\Framework\\TestSuite->run()\n#13 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/Command.php(146): PHPUnit\\TextUI\\TestRunner->run()\n#14 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/Command.php(99): PHPUnit\\TextUI\\Command->run()\n#15 /home/circleci/app/tests/vendor/phpunit/phpunit/phpunit(107): PHPUnit\\TextUI\\Command::main()\n#16 /home/circleci/app/tests/vendor/bin/phpunit(122): include()\n#17 {main}"
+ "error.message": "Uncaught OpenAI\\Exceptions\\ErrorException: Invalid schema for function 'get_current_weather':\nIn context=('properties', 'location'), array schema missing items in /home/circleci/app/tests/Integrations/OpenAI/Latest/vendor/openai-php/client/src/Transporters/HttpTransporter.php:133"
+ "error.stack": "#0 /home/circleci/app/tests/Integrations/OpenAI/Latest/vendor/openai-php/client/src/Transporters/HttpTransporter.php(57): OpenAI\\Transporters\\HttpTransporter->throwIfJsonError()\n#1 /home/circleci/app/tests/Integrations/OpenAI/Latest/vendor/openai-php/client/src/Resources/Completions.php(33): OpenAI\\Transporters\\HttpTransporter->requestObject()\n#2 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(81): OpenAI\\Resources\\Completions->create()\n#3 /home/circleci/app/tests/Common/SnapshotTestTrait.php(347): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->DDTrace\\Tests\\Integrations\\OpenAI\\{closure}()\n#4 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(76): DDTrace\\Tests\\Common\\IntegrationTestCase->isolateTracerSnapshot()\n#5 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(468): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->call()\n#6 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(1617): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->testCreateCompletionsWithMultipleErrorMessages()\n#7 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(1223): PHPUnit\\Framework\\TestCase->runTest()\n#8 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestResult.php(729): PHPUnit\\Framework\\TestCase->runBare()\n#9 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(973): PHPUnit\\Framework\\TestResult->run()\n#10 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestSuite.php(685): PHPUnit\\Framework\\TestCase->run()\n#11 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestSuite.php(685): PHPUnit\\Framework\\TestSuite->run()\n#12 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/TestRunner.php(651): PHPUnit\\Framework\\TestSuite->run()\n#13 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/Command.php(146): PHPUnit\\TextUI\\TestRunner->run()\n#14 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/Command.php(99): PHPUnit\\TextUI\\Command->run()\n#15 /home/circleci/app/tests/vendor/phpunit/phpunit/phpunit(107): PHPUnit\\TextUI\\Command::main()\n#16 /home/circleci/app/tests/vendor/bin/phpunit(122): include()\n#17 {main}"

1 occurrences of :

- "error.message": "Uncaught OpenAI\\Exceptions\\ErrorException: Incorrect API key provided: foo. You can find your API key at https://platform.openai.com. in /home/circleci/app/tests/Integrations/OpenAI/vendor/openai-php/client/src/Transporters/HttpTransporter.php:133"
- "error.stack": "#0 /home/circleci/app/tests/Integrations/OpenAI/vendor/openai-php/client/src/Transporters/HttpTransporter.php(57): OpenAI\\Transporters\\HttpTransporter->throwIfJsonError()\n#1 /home/circleci/app/tests/Integrations/OpenAI/vendor/openai-php/client/src/Resources/Models.php(28): OpenAI\\Transporters\\HttpTransporter->requestObject()\n#2 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(83): OpenAI\\Resources\\Models->list()\n#3 /home/circleci/app/tests/Common/SnapshotTestTrait.php(347): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->DDTrace\\Tests\\Integrations\\OpenAI\\{closure}()\n#4 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(76): DDTrace\\Tests\\Common\\IntegrationTestCase->isolateTracerSnapshot()\n#5 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(463): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->call()\n#6 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(1617): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->testListModelsWithError()\n#7 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(1223): PHPUnit\\Framework\\TestCase->runTest()\n#8 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestResult.php(729): PHPUnit\\Framework\\TestCase->runBare()\n#9 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(973): PHPUnit\\Framework\\TestResult->run()\n#10 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestSuite.php(685): PHPUnit\\Framework\\TestCase->run()\n#11 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestSuite.php(685): PHPUnit\\Framework\\TestSuite->run()\n#12 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/TestRunner.php(651): PHPUnit\\Framework\\TestSuite->run()\n#13 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/Command.php(146): PHPUnit\\TextUI\\TestRunner->run()\n#14 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/Command.php(99): PHPUnit\\TextUI\\Command->run()\n#15 /home/circleci/app/tests/vendor/phpunit/phpunit/phpunit(107): PHPUnit\\TextUI\\Command::main()\n#16 /home/circleci/app/tests/vendor/bin/phpunit(122): include()\n#17 {main}"
+ "error.message": "Uncaught OpenAI\\Exceptions\\ErrorException: Incorrect API key provided: foo. You can find your API key at https://platform.openai.com. in /home/circleci/app/tests/Integrations/OpenAI/Latest/vendor/openai-php/client/src/Transporters/HttpTransporter.php:133"
+ "error.stack": "#0 /home/circleci/app/tests/Integrations/OpenAI/Latest/vendor/openai-php/client/src/Transporters/HttpTransporter.php(57): OpenAI\\Transporters\\HttpTransporter->throwIfJsonError()\n#1 /home/circleci/app/tests/Integrations/OpenAI/Latest/vendor/openai-php/client/src/Resources/Models.php(28): OpenAI\\Transporters\\HttpTransporter->requestObject()\n#2 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(83): OpenAI\\Resources\\Models->list()\n#3 /home/circleci/app/tests/Common/SnapshotTestTrait.php(347): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->DDTrace\\Tests\\Integrations\\OpenAI\\{closure}()\n#4 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(76): DDTrace\\Tests\\Common\\IntegrationTestCase->isolateTracerSnapshot()\n#5 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(463): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->call()\n#6 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(1617): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->testListModelsWithError()\n#7 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(1223): PHPUnit\\Framework\\TestCase->runTest()\n#8 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestResult.php(729): PHPUnit\\Framework\\TestCase->runBare()\n#9 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(973): PHPUnit\\Framework\\TestResult->run()\n#10 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestSuite.php(685): PHPUnit\\Framework\\TestCase->run()\n#11 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestSuite.php(685): PHPUnit\\Framework\\TestSuite->run()\n#12 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/TestRunner.php(651): PHPUnit\\Framework\\TestSuite->run()\n#13 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/Command.php(146): PHPUnit\\TextUI\\TestRunner->run()\n#14 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/Command.php(99): PHPUnit\\TextUI\\Command->run()\n#15 /home/circleci/app/tests/vendor/phpunit/phpunit/phpunit(107): PHPUnit\\TextUI\\Command::main()\n#16 /home/circleci/app/tests/vendor/bin/phpunit(122): include()\n#17 {main}"

1 occurrences of :

- "error.message": "Uncaught OpenAI\\Exceptions\\ErrorException: You exceeded your current quota, please check in /home/circleci/app/tests/Integrations/OpenAI/vendor/openai-php/client/src/Transporters/HttpTransporter.php:133"
- "error.stack": "#0 /home/circleci/app/tests/Integrations/OpenAI/vendor/openai-php/client/src/Transporters/HttpTransporter.php(57): OpenAI\\Transporters\\HttpTransporter->throwIfJsonError()\n#1 /home/circleci/app/tests/Integrations/OpenAI/vendor/openai-php/client/src/Resources/Models.php(28): OpenAI\\Transporters\\HttpTransporter->requestObject()\n#2 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(83): OpenAI\\Resources\\Models->list()\n#3 /home/circleci/app/tests/Common/SnapshotTestTrait.php(347): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->DDTrace\\Tests\\Integrations\\OpenAI\\{closure}()\n#4 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(76): DDTrace\\Tests\\Common\\IntegrationTestCase->isolateTracerSnapshot()\n#5 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(473): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->call()\n#6 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(1617): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->testListModelsWithNullErrorType()\n#7 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(1223): PHPUnit\\Framework\\TestCase->runTest()\n#8 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestResult.php(729): PHPUnit\\Framework\\TestCase->runBare()\n#9 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(973): PHPUnit\\Framework\\TestResult->run()\n#10 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestSuite.php(685): PHPUnit\\Framework\\TestCase->run()\n#11 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestSuite.php(685): PHPUnit\\Framework\\TestSuite->run()\n#12 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/TestRunner.php(651): PHPUnit\\Framework\\TestSuite->run()\n#13 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/Command.php(146): PHPUnit\\TextUI\\TestRunner->run()\n#14 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/Command.php(99): PHPUnit\\TextUI\\Command->run()\n#15 /home/circleci/app/tests/vendor/phpunit/phpunit/phpunit(107): PHPUnit\\TextUI\\Command::main()\n#16 /home/circleci/app/tests/vendor/bin/phpunit(122): include()\n#17 {main}"
+ "error.message": "Uncaught OpenAI\\Exceptions\\ErrorException: You exceeded your current quota, please check in /home/circleci/app/tests/Integrations/OpenAI/Latest/vendor/openai-php/client/src/Transporters/HttpTransporter.php:133"
+ "error.stack": "#0 /home/circleci/app/tests/Integrations/OpenAI/Latest/vendor/openai-php/client/src/Transporters/HttpTransporter.php(57): OpenAI\\Transporters\\HttpTransporter->throwIfJsonError()\n#1 /home/circleci/app/tests/Integrations/OpenAI/Latest/vendor/openai-php/client/src/Resources/Models.php(28): OpenAI\\Transporters\\HttpTransporter->requestObject()\n#2 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(83): OpenAI\\Resources\\Models->list()\n#3 /home/circleci/app/tests/Common/SnapshotTestTrait.php(347): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->DDTrace\\Tests\\Integrations\\OpenAI\\{closure}()\n#4 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(76): DDTrace\\Tests\\Common\\IntegrationTestCase->isolateTracerSnapshot()\n#5 /home/circleci/app/tests/Integrations/OpenAI/OpenAITest.php(473): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->call()\n#6 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(1617): DDTrace\\Tests\\Integrations\\OpenAI\\OpenAITest->testListModelsWithNullErrorType()\n#7 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(1223): PHPUnit\\Framework\\TestCase->runTest()\n#8 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestResult.php(729): PHPUnit\\Framework\\TestCase->runBare()\n#9 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestCase.php(973): PHPUnit\\Framework\\TestResult->run()\n#10 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestSuite.php(685): PHPUnit\\Framework\\TestCase->run()\n#11 /home/circleci/app/tests/vendor/phpunit/phpunit/src/Framework/TestSuite.php(685): PHPUnit\\Framework\\TestSuite->run()\n#12 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/TestRunner.php(651): PHPUnit\\Framework\\TestSuite->run()\n#13 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/Command.php(146): PHPUnit\\TextUI\\TestRunner->run()\n#14 /home/circleci/app/tests/vendor/phpunit/phpunit/src/TextUI/Command.php(99): PHPUnit\\TextUI\\Command->run()\n#15 /home/circleci/app/tests/vendor/phpunit/phpunit/phpunit(107): PHPUnit\\TextUI\\Command::main()\n#16 /home/circleci/app/tests/vendor/bin/phpunit(122): include()\n#17 {main}"

codecov-commenter · 2025-01-16T16:59:51Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.77%. Comparing base (e44d78d) to head (52fb7c0).
Report is 1 commits behind head on master.

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #3041      +/-   ##
============================================
- Coverage     74.79%   74.77%   -0.02%     
  Complexity     2787     2787              
============================================
  Files           112      112              
  Lines         11033    11033              
============================================
- Hits           8252     8250       -2     
- Misses         2781     2783       +2

Flag	Coverage Δ
tracer-php	`74.77% <ø> (-0.02%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 1 file with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e44d78d...52fb7c0. Read the comment docs.

…IDM-529_testing-latest

bouwkast

Awesome!

Just a few comments :)

bouwkast · 2025-01-23T15:42:48Z

.github/workflows/update_latest_versions.yml

+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0


nitpicky but do you need to checkout the entire history?
1 is the default and just grabs the latest

bouwkast · 2025-01-23T15:43:36Z

.github/workflows/update_latest_versions.yml

+          title: "chore: update latest versions"
+          branch: "update-latest-versions"
+          base: "master"
+          draft: true


Assuming that will be removed at a future date to become a non-draft PR?

bouwkast · 2025-01-23T15:47:51Z

tests/PackageUpdater.php

+class PackageUpdater
+{
+    private const INTEGRATIONS_DIR = __DIR__ . '/Integrations';
+    private const PACKAGIST_API_URL = 'https://repo.packagist.org/p2/%s.json';


Reading through their API documentation I don't see much mention of a hard rate limit, have you ran into anything locally? Just curious.

Seeing this in their documentation:

If you do scheduled jobs, avoid running things at midnight or once an hour at XX:00. Most people do so and we do see traffic peaks every hour. Pick a "random" time by hand for your cron jobs, or even better (if you can) is to make it run on a really randomized schedule.

Very nice to have thought about that 👍

quinna-h · 2025-01-24T14:47:55Z

tests/PackageUpdater.php

+        $latestVersion = $this->getLatestVersion($library);
+        if (!$latestVersion) return;


question: does this mean that in the case of rate limits / timeouts from the PACKAGIST_API_URL it does nothing?

Currently yes... Perhaps you're considering throwing an exception there so that we're more easily aware?

yes, maybe some logging / exception throwing for awareness (that we aren't updating the library because we got a non-OK response)

.github/workflows/update_latest_versions.yml

quinna-h

Looks good to me! I reviewed the tests/PackageUpdater.php and update_latest_versions.yml files

...ots/tests.integrations.laravel.latest.common_scenarios_test.test_scenario_get_with_view.json

PROFeNoM added 14 commits January 15, 2025 15:48

Add Base

7d7564c

Create Pull Request

cf2e894

Update to v7

8484e61

Copy composer to /tmp/

7f74f1f

Set insert_final_newline to false

1ea6eda

Use Github Actions

c2035b4

typo

e2418f8

Remove composer install step

f2b4748

Set Laravel's insert_final_newline to false

bbecea3

Add some Latest/

31d0eab

Update Latest Version Dependencies

00dbb8c

Prettify script + summary

fba243f

Check for Latest in basename

32c4e6d

newline

aa127df

PROFeNoM self-assigned this Jan 16, 2025

Stup Nette & Laminas latests frameworks

029a91f

PROFeNoM added 12 commits January 17, 2025 08:51

Setup Yii latest framework

acf1d0c

Setup Yii latest framework

44cf2a5

Fix Laminas calls

56e9dfe

Makefile fixes

503d00f

update openai snapshots

15cd02a

fix OpenAI fixture path

b9b6c07

fix Guzzle latest snapshot name

c3b825d

fix Predis Tests

f412b76

Don't try to fix predis in this PR

879dd85

Fix nette test range

8220a26

Fix laminas test range

4fa69d5

Update snapshots & namespaces

912710c

PROFeNoM added 15 commits January 21, 2025 14:25

Laravel Octane Flakiness

ba3b8a9

Add missing breaking intermediate versions

29b878c

Remove class hardcoding in mongodb tests

990ed24

fix mongodb & yii tests

64db5bf

fix yii & slim composer constraints

e342923

fix yii path

93065f1

fix imports

4ba0a8f

fix mongodb namespaces

0455221

fix mongodb imports

c07146b

fix mongodb imports

4932d87

push .env (Octane)

83bef9e

Merge remote-tracking branch 'refs/remotes/origin/master' into alex/A…

73977f3

…IDM-529_testing-latest

Update latest pins

1a4b417

Generate test PR

d268340

Resolve Github Action's TODOs

2c01977

PROFeNoM requested review from bouwkast and quinna-h January 23, 2025 08:38

PROFeNoM marked this pull request as ready for review January 23, 2025 08:42

PROFeNoM requested a review from a team as a code owner January 23, 2025 08:42

bouwkast approved these changes Jan 23, 2025

View reviewed changes

(cr) Modify update_latest_versions.yml

35c6dc7

quinna-h reviewed Jan 24, 2025

View reviewed changes

.github/workflows/update_latest_versions.yml Show resolved Hide resolved

quinna-h approved these changes Jan 24, 2025

View reviewed changes

PROFeNoM commented Jan 24, 2025

View reviewed changes

...ots/tests.integrations.laravel.latest.common_scenarios_test.test_scenario_get_with_view.json Outdated Show resolved Hide resolved

PROFeNoM added 2 commits January 27, 2025 08:54

throw an exception on library version retrieval failure

d11bc8a

Update Laravel snapshots

52fb7c0

PROFeNoM merged commit a958dad into master Jan 28, 2025
669 of 699 checks passed

PROFeNoM deleted the alex/AIDM-529_testing-latest branch January 28, 2025 09:35

github-actions bot added this to the 1.7.0 milestone Jan 28, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AIDM-529: Ensuring latest version of composer package are tested #3041

AIDM-529: Ensuring latest version of composer package are tested #3041

PROFeNoM commented Jan 16, 2025 •

edited

Loading

github-actions bot commented Jan 16, 2025 •

edited

Loading

codecov-commenter commented Jan 16, 2025 •

edited

Loading

bouwkast left a comment

bouwkast Jan 23, 2025

bouwkast Jan 23, 2025

PROFeNoM Jan 24, 2025

bouwkast Jan 23, 2025

bouwkast Jan 23, 2025

PROFeNoM Jan 24, 2025

quinna-h Jan 24, 2025

PROFeNoM Jan 24, 2025

quinna-h Jan 24, 2025

quinna-h left a comment

		$latestVersion = $this->getLatestVersion($library);
		if (!$latestVersion) return;

AIDM-529: Ensuring latest version of composer package are tested #3041

AIDM-529: Ensuring latest version of composer package are tested #3041

Conversation

PROFeNoM commented Jan 16, 2025 • edited Loading

Description

Reviewer checklist

github-actions bot commented Jan 16, 2025 • edited Loading

Snapshots difference summary

codecov-commenter commented Jan 16, 2025 • edited Loading

Codecov Report

bouwkast left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

quinna-h left a comment

Choose a reason for hiding this comment

PROFeNoM commented Jan 16, 2025 •

edited

Loading

github-actions bot commented Jan 16, 2025 •

edited

Loading

codecov-commenter commented Jan 16, 2025 •

edited

Loading