test pass / fail based on other tests

[rrd108]

Runing vendor/bin/phpunit tests/TestCase/Model passes, but running vendor/bin/phpunit tests/TestCase fails.

The error message is:

There was 1 error:

1) App\Test\TestCase\Model\Table\ItemsTableTest::testFindForPartner
CakephpFixtureFactories\Error\PersistenceException: Error in Factory App\Test\Factory\InvoiceFactory.
 Message: SQLSTATE[23000]: Integrity constraint violation: 1452 Cannot add or update a child row: a foreign key constraint fails (`laksmi3_test`.`items`, CONSTRAINT `items_ibfk_5` FOREIGN KEY (`invoice_id`) REFERENCES `invoices` (`id`) ON DELETE NO ACTION ON UPDATE NO ACTION) 


/home/rrd/public_html/laksmi3/api/vendor/vierge-noire/cakephp-fixture-factories/src/Factory/BaseFactory.php:310
/home/rrd/public_html/laksmi3/api/tests/TestCase/Model/Table/ItemsTableTest.php:177

I have controller and event tests also. May any of them cause a table test fail? Like leaving some data in the database?

I use TruncateDirtyTables in the table tests, but not in the others. Should I use it? OR is not related?

[rrd108]

Hm. It seems it happens because of the events tests.

I have 2 event tests. If both are present the test above fails. If there is only any of them the test passes.

[pabloelcolombiano]

I'll let you further investigate for the moment, alright? Let me know if you are blocked and if you see anything we can improve on the plugin side.

[rrd108]

I spent quite a time trying to figure out what happens here.

The events are not related to the table tests, they are fired by a controller, not any models, so I do not know how they even connected.

I used my debugger to stop at the very beginning of the failing test, and checked the database tables. It seems I have the same data there when I run only the affected test (what passes) and when I run all tests (when the affected one fails). So it seems it is not caused by some data "stucked" in the database tables.

I added TruncateDirtyTables to the (theoretically unrelated) events tests, no change.

The affected test is the only one where I use dot notation in the with() call.

InvoiceFactory::make(['partnerName' => 'Webmania'], 2)
  ->with('Companies', $this->company)
  ->with('Items.Departments.Corporates', $this->corporate)
  ->persist();

I guess the next step would be trying to create a failing test for you, with the plugin's test environment, right?

The problem is that I did not find out how the event tests can have an effect on this one..
Last time I spend quite a time on that, so I would like to make sure I did everything well and it is not a user error :)

[pabloelcolombiano]

Thanks for digging!
I would suggest that you reduce your example to the simplest possible.
The issue you were having was on items.invoice_id, right? What is the simplest factory you come up qith that creates the issue? E.g. is the Companies association relevant? Is the 2 relevant? Is the Corporatesrelevant?

From here I cannot see where your issue could come from :-/

[rrd108]

I was not able to find any connection between the factory setup and the failing test.

BUT if I move the failing test method before the previous method, or after the next method all is fine... 😱

I have 15 methods in this file (+setUp and tearDown) and it only fails if it is the 6th.

I can not see any logical explanation how does it happen.

[pabloelcolombiano]

Any news here? Should I close the issue for now?

[rrd108]

As I can not find any other way to reproduce the issue except this clearly illogical one...

If it pops up again, I will give it another try.

[rrd108]

I run into the same error with another small application. I tried to track down what causes the issue but I was not successful.

My minimal replication app has 3 Model classes, 2 Factories and 2 Test files with 4 tests. If I touch it the problem disappears.

Can you take a look? I guess you will find out what I am doing wrong. I did not see any other issues from other users, so I think I miss something, but after several tries I was not able to find it.

If you open to look at it I upload my code to github and write down the steps of reproduction.

[pabloelcolombiano]

I @rrd108 ,

sure feel free to share your code!

[rrd108]

Ah great. I really appreciate.

Here is the repo: https://github.com/rrd108/cakephp-fixture-factories-error

1. Clone the repo
2. Run composer install
3. Create a MySQL database for the app and the tests
4. Change the MySQL db name, user and pass for the tests in app_local.php
5. Run the following tests:
   5.1. vendor/bin/phpunit tests/TestCase/Model/Table/CarsTableTest.php - will pass
   5.2. vendor/bin/phpunit tests/TestCase/Model/Table/BookingsTableTest.php - will pass
   5.3. vendor/bin/phpunit tests/TestCase/Model/Table/ - will fail

The last test fails for me with this error

1) App\Test\TestCase\Model\Table\CarsTableTest::testFindAvailableNoOverlappingBooking
CakephpFixtureFactories\Error\PersistenceException: Error in Factory App\Test\Factory\BookingFactory.
 Message: SQLSTATE[23000]: Integrity constraint violation: 1452 Cannot add or update a child row: a foreign key constraint fails (`x_test`.`bookings`, CONSTRAINT `fk_bookings_prices` FOREIGN KEY (`price_id`) REFERENCES `prices` (`id`)) 

6. Now comment out testFindForTimes in BookingsTableTest and run vendor/bin/phpunit tests/TestCase/Model/Table/ again - will pass

What I expect to happen?

1. As 5.3 do not execute anything else then 5.1 and 5.2 it should also pass, but it fails.
2. Commenting out (or change the order of test methods) should not create any errors.

[rrd108]

Did you manged to reproduce it?

[rrd108]

Is there anything I can help to find out what causes this?