Bound admin product grid keyword search to the paged query (#36408)

[lbajsarowicz]

Description (*)

The admin product grid keyword filter (AddFulltextFilterToCollection) ran the LIKE-based search collection with load(), instantiating a product model for every match, then getAllIds() ran the same unlimited query a second time, and the resulting id array was passed back to MySQL as a literal IN (...) list on the grid query. Paging applied only after all of that. At 50k products that is around 600 ms per keystroke (#36408); at millions of products it is a timeout and a memory blowup (#38867).

This attaches the id select to the grid collection as an INNER JOIN on a SELECT DISTINCT derived table instead, so one paged query runs and no ids are materialized in PHP. The derived table is materialized once (it contains UNION ALL and DISTINCT, so the optimizer cannot merge it or rewrite it into a correlated subquery). Search\Collection::getBackendSearchEntityIdsSelect() is a new public method that reuses _getSearchEntityIdsSql() unchanged, so the attributes searched, the LIKE pattern, the store_id = 0 scope and the IFNULL(t2.value, t1.value) fallback are the same as addBackendSearchFilter(). The DISTINCT is what the UNION ALL requires so a product matching in sku and name is one grid row.

What this does not change: the LIKE '%q%' scan over the searchable EAV rows is still linear in catalog size. It removes the multiplier on top of it. A FULLTEXT index or routing the grid through the search engine would change the scaling law and is a separate decision.

Before, three statements plus model hydration:

SELECT e.* FROM catalog_product_entity AS e WHERE e.entity_id IN (<UNION>);            -- load()
SELECT e.entity_id FROM catalog_product_entity AS e WHERE e.entity_id IN (<UNION>);    -- getAllIds()
SELECT e.* FROM catalog_product_entity AS e WHERE e.entity_id IN (1,2,...,N) LIMIT 20; -- the grid

After, one:

SELECT e.* FROM catalog_product_entity AS e
INNER JOIN (SELECT DISTINCT search_result.entity_id FROM (<UNION>) AS search_result) AS search_result
  ON search_result.entity_id = e.entity_id
LIMIT 20;

SVC: getBackendSearchEntityIdsSelect() is an additive public method on an @api class (MINOR).

Fixed Issues (if relevant)

1. Fixes Catalog fulltext search performance #36408
2. Fixes Slow full text search on catalog products grid with 4 million products #38867

Manual testing scenarios (*)

1. Admin > Catalog > Products, type a keyword matching a sku, a name and a description into the grid search. The same products are returned as before, each once, and the record count matches.
2. Set the grid page size to 20 with more than 20 matches. The count stays at the full match total while paging.
3. Admin product form > Related Products > Add, search by keyword. Results are unchanged.
4. Enable the MySQL general log: the grid issues one paged query with the derived-table join and no IN (id, id, ...) list.

Contribution checklist (*)

• Pull request has a meaningful description of its purpose
• All commits are accompanied by meaningful commit messages
• All new or changed code is covered with unit/integration tests (if applicable)
• README.md files for modified modules are updated and included in the pull request if any README.md predefined sections require an update
• All automated tests passed successfully (all builds are green)

[m2-assistant]

Hi @lbajsarowicz. Thank you for your contribution!
Here are some useful tips on how you can test your changes using Magento test environment.
❗ Automated tests can be triggered manually with an appropriate comment:

• @magento run all tests - run or re-run all required tests against the PR changes
• @magento run <test-build(s)> - run or re-run specific test build(s)
  For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names.

Allowed build names are:

1. Database Compare
2. Functional Tests CE
3. Functional Tests EE
4. Functional Tests B2B
5. Integration Tests
6. Magento Health Index
7. Sample Data Tests CE
8. Sample Data Tests EE
9. Sample Data Tests B2B
10. Static Tests
11. Unit Tests
12. WebAPI Tests
13. Semantic Version Checker

You can find more information about the builds here
ℹ️ Run only required test builds during development. Run all test builds before sending your pull request for review.

---

For more details, review the Code Contributions documentation.
Join Magento Community Engineering Slack and ask your questions in #github channel.

[lbajsarowicz]

@magento run all tests