Test ty on the CI

[twoertwein]

If ty becomes as popular as ruff, it would be good to test against ty on the CI.

Since ty is still in an alpha stage, this should at most be an optional check for now.

[loicdiridollou]

It’s been on my radar for a while since it is likely to get a lot of traction as ruff did.
I tried it this morning and it was not able to complete on pandas-stubs folder and raised almost 3000 diagnostics on the tests folder.
For example with a data frame df, the following will match to Unknown: df.ewm().mean(). I don’t know if it is missing features with the project being so young. Let’s keep in on the radar but may be too early to put it on CI.
If agreed I will add it to our optional checks but likely not to be useful for a bit of time.

[loicdiridollou]

https://github.com/pandas-dev/pandas-stubs/actions/runs/15123998412/job/42512468714?pr=1224
This is what it would be on the tests so there is a bit of noise but not crazy.

[loicdiridollou]

https://github.com/pandas-dev/pandas-stubs/actions/runs/15124251567/job/42513218163?pr=1224
This is what it would be on the pandas-stubs folder so there is a ton of noise. Seems to me like it may be a parameter issue or just a lack of support, for example for Series[int], subscribing the type does not seem well supported yet.

[twoertwein]

Thank you for demonstrating the impact! Seems like we should wait quite a bit :)

[loicdiridollou]

FWIW I have also experimented with pyrefly since it was realeased a few days ago and same spirit (written in rust) but it is also a little too young to be inserted in our CI. It raises weird issues like:

ERROR /Users/loic.diridollou/Documents/Code/pandas-stubs/main/pandas-stubs/
core/series.pyi:1673:22-57: Expected a type form, got instance of `UnionType` [not-a-type]

for the following line: def __rpow__(self, other: num | _ListLike | Series[S1]) -> Series[S1]: ..., the issue being flagged is the | that makes it a Union.

So yes those are super interesting yet do not support pandas very well.

[Dr-Irv]

What might be worth doing for both those projects is to suggest that they test their projects on pandas-stubs. Feel free to do so.

Both mypy and pyright run their CI against pandas-stubs and it's a way for them to pick up any changes (good or bad) in behavior.

[loicdiridollou]

Opened two issues on ty and pyrefly, we will see which insights they can add, in the mean time I will add a step in the poe scripts to run it by hand like we do for stubtest.
facebook/pyrefly#427 (comment)
astral-sh/ty#580 (comment)

[yangdanny97]

@Dr-Irv we are running Pyrefly on both pandas and pandas-stubs as part of our CI

[Dr-Irv]

@Dr-Irv we are running Pyrefly on both pandas and pandas-stubs as part of our CI

I did a search for pandas in your repo and didn't see it. Can you point me to how you're doing that? It's important to do the tests on pandas-stubs/tests (as example source code) with pandas-stubs installed.

[yangdanny97]

@Dr-Irv ah, it's not open sourced right now because it logs the errors to an internal database. The way we set it up is essentially a fork of mypy primer w/ additional projects, that only runs on the current commit. We plan to eventually open source that part of the CI and move to using mypy primer directly, but we haven't gotten to it yet.

It's important to do the tests on pandas-stubs/tests (as example source code) with pandas-stubs installed.

I'll keep this in mind, thanks!

I think some of the noise you're seeing is that imports aren't being understood correctly out of the box. Based on my own local testing, import failures + bad specialization (because type vars weren't imported) account for around 75% of the errors I see.