When a function returns a queryset that is only annotated for a filter

[delfick]

I originally made this as a discussion but I get this feeling you don't get notifications for those. My apologies for the spam!

So in making my QuerySets defined as generic so that django-stubs starts to understand .annotate properly I have a problem where I don't want to always recognise that a queryset is annotated.

For example

def make_my_filtered_queryset(qs: MyQuerySet) -> MyQuerySet:
    return qs.annotate(some_count=Count(....)).filter(some_count==1)

After I make my queryset generic it starts to complain at me along the lines of

Incompatible return value type (got "MyQuerySet[MyModel@AnnotatedWith[TypedDict({'some_count': Any})], MyModel@AnnotatedWith[TypedDict({'some_count': Any})]]", expected "MyQuerySet[MyModel]")  [return-value]

I'd rather not add several hundred TypedDicts for annotations that aren't relevant out of the immediate .filter that uses them.

I'm wondering what y'all think about this situation and whether it's reasonable for mypy to say an annotated queryset is a superset of it's unannotated equivalent?

[sobolevn]

Indeed, I don't for some reason 😅

I'm wondering what y'all think about this situation and whether it's reasonable for mypy to say an annotated queryset is a superset of it's unannotated equivalent?

We can match this in theory for the return types. It is safe to return a more specific type that return annotation allows.

But, we can't match it in the parameter annotation.

So yeah, qs.annotate(some_count=Count(....)).filter(some_count==1) is a more specific type of qs

Any ideas / PRs on how to do that are welcome :)

[delfick]

@sobolevn

We can match this in theory for the return types. It is safe to return a more specific type that return annotation allows.

nice. Cause my question is about at runtime whether there is a difference in behaviour if the queryset is annotated or not. I'm not entirely sure.

Any ideas / PRs on how to do that are welcome :)

I suppose it's a question of what mypy plugin hook would be relevant. And unless I'm mistaken, that comparison happens within mypy and we don't have a hook to override the type of the object that is used to determine what is returned from the function right?

[delfick]

@sobolevn I think the reason mypy doesn't like it is because the type vars on a QuerySet appear both as inputs and outputs and so mypy doesn't recognise that querysets are read only. Though I made a version locally that removes that and it still doesn't like it, so maybe I'm wrong.

In the meantime I've added a feature to my plugin (across these two PRs, PR1, PR2) that lets me turn an annotated type into a non annotated type.

I'll use that so I can resolve the remaining 250 errors we get going from django-stubs 6.0.1 to 6.0.5 (and I'll see if there's any other oversights I've missed in this)

Is this something you'd be interested in including in django-stubs itself?

[sobolevn]

@delfick yes, sounds great to have :)
But, I would prefer not to have hide_queryset_annotations, but to have this feature built in :)
(if this is possible, of course)

[delfick]

@sobolevn what do you mean by "built in"?

[sobolevn]

Ideally qs.annotate(some_count=Count(....)).filter(some_count==1) should be compatible with MyQuerySet return type without any use code changes. Because MyQuerySet[MyModel@AnnotatedWith[TypedDict({'some_count': Any})], MyModel@AnnotatedWith[TypedDict({'some_count': Any})]] looks like a super type of MyQuerySet[MyModel]

[delfick]

yeah, for sure. I imagine for that to work we'd need to create a strict subset of QuerySet that actually is read only. I imagine that would require a bunch of other work so that methods on such a type propagate such a readonly type though. And probs would result in a more awkward situation.