Should `asyncio.iscoroutinefunction` return some kind of `TypeGuard`?

[ajoino]

I have the following use-case of a class that takes a regular function or a coroutine as an attribute, and then provides access to it through two properties. Basically the code looks like this:

"""example.py"""

from asyncio import iscoroutinefunction
from typing import TypeVar, Generic, Callable, Awaitable, Any

T = TypeVar('T')

class Response(Generic[T]):
    ...

class Request(Generic[T]):
    ...

class Foo:
    def __init__(self, callback: Callable[[Request[T]], Response[T] | Awaitable[Response[T]]]):
        self._callback = callback

    @property
    def func(self) -> Callable[[Request[T]], Response[T]]:
        if iscoroutinefunction(self._callback):
            raise AttributeError()

        return self._callback

    @property
    def coro(self) -> Callable[[Request[T]], Awaitable[Response[T]]:
        if not iscoroutinefunction(self._callback):
            raise AttributeError()

        return self._callback

async def bar(req: Request[T]) -> Response[T]:
    ...

foo = Foo(bar)

Now, I expected mypy to give no errors due to type narrowing from `iscoroutinefunction, but instead I get

example.py:21: error: Incompatible return value type (got "Callable[[Request[T]], Union[Response[T], Awaitable[Response[T]]]]", expected "Callable[[Request[T]], Response[T]]")
example.py:28: error: Incompatible return value type (got "Callable[[Request[T]], Union[Response[T], Awaitable[Response[T]]]]", expected "Callable[[Request[T]], Awaitable[Response[T]]]")

After finding this, I started searching through the issues in both mypy and typeshed but couldn't find anything related to iscoroutinefunction. I then looked into typeshed and saw that the return type of iscoroutinefunction is just bool, and not a TypeGuard, despite the similar iscoroutine having one. This makes me think that iscoroutinefunction is annotated like that on purpose.

So my question is, should the return type of asyncio.iscoroutinefunction be a TypeGuard of some kind?

I'm new to the more advanced features of Python typing, and my naive implementation would be something like

from typing import TypeVar, TypeGuard

T = TypeVar('T')

def iscoroutinefunction(func: T) -> TypeGuard[T]: ...

If you feel like this should be changed, I'd be happy to submit a PR once I know what the type should be.

[AlexWaygood]

I've filed #8057 to improve the situation. It should fix the error emitted for the coro property, but it won't fix the error emitted for the func property. To do that, we'd need type narrowing in the negative case. Currently there's no mechanism to do that with TypeGuards, but @erictraut has implemented some experimental support for this kind of thing in pyright. For more information, see:

• TypeGuard type narrowing on returning False typing#996
• Change PEP 647 to allow for narrowing in the negative case typing#926

Here's how mypy handles your code snippet with my proposed #8057: https://mypy-play.net/?mypy=latest&python=3.10&gist=43dc2daa3e5708d97bfbac1709790590

[ajoino]

Thanks for making a PR!

I read the PR and linked issues but I still don't understand what makes the negative case difficult. I'm not well versed in typing and type theory so if you could explain it in simple terms it would be greatly appreciated.

[AlexWaygood]

I think Eric's explanation here is really good, and I'm not sure I can really do any better if I'm honest :) python/typing#926 (comment)

I'm not well versed in typing and type theory so if you could explain it in simple terms it would be greatly appreciated.

This is really complicated stuff! I wish I could explain it better :(

[ajoino]

So is the problem essentially that if a is a union of A and B, type narrowing in the positive case will guarantee that the type is A, but in the negative case it the union of A and B cannot be not excluded?

[AlexWaygood]

So is the problem essentially that if a is a union of A and B, type narrowing in the positive case will guarantee that the type is A, but in the negative case it the union of A and B cannot be not excluded?

Yes. In general, this can't be guaranteed to be type-safe, so type checkers decline to perform type-narrowing in the negative case for these situations.

[ajoino]

I guess a dirty hack to solve this would be to add a isnotcoroutine function to the inspect or asyncio modules, but I'm pretty sure we don't want that. # type: ignore works well enough I think. Thanks for the discussion!

[AlexWaygood]

I guess a dirty hack to solve this would be to add a isnotcoroutine function to the inspect or asyncio modules

This actually wouldn't do much to help the situation since, from the perspective of type checkers, coroutine functions are subtypes of common-or-garden functions!

from collections.abc import Callable, Coroutine
from typing import Any, TypeGuard

AnyFunction = Callable[..., Any]
AnyCoroutineFunction = Callable[..., Coroutine[Any, Any, Any]]  # A subtype of AnyFunction!

def iscoroutinefunction(obj: object) -> TypeGuard[AnyCoroutineFunction]:
    """Narrow the type to a coroutine function"""

def isnotcoroutinefunction(obj: object) -> TypeGuard[???]:
    """We can't narrow the type to <not a coroutine function>, since <coroutine function> is a subtype of <any old function>"""

[erictraut]

This is further evidence that there is need for StrictTypeGuard or something similar in the type system.