Preferred idiom for exhaustiveness checking of unions

[bluetech]

• Are you reporting a bug, or opening a feature request?

A question...

• Please insert below the code you are checking with mypy.

I have some union, e.g.

from typing import Union

MyUnion = Union[int, str]

and I have a function which takes the union and matches on it:

def handle_my_union(x: MyUnion) -> None:
    if isinstance(x, int): ...
    elif isinstance(x, str): ...

Now suppose that the union gets added a new member:

- MyUnion = Union[int, str]
+ MyUnion = Union[int, str, bytes]

I have to remember to add it to all of the places which match on the union, for example handle_my_union. As written above, mypy doesn't warn (and has no reason to). So what I use is this function (stolen mostly verbatim from TypeScript):

from typing import NoReturn

def assert_never(x: NoReturn) -> NoReturn:
    assert False, "Unhandled type: {}".format(type(x).__name__)

and use it as follows:

 def handle_my_union(x: MyUnion) -> None:
     if isinstance(x, int): ...
     elif isinstance(x, str): ...
+    else:
+        assert_never(x)

• What is the actual behavior/output?

Now mypy warns:

error: Argument 1 to "assert_never" has incompatible type "bytes"; expected "NoReturn"

When I add a branch to handle bytes:

 def handle_my_union(x: MyUnion) -> None:
     if isinstance(x, int): ...
     elif isinstance(x, str): ...
+    elif isinstance(x, bytes): ...
     else:
         assert_never(x)

the error is gone.

• What is the behavior/output you expect?

I am happy with this solution, I just wonder if this is something that is considered appropriate to rely on, or if there's another preferable way to achieve exhaustiveness checking?

• What are the versions of mypy and Python you are using?

Python 3.5.5, mypy 0.630.

• What are the mypy flags you are using? (For example --strict-optional)

no_implicit_optional, check_untyped_defs, warn_unused_ignores.

[srittau]

Typescript also has a never type that is not compatible to anything, not even itself. I wonder whether it makes sense to have something like this in typing, which could be used in place of the argument type here.

[bluetech]

Isn't NoReturn equivalent to never? I'm just concluding by analogy here, specifically

• In TypeScript, as you say, the never is used for the "no return" case, e.g. node process.exit.

• In Rust, the "no return" case actually came first (written as e.g. pub fn exit(code: i32) -> !) and was then generalized to a never type.

[gvanrossum]

This is a clever hack, and I would never have come up with that (probably because the use case is not one I commonly encounter). It makes sense to legitimize some form of this and document it.

[Michael0x2a]

Typescript also has a never type that is not compatible to anything, not even itself.

Hmm, at least according to the docs, it seems like Typescript's never is compatible with itself. Nothing else is though, not even Any -- which is where Typescript's type system appears to diverge from ours. (We do seem to allow assigning values of type Any to variables of type NoReturn.)

This difference might actually make this pattern unsafe in a few cases. For example:

from typing import Union, Any

# where 'blah' is some 3rd party module with no type hints -- so 'Foo' has type 'Any'
from blah import Foo   

MyUnion = Union[int, str, Foo]

def assert_never(x: NoReturn) -> NoReturn:
    assert False

def handle_my_union(x: MyUnion) -> None:
    if isinstance(x, int):
        pass
    elif isinstance(x, str):
        pass
    else:
        assert_never(x)   # Uh-oh, no error here!

In this case, since blah has no type hints, Foo is inferred to be of type Any. And due to how Any interacts with NoReturn in our type system, the assert_never(...) doesn't end up warning us that our branches are non-exhaustive in this case.

Not sure how big of an issue this is though in practice -- I figured I ought to just bring it up since it seems like a useful caveat to mention if we do document this trick.

(Or I guess we could make NoReturn and Any interact in the same way they do in Typescript -- not sure how easy or hard that would be though.)

[emmatyping]

I would also point out this would be inconsistent with PEP 484 which states that NoReturn can only be used as a return type:

The NoReturn type is only valid as a return annotation of functions, and considered an error if it appears in other positions

Also, it might be nice to warn about using NoReturn in other places.

[bluetech]

(Or I guess we could make NoReturn and Any interact in the same way they do in Typescript -- not sure how easy or hard that would be though.)

FWIW, after applying the following change, to which I reached with some printf debugging, mypy's test suite still passes, and the NoReturn ← Any assignment fails with "error: Incompatible types in assignment (expression has type "Any", variable has type "NoReturn")", like TypeScript does.

diff --git a/mypy/subtypes.py b/mypy/subtypes.py
index d06ad40f..c4a62a08 100644
--- a/mypy/subtypes.py
+++ b/mypy/subtypes.py
@@ -162,7 +162,7 @@ class SubtypeVisitor(TypeVisitor[bool]):
         return True
 
     def visit_any(self, left: AnyType) -> bool:
-        return True
+        return not isinstance(self.right, UninhabitedType)
 
     def visit_none_type(self, left: NoneTyp) -> bool:
         if experiments.STRICT_OPTIONAL:

[bluetech]

I would also point out this would be inconsistent with PEP 484 which states that NoReturn can only be used as a return type:

The value of the argument of assert_never doesn't really matter as long as it doesn't match, I think. So assert_never can be defined like this instead:

from typing import NoReturn
import enum

class Impossible(enum.Enum):
    pass

def assert_never(x: Impossible) -> NoReturn:
    assert False, "Unhandled type: {}".format(type(x).__name__)

[gvanrossum]

In that case you don't even have to inherit from Enum. You can just refrain from creating instances. :-)

[ilevkivskyi]

The NoReturn type is only valid as a return annotation of functions, and considered an error if it appears in other positions

I would actually propose to remove this restriction from the PEP (if it is still OK), and make NoReturn a normal type. Proposed pattern is sufficient motivation for this, I could imagine it is useful to statically assert that a block is unreachable, especially that mypy already can do this.

Regarding the subtyping between Any and NoReturn we have a separate issue #3194.

[srittau]

Wouldn't it make sense to introduce an alias for NoReturn in that case? Annotating arguments with NoReturn is not an obvious pattern.

[ilevkivskyi]

The name is a bit unusual indeed, but typical alternatives Uninhabited, BottomType, etc. are not better IMO. Also after using it a bit, it actually kind of starts to make sense (at least for me).

[srittau]

Personally, I find Typescript's never type to be quite intuitive.