From 1899258a2287a40148fe1b425a6d309159f149d4 Mon Sep 17 00:00:00 2001 From: Irit Katriel Date: Tue, 26 Jan 2021 16:34:57 +0000 Subject: [PATCH 01/13] move towards a PEP structure (according the PEP-12 headings). Added intro sections --- except_star.md | 115 +++++++++++++++++++++++++++++++++++++------------ 1 file changed, 87 insertions(+), 28 deletions(-) diff --git a/except_star.md b/except_star.md index 6ab78fc..d56a733 100644 --- a/except_star.md +++ b/except_star.md @@ -1,31 +1,65 @@ # Introducing try..except* syntax -## Disclaimer -* We use the `ExceptionGroup` name, even though there - are other alternatives, e.g. `AggregateException` and `MultiError`. - Naming of the "exception group" object is out of scope of this proposal. +## Abstract + +This PEP proposes language extensions that allow programs to raise and handle +multiple unrelated exceptions simultaneously: + +* A new standard exception type, the `ExceptionGroup`, which represents a group +of unrelated exceptions being propagated together. + +* A new keyword `except*` for handling `ExceptionGroup`s. + +## Motivation + +The interpreter is currently able to propagate at most one exception at a time. +The chaining features introduced in PEP3134 [reference] link together exceptions +that are related to each other as the cause or context, but there are situations +where there are multiple unrelated exceptions that need to be propagated together +as the stack unwinds. Several real world use cases are listed below. + +[TODO: flesh these out] +* asyncio programs, trio, etc + +* Multiple errors from separate retries of an operation [https://bugs.python.org/issue29980] + +* Situations where multiple unrelated exceptiion may be of interest to calling code [https://bugs.python.org/issue40857] + +* Multiple teardowsn in pytest raising exceptions [https://github.com/pytest-dev/pytest/issues/8217] + +## Rationale + +Grouping several exceptions together can be done without changes to the language, simply by creating +a container exception type. Trio is an example of a library that has made use of this technique in +its `MultiError` type [reference to Trio MultiError]. However, such approaches require calling code +to catch the container exception type, and then inspect it to determine the types of errors that had +occurred, extract the ones it wants to handle and reraise the rest. + +Changes to the language are required in order to extend support for `ExceptionGroup`s in the style +of existing exception handling mechanisms. At the very least we would like to be able to catch an +`ExceptionGroup` only if it contains an exception type that we that chose to handle. Exceptions of +other types in the same `ExceptionGroup` need to be automatically reraised, otherwise it is too easy +for user code to inadvertently swallow exceptions that it is not handling. + +The purpose of this PEP, then, is to add the `except*` syntax for handling `ExceptionGroups`s in +the interpreter, which in turn requires that `ExceptionGroup` is added as a builtin type. The +semantics of handling `ExceptionGroup`s are not backwards compatible with the current exception +handling semantics, so could not modify the behaviour of the `except` keyword and instead added +the new `except*` syntax. + + +## Specification + * We use the term "naked" exception for regular Python exceptions **not wrapped** in an `ExceptionGroup`. E.g. a regular `ValueError` propagating through the stack is "naked". -* `ExceptionGroup` is an iterable object. - E.g. `list(ExceptionGroup(ValueError('a'), TypeError('b')))` is - equal to `[ValueError('a'), TypeError('b')]` - -* `ExceptionGroup` is not an indexable object; essentially - it's similar to Python `set`. The motivation for this is that exceptions - can occur in random order, and letting users write `group[0]` to access the - "first" error is error prone. Although the actual implementation of - `ExceptionGroup` will likely use an ordered list of errors to preserve - the actual occurrence order for rendering. - * `ExceptionGroup` is a subclass of `BaseException`, is assignable to `Exception.__context__`, and can be directly handled with `try: ... except ExceptionGroup: ...`. -* The behavior of the regular `try..except` statement will not be modified. ## Syntax @@ -706,6 +740,43 @@ in those tasks. Whereas handling `*CancelledError` makes sense -- it means that the current task is being canceled and this might be a good opportunity to do a cleanup. +## Backwards Compatibility + +## Security Implications + +## How to Teach This + +## Reference Implementation + +[An experimental implementation](https://github.com/iritkatriel/cpython/tree/exceptionGroup-stage4). + +(raise in except* not supported yet). + +## Rejected Ideas + +### The ExceptionGroup API + +We considered making `ExceptionGroup`s iterable, so that `list(eg)` would +produce a flattened list of the plain exceptions contained in the group. +We decided that this would not be not be a sound API, because the metadata +(cause, context and traceback) of the individual exceptions in a group are +incomplete and this could create problems. If use cases arise where this +can be helpful, we can document (or even provide in the standard library) +a sound recipe for accessing an individual exception: use the `project()` +method to create an `ExceptionGroup` for a single exception and then +transform it into a plain exception with the current metadata. + +### Traceback Representation + +We considered options for adapting the traceback data structure to represent +trees, but it became apparent that a traceback tree is not meaningful once separated +from the exceptions it refers to. While a simple-path traceback can be attached to +any exception by a `with_traceback()` call, it is hard to imagine a case where it +makes sense to assign a traceback tree to an exception group. Furthermore, a +useful display of the traceback includes information about the nested exceptions. +For this reason we decided it is best to leave the traceback mechanism as it is +and modify the traceback display code. + ### Adoption of try..except* syntax @@ -722,16 +793,6 @@ to start using the new `except *` syntax right away. They will have to use the new ExceptionGroup low-level APIs along with `try..except ExceptionGroup` to support running user code that can raise exception groups. -### Traceback Representation - -We considered options for adapting the traceback data structure to represent -trees, but it became apparent that a traceback tree is not meaningful once separated -from the exceptions it refers to. While a simple-path traceback can be attached to -any exception by a `with_traceback()` call, it is hard to imagine a case where it -makes sense to assign a traceback tree to an exception group. Furthermore, a -useful display of the traceback includes information about the nested exceptions. -For this reason we decided it is best to leave the traceback mechanism as it is -and modify the traceback display code. ## See Also @@ -739,8 +800,6 @@ and modify the traceback display code. programs: https://github.com/python/exceptiongroups/issues/3#issuecomment-716203284 -* A WIP implementation of the `ExceptionGroup` type by @iritkatriel - tracked [here](https://github.com/iritkatriel/cpython/tree/exceptionGroup-stage4). * The issue where this concept was first formalized: https://github.com/python/exceptiongroups/issues/4 From 1005e72ffcf8049e845cf82096cbffaadb6f2e06 Mon Sep 17 00:00:00 2001 From: Irit Katriel Date: Tue, 26 Jan 2021 18:09:51 +0000 Subject: [PATCH 02/13] added section with ExceptionGroup spec --- except_star.md | 137 +++++++++++++++++++++++++++++++++++++++---------- 1 file changed, 111 insertions(+), 26 deletions(-) diff --git a/except_star.md b/except_star.md index d56a733..0edd0e9 100644 --- a/except_star.md +++ b/except_star.md @@ -51,15 +51,122 @@ the new `except*` syntax. ## Specification +### ExceptionGroup + +The new builtin exception type, `ExceptionGroup` is a subclass of `BaseException`, +so it is assignable to `Exception.__cause__` and `Exception.__context__`, and can +be raised and handled as any exception with `raise ExceptionGroup(...)` and +`try: ... except ExceptionGroup: ...`. + +Its constructor takes two parameters: a message string and a sequence of the nested +exceptions, for example: +`ExceptionGroup('many problems', [ValueError('bad value'), TypeError('bad type')])`. + +The ExceptionGroup class exposes these parameters in the the fields `msg` and +`excs` (TODO: did we want to rename excs?). A nested exception can also be an +`ExceptionGroup` so the class represents a tree of exceptions (to avoid +cyclic graph we make the `excs` field immutable?). + +The `ExceptionGroup.split()` method gives us a way to extract a from an `ExceptionGroup` +a subset of the exceptions that satify a certain condition: + +```python +eg = ExceptionGroup("one", + [TypeError(1), + ExceptionGroup("two", + [TypeError(2), ValueError(3)]]) + +match, rest = eg.split(lambda e: isinstance(e, TypeError)) +``` + +`match` is an exception group with the same structure as `eg`, but with only the `TypeError` +exceptions: `ExceptionGroup('one', [TypeError(1), ExceptionGroup('two', [TypeError(2)])])`. +`rest` is the complement of match: `ExceptionGroup('one', [ExceptionGroup('two', [ValueError(3)])])`. + +Both `match` and `rest` are newly created exceptions, and the original `eg` is unchanged by the +`split`. The metadata (cause, context and traceback) is copied from the exception groups of `eg` +to those derived from the in `match` and `rest`. + +Since splitting by type is a very common use case, split also understands this as a shorthard: +`match, rest = eg.split(TypeError)`. + +`split` also has a parameter that tells it to not create the `rest` exception if only the +`match` is required: `eg.split(TypeError, with_complement=False)`. + + +#### The Traceback of and `ExceptionGroup` + +For regular exceptions, the traceback represents a simple path of frames, +from the frame in which the exception was raised to the frame in which it was +was caught or, if it hasn't been caught yet, the frame that the program's +execution is currently in. The list is constructed by the interpreter which, +appends any frame it exits to the traceback of the 'current exception' if one +exists (the exception returned by `sys.exc_info()`). To support efficient appends, +the links in a traceback's list of frames are from the oldest to the newest frame. +Appending a new frame is then simply a matter of inserting a new head to the +linked list referenced from the exception's `__traceback__` field. Crucially, +the traceback's frame list is immutable in the sense that frames only need to +be added at the head, and never need to be removed. + +We do not need to make any changes to this data structure. The `__traceback__` +field of the ExceptionGroup object represents the path that the exceptions travelled +through together after being joined into the `ExceptionGroup`, and the same field +on each of the nested exceptions represents that path through which each +exception arrived to the frame of the merge. + +What we do need to change is any code that interprets and displays tracebacks, +because it will now need to continue into tracebacks of nested exceptions +once the traceback of an ExceptionGroup has been processed. For example: + + +```python +def f(v): + try: + raise ValueError(v) + except ValueError as e: + return e + +try: + raise ExceptionGroup("one", [f(1)]) +except ExceptionGroup as e: + eg1 = e + +try: + raise ExceptionGroup("two", [f(2), eg1]) +except ExceptionGroup as e: + eg2 = e + +import traceback +traceback.print_exception(eg2) + +# Output: + +Traceback (most recent call last): + File "C:\src\cpython\tmp.py", line 13, in + raise ExceptionGroup("two", [f(2), eg1]) +ExceptionGroup: two + ------------------------------------------------------------ + Traceback (most recent call last): + File "C:\src\cpython\tmp.py", line 3, in f + raise ValueError(v) + ValueError: 2 + ------------------------------------------------------------ + Traceback (most recent call last): + File "C:\src\cpython\tmp.py", line 8, in + raise ExceptionGroup("one", [f(1)]) + ExceptionGroup: one + ------------------------------------------------------------ + Traceback (most recent call last): + File "C:\src\cpython\tmp.py", line 3, in f + raise ValueError(v) + ValueError: 1 +``` + * We use the term "naked" exception for regular Python exceptions **not wrapped** in an `ExceptionGroup`. E.g. a regular `ValueError` propagating through the stack is "naked". -* `ExceptionGroup` is a subclass of `BaseException`, - is assignable to `Exception.__context__`, and can be - directly handled with `try: ... except ExceptionGroup: ...`. - ## Syntax @@ -573,29 +680,7 @@ errors is error prone. We can consider allowing some of them in future versions of Python. -## The Traceback of an Exception Group -For regular exceptions, the traceback represents a simple path of frames, -from the frame in which the exception was raised to the frame in which it was -was caught or, if it hasn't been caught yet, the frame that the program's -execution is currently in. The list is constructed by the interpreter which, -appends any frame it exits to the traceback of the 'current exception' if one -exists (the exception returned by `sys.exc_info()`). To support efficient appends, -the links in a traceback's list of frames are from the oldest to the newest frame. -Appending a new frame is then simply a matter of inserting a new head to the -linked list referenced from the exception's `__traceback__` field. Crucially, -the traceback's frame list is immutable in the sense that frames only need to -be added at the head, and never need to be removed. - -We will not need to make any changes to this data structure. The -`__traceback__` field of the ExceptionGroup object represents that path that -the exceptions travelled through together after being joined into the -ExceptionGroup, and the same field on each of the nested exceptions represents -that path through which each exception arrived to the frame of the merge. - -What we do need to change is any code that interprets and displays tracebacks, -because it will now need to continue into tracebacks of nested exceptions -once the traceback of an ExceptionGroup has been processed. ## Design Considerations From 5f4d2247dca388a1ee07d66b19c115cc43703b88 Mon Sep 17 00:00:00 2001 From: Irit Katriel Date: Wed, 27 Jan 2021 20:20:38 +0000 Subject: [PATCH 03/13] Update except_star.md Co-authored-by: Guido van Rossum --- except_star.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/except_star.md b/except_star.md index 0edd0e9..5fc61e0 100644 --- a/except_star.md +++ b/except_star.md @@ -26,7 +26,7 @@ as the stack unwinds. Several real world use cases are listed below. * Situations where multiple unrelated exceptiion may be of interest to calling code [https://bugs.python.org/issue40857] -* Multiple teardowsn in pytest raising exceptions [https://github.com/pytest-dev/pytest/issues/8217] +* Multiple teardowns in pytest raising exceptions [https://github.com/pytest-dev/pytest/issues/8217] ## Rationale From 406392c98d78033864a7c49bbdce8f7956c5ad7d Mon Sep 17 00:00:00 2001 From: Irit Katriel Date: Wed, 27 Jan 2021 20:20:48 +0000 Subject: [PATCH 04/13] Update except_star.md Co-authored-by: Guido van Rossum --- except_star.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/except_star.md b/except_star.md index 5fc61e0..350ef29 100644 --- a/except_star.md +++ b/except_star.md @@ -45,7 +45,7 @@ for user code to inadvertently swallow exceptions that it is not handling. The purpose of this PEP, then, is to add the `except*` syntax for handling `ExceptionGroups`s in the interpreter, which in turn requires that `ExceptionGroup` is added as a builtin type. The semantics of handling `ExceptionGroup`s are not backwards compatible with the current exception -handling semantics, so could not modify the behaviour of the `except` keyword and instead added +handling semantics, so we could not modify the behaviour of the `except` keyword and instead added the new `except*` syntax. From cea7cbe0d2405560d3072b2c0caf00a61ce0aaef Mon Sep 17 00:00:00 2001 From: Irit Katriel Date: Wed, 27 Jan 2021 20:21:50 +0000 Subject: [PATCH 05/13] Update except_star.md Co-authored-by: Yury Selivanov --- except_star.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/except_star.md b/except_star.md index 350ef29..061f562 100644 --- a/except_star.md +++ b/except_star.md @@ -58,7 +58,7 @@ so it is assignable to `Exception.__cause__` and `Exception.__context__`, and ca be raised and handled as any exception with `raise ExceptionGroup(...)` and `try: ... except ExceptionGroup: ...`. -Its constructor takes two parameters: a message string and a sequence of the nested +Its constructor takes two positional-only parameters: a message string and a sequence of the nested exceptions, for example: `ExceptionGroup('many problems', [ValueError('bad value'), TypeError('bad type')])`. From 5098affd61c22cec38f987d7f38ad61f44516d47 Mon Sep 17 00:00:00 2001 From: Irit Katriel Date: Wed, 27 Jan 2021 20:22:10 +0000 Subject: [PATCH 06/13] Update except_star.md Co-authored-by: Guido van Rossum --- except_star.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/except_star.md b/except_star.md index 061f562..e7a574f 100644 --- a/except_star.md +++ b/except_star.md @@ -73,8 +73,8 @@ a subset of the exceptions that satify a certain condition: ```python eg = ExceptionGroup("one", [TypeError(1), - ExceptionGroup("two", - [TypeError(2), ValueError(3)]]) + ExceptionGroup("two", + [TypeError(2), ValueError(3)]]) match, rest = eg.split(lambda e: isinstance(e, TypeError)) ``` From 330017a0e80d1e8f18c88bd8ce0af35c67ccb624 Mon Sep 17 00:00:00 2001 From: Irit Katriel Date: Wed, 27 Jan 2021 20:22:22 +0000 Subject: [PATCH 07/13] Update except_star.md Co-authored-by: Guido van Rossum --- except_star.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/except_star.md b/except_star.md index e7a574f..6804b07 100644 --- a/except_star.md +++ b/except_star.md @@ -835,7 +835,7 @@ a cleanup. [An experimental implementation](https://github.com/iritkatriel/cpython/tree/exceptionGroup-stage4). -(raise in except* not supported yet). +(`raise` in `except*` not supported yet). ## Rejected Ideas From b26f318a916409d73f113c4dbbeb4e5482b7ceaa Mon Sep 17 00:00:00 2001 From: Irit Katriel Date: Wed, 27 Jan 2021 21:00:05 +0000 Subject: [PATCH 08/13] limit lines to 79 chars --- except_star.md | 176 ++++++++++++++++++++++++++----------------------- 1 file changed, 95 insertions(+), 81 deletions(-) diff --git a/except_star.md b/except_star.md index 6804b07..1d8aa09 100644 --- a/except_star.md +++ b/except_star.md @@ -6,92 +6,101 @@ This PEP proposes language extensions that allow programs to raise and handle multiple unrelated exceptions simultaneously: -* A new standard exception type, the `ExceptionGroup`, which represents a group -of unrelated exceptions being propagated together. +* A new standard exception type, the `ExceptionGroup`, which represents a +group of unrelated exceptions being propagated together. * A new keyword `except*` for handling `ExceptionGroup`s. ## Motivation -The interpreter is currently able to propagate at most one exception at a time. -The chaining features introduced in PEP3134 [reference] link together exceptions -that are related to each other as the cause or context, but there are situations -where there are multiple unrelated exceptions that need to be propagated together -as the stack unwinds. Several real world use cases are listed below. +The interpreter is currently able to propagate at most one exception at a +time. The chaining features introduced in PEP3134 [reference] link together +exceptions that are related to each other as the cause or context, but there +are situations where there are multiple unrelated exceptions that need to be +propagated together as the stack unwinds. Several real world use cases are +listed below. [TODO: flesh these out] * asyncio programs, trio, etc * Multiple errors from separate retries of an operation [https://bugs.python.org/issue29980] -* Situations where multiple unrelated exceptiion may be of interest to calling code [https://bugs.python.org/issue40857] +* Situations where multiple unrelated exceptions may be of interest to calling code [https://bugs.python.org/issue40857] * Multiple teardowns in pytest raising exceptions [https://github.com/pytest-dev/pytest/issues/8217] ## Rationale -Grouping several exceptions together can be done without changes to the language, simply by creating -a container exception type. Trio is an example of a library that has made use of this technique in -its `MultiError` type [reference to Trio MultiError]. However, such approaches require calling code -to catch the container exception type, and then inspect it to determine the types of errors that had -occurred, extract the ones it wants to handle and reraise the rest. - -Changes to the language are required in order to extend support for `ExceptionGroup`s in the style -of existing exception handling mechanisms. At the very least we would like to be able to catch an -`ExceptionGroup` only if it contains an exception type that we that chose to handle. Exceptions of -other types in the same `ExceptionGroup` need to be automatically reraised, otherwise it is too easy -for user code to inadvertently swallow exceptions that it is not handling. - -The purpose of this PEP, then, is to add the `except*` syntax for handling `ExceptionGroups`s in -the interpreter, which in turn requires that `ExceptionGroup` is added as a builtin type. The -semantics of handling `ExceptionGroup`s are not backwards compatible with the current exception -handling semantics, so we could not modify the behaviour of the `except` keyword and instead added -the new `except*` syntax. +Grouping several exceptions together can be done without changes to the +language, simply by creating a container exception type. Trio is an example of +a library that has made use of this technique in its `MultiError` type +[reference to Trio MultiError]. However, such approaches require calling code +to catch the container exception type, and then inspect it to determine the +types of errors that had occurred, extract the ones it wants to handle and +reraise the rest. + +Changes to the language are required in order to extend support for +`ExceptionGroup`s in the style of existing exception handling mechanisms. At +the very least we would like to be able to catch an `ExceptionGroup` only if +it contains an exception type that we that chose to handle. Exceptions of +other types in the same `ExceptionGroup` need to be automatically reraised, +otherwise it is too easy for user code to inadvertently swallow exceptions +that it is not handling. + +The purpose of this PEP, then, is to add the `except*` syntax for handling +`ExceptionGroups`s in the interpreter, which in turn requires that +`ExceptionGroup` is added as a builtin type. The semantics of handling +`ExceptionGroup`s are not backwards compatible with the current exception +handling semantics, so we could not modify the behaviour of the `except` +keyword and instead added the new `except*` syntax. ## Specification ### ExceptionGroup -The new builtin exception type, `ExceptionGroup` is a subclass of `BaseException`, -so it is assignable to `Exception.__cause__` and `Exception.__context__`, and can -be raised and handled as any exception with `raise ExceptionGroup(...)` and -`try: ... except ExceptionGroup: ...`. +The new builtin exception type, `ExceptionGroup` is a subclass of +`BaseException`, so it is assignable to `Exception.__cause__` and +`Exception.__context__`, and can be raised and handled as any exception +with `raise ExceptionGroup(...)` and `try: ... except ExceptionGroup: ...`. -Its constructor takes two positional-only parameters: a message string and a sequence of the nested -exceptions, for example: -`ExceptionGroup('many problems', [ValueError('bad value'), TypeError('bad type')])`. +Its constructor takes two positional-only parameters: a message string and a +sequence of the nested exceptions, for example: +`ExceptionGroup('issues', [ValueError('bad value'), TypeError('bad type')])`. The ExceptionGroup class exposes these parameters in the the fields `msg` and `excs` (TODO: did we want to rename excs?). A nested exception can also be an `ExceptionGroup` so the class represents a tree of exceptions (to avoid cyclic graph we make the `excs` field immutable?). -The `ExceptionGroup.split()` method gives us a way to extract a from an `ExceptionGroup` -a subset of the exceptions that satify a certain condition: +The `ExceptionGroup.split()` method gives us a way to extract a from an +`ExceptionGroup` a subset of the exceptions that satify a certain condition: ```python eg = ExceptionGroup("one", [TypeError(1), ExceptionGroup("two", - [TypeError(2), ValueError(3)]]) + [TypeError(2), ValueError(3)])]) match, rest = eg.split(lambda e: isinstance(e, TypeError)) ``` -`match` is an exception group with the same structure as `eg`, but with only the `TypeError` -exceptions: `ExceptionGroup('one', [TypeError(1), ExceptionGroup('two', [TypeError(2)])])`. -`rest` is the complement of match: `ExceptionGroup('one', [ExceptionGroup('two', [ValueError(3)])])`. +`match` is an exception group with the same structure as `eg`, but with only +the `TypeError` exceptions: +`ExceptionGroup('one', [TypeError(1), ExceptionGroup('two', [TypeError(2)])])`. +`rest` is the complement of match: +`ExceptionGroup('one', [ExceptionGroup('two', [ValueError(3)])])`. -Both `match` and `rest` are newly created exceptions, and the original `eg` is unchanged by the -`split`. The metadata (cause, context and traceback) is copied from the exception groups of `eg` -to those derived from the in `match` and `rest`. +Both `match` and `rest` are newly created exceptions, and the original `eg` is +unchanged by the `split`. The metadata (cause, context and traceback) is +copied from the exception groups of `eg` to those derived from the in `match` +and `rest`. -Since splitting by type is a very common use case, split also understands this as a shorthard: -`match, rest = eg.split(TypeError)`. +Since splitting by type is a very common use case, split also understands this +as a shorthard: `match, rest = eg.split(TypeError)`. -`split` also has a parameter that tells it to not create the `rest` exception if only the -`match` is required: `eg.split(TypeError, with_complement=False)`. +`split` also has a parameter that tells it to not create the `rest` exception +if only the `match` is required: `eg.split(TypeError, with_complement=False)`. #### The Traceback of and `ExceptionGroup` @@ -101,18 +110,18 @@ from the frame in which the exception was raised to the frame in which it was was caught or, if it hasn't been caught yet, the frame that the program's execution is currently in. The list is constructed by the interpreter which, appends any frame it exits to the traceback of the 'current exception' if one -exists (the exception returned by `sys.exc_info()`). To support efficient appends, -the links in a traceback's list of frames are from the oldest to the newest frame. -Appending a new frame is then simply a matter of inserting a new head to the -linked list referenced from the exception's `__traceback__` field. Crucially, -the traceback's frame list is immutable in the sense that frames only need to -be added at the head, and never need to be removed. +exists (the exception returned by `sys.exc_info()`). To support efficient +appends, the links in a traceback's list of frames are from the oldest to the +newest frame. Appending a new frame is then simply a matter of inserting a new +head to the linked list referenced from the exception's `__traceback__` field. +Crucially, the traceback's frame list is immutable in the sense that frames +only need to be added at the head, and never need to be removed. We do not need to make any changes to this data structure. The `__traceback__` -field of the ExceptionGroup object represents the path that the exceptions travelled -through together after being joined into the `ExceptionGroup`, and the same field -on each of the nested exceptions represents that path through which each -exception arrived to the frame of the merge. +field of the ExceptionGroup object represents the path that the exceptions +travelled through together after being joined into the `ExceptionGroup`, and +the same field on each of the nested exceptions represents that path through +which each exception arrived to the frame of the merge. What we do need to change is any code that interprets and displays tracebacks, because it will now need to continue into tracebacks of nested exceptions @@ -308,27 +317,29 @@ will be raised in the except* blocks: a "reraised" list for the naked raises and a "raised" list of the parameterised raises. * Every `except *` clause, run from top to bottom, can match a subset of the - exceptions out of the group forming a "working set" of errors for the current - clause. These exceptions are removed from the "incoming" group. If the except - block raises an exception, that exception is added to the appropriate result - list ("raised" or "reraised"), and in the case of "raise" it gets its - "working set" of errors linked to it via the `__context__` attribute. + exceptions out of the group forming a "working set" of errors for the + current clause. These exceptions are removed from the "incoming" group. + If the except block raises an exception, that exception is added to the + appropriate result list ("raised" or "reraised"), and in the case of "raise" + it gets its "working set" of errors linked to it via the `__context__` + attribute. * After there are no more `except*` clauses to evaluate, there are the following possibilities: -* Both the "incoming" `ExceptionGroup` and the two result lists are empty. This -means that all exceptions were processed and silenced. +* Both the "incoming" `ExceptionGroup` and the two result lists are empty. +This means that all exceptions were processed and silenced. * The "incoming" `ExceptionGroup` is non-empty but the result lists are: -not all exceptions were processed. The interpreter raises the "incoming" group. +not all exceptions were processed. The interpreter raises the "incoming" +group. * At least one of the result lists is non-empty: there are exceptions raised -from the except* clauses. The interpreter constructs a new `ExceptionGroup` with -an empty message and an exception list that contains all exceptions in "raised" -in addition to a single ExceptionGroup which holds the exceptions in "reraised" -and "incoming", in the same nested structure and with the same metadata as in -the original incoming exception. +from the except* clauses. The interpreter constructs a new `ExceptionGroup` +with an empty message and an exception list that contains all exceptions in +"raised" in addition to a single ExceptionGroup which holds the exceptions in +"reraised" and "incoming", in the same nested structure and with the same +metadata as in the original incoming exception. @@ -368,8 +379,8 @@ except *OSerror as errors: The above code ignores all `EPIPE` OS errors, while letting all other exceptions propagate. -Raising exceptions while handling an `ExceptionGroup` introduces nesting because -the traceback and chaining information needs to be maintained: +Raising exceptions while handling an `ExceptionGroup` introduces nesting +because the traceback and chaining information needs to be maintained: ```python try: @@ -393,8 +404,9 @@ except *ValueError: # ) ``` -A regular `raise Exception` would not wrap `Exception` in its own group, but a new group would still be -created to merged it with the ExceptionGroup of unhandled exceptions: +A regular `raise Exception` would not wrap `Exception` in its own group, but a +new group would still be created to merged it with the ExceptionGroup of +unhandled exceptions: ```python try: @@ -413,9 +425,10 @@ except *ValueError: ### Exception Chaining -If an error occurs during processing a set of exceptions in a `except *` block, -all matched errors would be put in a new `ExceptionGroup` which would be -referenced from the just occurred exception via its `__context__` attribute: +If an error occurs during processing a set of exceptions in a `except *` +block, all matched errors would be put in a new `ExceptionGroup` which would +be referenced from the just occurred exception via its `__context__` +attribute: ```python try: @@ -854,12 +867,13 @@ transform it into a plain exception with the current metadata. ### Traceback Representation We considered options for adapting the traceback data structure to represent -trees, but it became apparent that a traceback tree is not meaningful once separated -from the exceptions it refers to. While a simple-path traceback can be attached to -any exception by a `with_traceback()` call, it is hard to imagine a case where it -makes sense to assign a traceback tree to an exception group. Furthermore, a -useful display of the traceback includes information about the nested exceptions. -For this reason we decided it is best to leave the traceback mechanism as it is +trees, but it became apparent that a traceback tree is not meaningful once +separated from the exceptions it refers to. While a simple-path traceback can +be attached to any exception by a `with_traceback()` call, it is hard to +imagine a case where it makes sense to assign a traceback tree to an exception +group. Furthermore, a useful display of the traceback includes information +about the nested exceptions. For this reason we decided it is best to leave +the traceback mechanism as it is and modify the traceback display code. From 7aad6feeb0869af32b6468b517ece04e347d78c5 Mon Sep 17 00:00:00 2001 From: Irit Katriel Date: Wed, 27 Jan 2021 22:21:08 +0000 Subject: [PATCH 09/13] refined following reviews --- except_star.md | 70 +++++++++++++++++++++++++++++--------------------- 1 file changed, 41 insertions(+), 29 deletions(-) diff --git a/except_star.md b/except_star.md index 1d8aa09..4697642 100644 --- a/except_star.md +++ b/except_star.md @@ -9,7 +9,7 @@ multiple unrelated exceptions simultaneously: * A new standard exception type, the `ExceptionGroup`, which represents a group of unrelated exceptions being propagated together. -* A new keyword `except*` for handling `ExceptionGroup`s. +* A new syntax `except*` for handling `ExceptionGroup`s. ## Motivation @@ -68,13 +68,16 @@ Its constructor takes two positional-only parameters: a message string and a sequence of the nested exceptions, for example: `ExceptionGroup('issues', [ValueError('bad value'), TypeError('bad type')])`. -The ExceptionGroup class exposes these parameters in the the fields `msg` and -`excs` (TODO: did we want to rename excs?). A nested exception can also be an -`ExceptionGroup` so the class represents a tree of exceptions (to avoid -cyclic graph we make the `excs` field immutable?). +The `ExceptionGroup` class exposes these parameters in the fields `message` +and `errors`. A nested exception can also be an `ExceptionGroup` so the class +represents a tree of exceptions, where the leaves are plain exceptions and +each internal node represent a time at which the program grouped some +unrelated exceptions into a new `ExceptionGroup`. -The `ExceptionGroup.split()` method gives us a way to extract a from an -`ExceptionGroup` a subset of the exceptions that satify a certain condition: +The `ExceptionGroup.subgroup(condition)` method gives us a way to obtain an +`ExceptionGroup` that has the same metadata (cause, context, traceback) as +the original group, but contains only those exceptions for which the condition +is true: ```python eg = ExceptionGroup("one", @@ -82,25 +85,33 @@ eg = ExceptionGroup("one", ExceptionGroup("two", [TypeError(2), ValueError(3)])]) -match, rest = eg.split(lambda e: isinstance(e, TypeError)) +type_errors = eg.subgroup(lambda e: isinstance(e, TypeError)) ``` -`match` is an exception group with the same structure as `eg`, but with only -the `TypeError` exceptions: +The value of `type_errors` is: `ExceptionGroup('one', [TypeError(1), ExceptionGroup('two', [TypeError(2)])])`. -`rest` is the complement of match: -`ExceptionGroup('one', [ExceptionGroup('two', [ValueError(3)])])`. -Both `match` and `rest` are newly created exceptions, and the original `eg` is -unchanged by the `split`. The metadata (cause, context and traceback) is -copied from the exception groups of `eg` to those derived from the in `match` -and `rest`. +If both the subgroup and its complement are needed, the `ExceptionGroup.split` +method can be used: + +``` +type_errors, other_errors = eg.subgroup(lambda e: isinstance(e, TypeError)) +``` + +Now `type_errors` is the same as above, and `other_errors` is the complement: +`ExceptionGroup('one', [ExceptionGroup('two', [ValueError(3)])])`. -Since splitting by type is a very common use case, split also understands this -as a shorthard: `match, rest = eg.split(TypeError)`. +The original `eg` is unchanged by `subgroup` or `split`. If it, or any +nested `ExceptionGroup` is not included in the result in full, a new +`ExceptionGroup` is created, containing a subset of the exceptions in +its `errors` list. This partition is done recursively, so potentially +the entire `ExceptionTree` is copied. There is no need to copy the leaf +exceptions and the metadata elements (cause, context, traceback). -`split` also has a parameter that tells it to not create the `rest` exception -if only the `match` is required: `eg.split(TypeError, with_complement=False)`. +Since splitting by exception type is a very common use case, `subgroup` and +`split` also understand this as a shorthard: +`match, rest = eg.split(TypeError)`, so if the condition is an exception type, +it is checked with `isinstance` rather than being treated as a callable. #### The Traceback of and `ExceptionGroup` @@ -109,13 +120,14 @@ For regular exceptions, the traceback represents a simple path of frames, from the frame in which the exception was raised to the frame in which it was was caught or, if it hasn't been caught yet, the frame that the program's execution is currently in. The list is constructed by the interpreter which, -appends any frame it exits to the traceback of the 'current exception' if one -exists (the exception returned by `sys.exc_info()`). To support efficient -appends, the links in a traceback's list of frames are from the oldest to the -newest frame. Appending a new frame is then simply a matter of inserting a new -head to the linked list referenced from the exception's `__traceback__` field. -Crucially, the traceback's frame list is immutable in the sense that frames -only need to be added at the head, and never need to be removed. +appends any frame from which it exits to the traceback of the 'current +exception' if one exists (the exception returned by `sys.exc_info()`). To +support efficient appends, the links in a traceback's list of frames are from +the oldest to the newest frame. Appending a new frame is then simply a matter +of inserting a new head to the linked list referenced from the exception's +`__traceback__` field. Crucially, the traceback's frame list is immutable in +the sense that frames only need to be added at the head, and never need to be +removed. We do not need to make any changes to this data structure. The `__traceback__` field of the ExceptionGroup object represents the path that the exceptions @@ -172,7 +184,7 @@ ExceptionGroup: two ``` -* We use the term "naked" exception for regular Python exceptions + We use the term "naked" exception for regular Python exceptions **not wrapped** in an `ExceptionGroup`. E.g. a regular `ValueError` propagating through the stack is "naked". @@ -860,7 +872,7 @@ We decided that this would not be not be a sound API, because the metadata (cause, context and traceback) of the individual exceptions in a group are incomplete and this could create problems. If use cases arise where this can be helpful, we can document (or even provide in the standard library) -a sound recipe for accessing an individual exception: use the `project()` +a sound recipe for accessing an individual exception: use the `split()` method to create an `ExceptionGroup` for a single exception and then transform it into a plain exception with the current metadata. From 4c51a89380a12ff71a572ac67d9865b94cf05d69 Mon Sep 17 00:00:00 2001 From: Irit Katriel Date: Wed, 27 Jan 2021 22:34:23 +0000 Subject: [PATCH 10/13] added para on catch to 'rejected ideas' --- except_star.md | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/except_star.md b/except_star.md index 4697642..f78b4e7 100644 --- a/except_star.md +++ b/except_star.md @@ -885,11 +885,22 @@ be attached to any exception by a `with_traceback()` call, it is hard to imagine a case where it makes sense to assign a traceback tree to an exception group. Furthermore, a useful display of the traceback includes information about the nested exceptions. For this reason we decided it is best to leave -the traceback mechanism as it is -and modify the traceback display code. +the traceback mechanism as it is and modify the traceback display code. +### A full redesign of `except` -### Adoption of try..except* syntax +We considered introducing a new keyword (such as `catch`) which can be used +to handle both plain exceptions and `ExceptionGroup`s. Its semantics would +be the same as those of `except*` when catching an `ExceptionGroup`, but +it would not wrap plain a exception to create an `ExceptionGroup`. This +would have been part of a long term plan to replace `except` by `catch`, +but we decided that deprecating `except` in favour of an improved keyword +is too hard at this time, and it is more realistic to introduce the +`except*` syntax for `ExceptionGroup`s while `except` continues to be +used for simple exceptions. + + +## Adoption of try..except* syntax Application code typically can dictate what version of Python it requires. Which makes introducing TaskGroups and the new `except *` clause somewhat From d919674658843402b09a561fd820354ae17ec3ec Mon Sep 17 00:00:00 2001 From: Guido van Rossum Date: Wed, 27 Jan 2021 16:06:27 -0800 Subject: [PATCH 11/13] s/ which,/, which/ --- except_star.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/except_star.md b/except_star.md index f78b4e7..eaf812e 100644 --- a/except_star.md +++ b/except_star.md @@ -119,7 +119,7 @@ it is checked with `isinstance` rather than being treated as a callable. For regular exceptions, the traceback represents a simple path of frames, from the frame in which the exception was raised to the frame in which it was was caught or, if it hasn't been caught yet, the frame that the program's -execution is currently in. The list is constructed by the interpreter which, +execution is currently in. The list is constructed by the interpreter, which appends any frame from which it exits to the traceback of the 'current exception' if one exists (the exception returned by `sys.exc_info()`). To support efficient appends, the links in a traceback's list of frames are from From 776611c1fd69b6374c1ef2f82b06ebafad8a699e Mon Sep 17 00:00:00 2001 From: Irit Katriel Date: Thu, 28 Jan 2021 10:23:23 +0000 Subject: [PATCH 12/13] edited rejected ideas section following review --- except_star.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/except_star.md b/except_star.md index eaf812e..16c0fd4 100644 --- a/except_star.md +++ b/except_star.md @@ -894,10 +894,10 @@ to handle both plain exceptions and `ExceptionGroup`s. Its semantics would be the same as those of `except*` when catching an `ExceptionGroup`, but it would not wrap plain a exception to create an `ExceptionGroup`. This would have been part of a long term plan to replace `except` by `catch`, -but we decided that deprecating `except` in favour of an improved keyword -is too hard at this time, and it is more realistic to introduce the -`except*` syntax for `ExceptionGroup`s while `except` continues to be -used for simple exceptions. +but we decided that deprecating `except` in favour of an enhanced keyword +would be too confusing for users at this time, so it is more appropriate +to introduce the `except*` syntax for `ExceptionGroup`s while `except` +continues to be used for simple exceptions. ## Adoption of try..except* syntax From 9195cc4b94e8df02fd8832d94fe0ea288bede9b0 Mon Sep 17 00:00:00 2001 From: Irit Katriel Date: Thu, 28 Jan 2021 13:20:33 +0000 Subject: [PATCH 13/13] updates to the old text (ExceptionGroup constructor not vararg, EG not iterable, raise and reraise not giving the same thing) --- except_star.md | 284 +++++++++++++++++++++---------------------------- 1 file changed, 123 insertions(+), 161 deletions(-) diff --git a/except_star.md b/except_star.md index 16c0fd4..47bd0ea 100644 --- a/except_star.md +++ b/except_star.md @@ -184,13 +184,9 @@ ExceptionGroup: two ``` - We use the term "naked" exception for regular Python exceptions - **not wrapped** in an `ExceptionGroup`. E.g. a regular `ValueError` - propagating through the stack is "naked". +### except* -## Syntax - We're proposing to introduce a new variant of the `try..except` syntax to simplify working with exception groups: @@ -211,18 +207,20 @@ processed by one `except *` clause. ## Semantics -### Overview + In the following we use the term "naked" exception for regular Python + exceptions **not wrapped** in an `ExceptionGroup`. E.g. a regular + `ValueError` propagating through the stack is "naked". The `except *SpamError` block will be run if the `try` code raised an `ExceptionGroup` with one or more instances of `SpamError`. It would also be triggered if a naked instance of `SpamError` was raised. The `except *BazError as e` block would create an ExceptionGroup with the -same nested structure and metadata (msg, cause and context) as the one -raised, but containing only the instances of `BazError`. This ExceptionGroup -is assigned to `e`. The type of `e` would be `ExceptionGroup[BazError]`. -If there was just one naked instance of `BazError`, it would be wrapped -into an `ExceptionGroup` and assigned to `e`. +same nested structure and metadata (msg, cause, context and traceback) as the +one raised, but containing only the instances of `BazError`. This +`ExceptionGroup` is assigned to `e`. The type of `e` would be +`ExceptionGroup[BazError]`. If there was just one naked instance of `BazError`, +it would be wrapped into an `ExceptionGroup` and assigned to `e`. The `except *(BarError, FooError) as e` would split out all instances of `BarError` or `FooError` into such an ExceptionGroup and assign it to `e`. @@ -271,12 +269,12 @@ Exceptions are matched using a subclass check. For example: ```python try: low_level_os_operation() -except *OSerror as errors: - for e in errors: +except *OSerror as eg: + for e in eg.errors: print(type(e).__name__) ``` -would output: +could output: ``` BlockingIOError @@ -293,7 +291,7 @@ Example: ```python try: raise ExceptionGroup( - "msg", ValueError('a'), TypeError('b'), TypeError('c'), KeyError('e') + "msg", [ValueError('a'), TypeError('b'), TypeError('c'), KeyError('e')] ) except *ValueError as e: print(f'got some ValueErrors: {e}') @@ -305,8 +303,8 @@ except *TypeError as e: The above code would print: ``` -got some ValueErrors: ExceptionGroup("msg", ValueError('a')) -got some TypeErrors: ExceptionGroup("msg", TypeError('b'), TypeError('c')) +got some ValueErrors: ExceptionGroup("msg", [ValueError('a')]) +got some TypeErrors: ExceptionGroup("msg", TypeError[('b'), TypeError('c')]) ``` and then terminate with an unhandled `ExceptionGroup`: @@ -314,9 +312,9 @@ and then terminate with an unhandled `ExceptionGroup`: ``` ExceptionGroup( "msg", - TypeError('b'), - TypeError('c'), - KeyError('e'), + [TypeError('b'), + TypeError('c'), + KeyError('e')] ) ``` @@ -373,19 +371,15 @@ except *BlockingIOError: # ExceptionGroup(BlockingIOError()) ``` -### Raising ExceptionGroups manually +### Raising ExceptionGroups explicitly -Exception groups can be created and raised as follows: +Exception groups can be derived from other exception groups and raised as follows: ```python try: low_level_os_operation() except *OSerror as errors: - new_errors = [] - for e in errors: - if e.errno != errno.EPIPE: - new_errors.append(e) - raise ExceptionGroup(errors.msg, *new_errors) + raise errors.subgroup(lambda e: e.errno != errno.EPIPE) ``` The above code ignores all `EPIPE` OS errors, while letting all other @@ -396,9 +390,9 @@ because the traceback and chaining information needs to be maintained: ```python try: - raise ExceptionGroup("one", ValueError('a'), TypeError('b')) + raise ExceptionGroup("one", [ValueError('a'), TypeError('b')]) except *ValueError: - raise ExceptionGroup("two", KeyError('x'), KeyError('y')) + raise ExceptionGroup("two", [KeyError('x'), KeyError('y')]) # would result in: # @@ -406,12 +400,12 @@ except *ValueError: # "", # ExceptionGroup( <-- context = ExceptionGroup(ValueError('a')) # "two", -# KeyError('x'), -# KeyError('y'), +# [KeyError('x'), +# KeyError('y')], # ), # ExceptionGroup( <-- context, cause, tb same as the original "one" # "one", -# TypeError('b'), +# [TypeError('b')], # ) # ) ``` @@ -422,7 +416,7 @@ unhandled exceptions: ```python try: - raise ExceptionGroup("eg", ValueError('a'), TypeError('b')) + raise ExceptionGroup("eg", [ValueError('a'), TypeError('b')]) except *ValueError: raise KeyError('x') @@ -431,7 +425,7 @@ except *ValueError: # ExceptionGroup( # "", # KeyError('x'), -# ExceptionGroup("eg", TypeError('b')) +# ExceptionGroup("eg", [TypeError('b')]) # ) ``` @@ -444,7 +438,7 @@ attribute: ```python try: - raise ExceptionGroup("eg", ValueError('a'), ValueError('b'), TypeError('z')) + raise ExceptionGroup("eg", [ValueError('a'), ValueError('b'), TypeError('z')]) except *ValueError: 1 / 0 @@ -452,11 +446,11 @@ except *ValueError: # # ExceptionGroup( # "", -# ExceptionGroup( +# [ExceptionGroup( # "eg", -# TypeError('z'), -# ), -# ZeroDivisionError() +# [TypeError('z')], +# ), +# ZeroDivisionError()] # ) # # where the `ZeroDivisionError()` instance would have @@ -464,8 +458,7 @@ except *ValueError: # # ExceptionGroup( # "eg", -# ValueError('a'), -# ValueError('b') +# [ValueError('a'), ValueError('b')] # ) ``` @@ -480,11 +473,13 @@ except *ValueError as errors: # would result in: # # ExceptionGroup( -# ExceptionGroup( +# "", +# [ExceptionGroup( # "eg", -# TypeError('z'), -# ), -# RuntimeError('unexpected values') +# [TypeError('z')], +# ), +# RuntimeError('unexpected values') +# ] # ) # # where the `RuntimeError()` instance would have @@ -492,27 +487,23 @@ except *ValueError as errors: # # ExceptionGroup( # "eg", -# ValueError('a'), -# ValueError('b') +# [ValueError('a'), ValueError('b')] # ) ``` ### Recursive Matching The matching of `except *` clauses against an `ExceptionGroup` is performed -recursively. E.g.: +recursively, using the `ExceptionGroup.split()` method. E.g.: ```python try: raise ExceptionGroup( "eg", - ValueError('a'), - TypeError('b'), - ExceptionGroup( - "nested", - TypeError('c'), - KeyError('d') - ) + [ValueError('a'), + TypeError('b'), + ExceptionGroup("nested", [TypeError('c'), KeyError('d')]) + ] ) except *TypeError as e: print(f'e = {e}') @@ -521,32 +512,9 @@ except *Exception: # would print: # -# e = ExceptionGroup("eg", TypeError('b'), ExceptionGroup("nested", TypeError('c')) +# e = ExceptionGroup("eg", [TypeError('b'), ExceptionGroup("nested", [TypeError('c')])]) ``` -Iteration over an `ExceptionGroup` that has nested `ExceptionGroup` objects -in it effectively flattens the entire tree. E.g. - -```python -print( - list( - ExceptionGroup( - "eg", - ValueError('a'), - TypeError('b'), - ExceptionGroup( - "nested", - TypeError('c'), - KeyError('d') - ) - ) - ) -) - -# would output: -# -# [ValueError('a'), TypeError('b'), TypeError('c'), KeyError('d')] -``` ### Re-raising ExceptionGroups @@ -559,13 +527,10 @@ likely get lost: try: raise ExceptionGroup( "top", - ValueError('a'), - TypeError('b'), - ExceptionGroup( - "nested", - TypeError('c'), - KeyError('d') - ) + [ValueError('a'), + TypeError('b'), + ExceptionGroup("nested",[TypeError('c'), KeyError('d')]) + ] ) except *TypeError as e: e.foo = 'bar' @@ -573,43 +538,6 @@ except *TypeError as e: # destroyed after the `except*` clause. ``` -If the user wants to "flatten" the tree, they can explicitly create a new -`ExceptionGroup` and raise it: - - -```python -try: - raise ExceptionGroup( - "one", - ValueError('a'), - TypeError('b'), - ExceptionGroup( - "two", - TypeError('c'), - KeyError('d') - ) - ) -except *TypeError as e: - raise ExceptionGroup("three", *e) - -# would terminate with: -# -# ExceptionGroup( -# "three", -# ExceptionGroup( -# TypeError('b'), -# TypeError('c'), -# ), -# ExceptionGroup( -# "one", -# ValueError('a'), -# ExceptionGroup( -# "two", -# KeyError('d') -# ) -# ) -# ) -``` With the regular exceptions, there's a subtle difference between `raise e` and a bare `raise`: @@ -638,39 +566,64 @@ This difference is preserved with exception groups: * The `raise` form re-raises all exceptions from the group *without recording the current frame in their tracebacks*. -* The `raise e` form re-raises all exceptions from the group with tracebacks +* The `raise e` form re-raises the `ExceptionGroup` `e` with its traceback updated to point out to the current frame, effectively resulting in user seeing the `raise e` line in their tracebacks. -That said, both forms would not affect the overall shape of the exception -group: +After all `except *` blocks have been processed, the remaning unhandled +exceptions are merged together with the raised and re-reaised exceptions, +and the manner in which this is done depends on what the traceback needs +to contain: in the case of `raise e`, we have a new `ExceptionGroup` that +is merged with the unhandled `ExceptionGroup`, whereas in the case of +a naked `raise` we retain the re-reaised exceptions as if they were +unhandled: ```python -try: - raise ExceptionGroup( + +eg = raise ExceptionGroup( "one", - ValueError('a'), - TypeError('b'), - ExceptionGroup( - "two", - TypeError('c'), - KeyError('d') - ) + [ValueError('a'), + TypeError('b'), + ExceptionGroup("two", [TypeError('c'), KeyError('d')]) + ] ) + +try: + raise eg except *TypeError as e: - raise # or "raise e" + raise -# would both terminate with: +# would terminate with: # # ExceptionGroup( # "one", -# ValueError('a'), -# TypeError('b'), +# [ValueError('a'), +# TypeError('b'), +# ExceptionGroup("two", [TypeError('c'), KeyError('d')]) +# ] +# ) + +try: + raise eg: +except *TypeError as e: + raise e + +# would terminate with the following, where the re-raised exception +# (which has a different traceback now) is merged with the unhandled +# exceptions into a new `ExceptionGroup`: +# +# ExceptionGroup( +# "", +# [ExceptionGroup( +# "one", +# [ValueError('a'), +# ExceptionGroup("two", [KeyError('d')]), # ExceptionGroup( -# "two", -# TypeError('c'), -# KeyError('d') -# ) +# "one", +# [TypeError('b'), +# ExceptionGroup("two", [TypeError('c')]) +# ]), +# ] # ) ``` @@ -852,6 +805,24 @@ a cleanup. ## Backwards Compatibility +The behaviour of `except` is unchanged so existing code will continue to work. + +### Adoption of try..except* syntax + +Application code typically can dictate what version of Python it requires. +Which makes introducing TaskGroups and the new `except *` clause somewhat +straightforward. Upon switching to Python 3.10, the application developer +can grep their application code for every *control flow* exception they handle +(search for `except CancelledError`) and mechanically change it to +`except *CancelledError`. + +Library developers, on the other hand, will need to maintain backwards +compatibility with older Python versions, and therefore they wouldn't be able +to start using the new `except *` syntax right away. They will have to use +the new ExceptionGroup low-level APIs along with `try..except ExceptionGroup` +to support running user code that can raise exception groups. + + ## Security Implications ## How to Teach This @@ -900,28 +871,19 @@ to introduce the `except*` syntax for `ExceptionGroup`s while `except` continues to be used for simple exceptions. -## Adoption of try..except* syntax - -Application code typically can dictate what version of Python it requires. -Which makes introducing TaskGroups and the new `except *` clause somewhat -straightforward. Upon switching to Python 3.10, the application developer -can grep their application code for every *control flow* exception they handle -(search for `except CancelledError`) and mechanically change it to -`except *CancelledError`. - -Library developers, on the other hand, will need to maintain backwards -compatibility with older Python versions, and therefore they wouldn't be able -to start using the new `except *` syntax right away. They will have to use -the new ExceptionGroup low-level APIs along with `try..except ExceptionGroup` -to support running user code that can raise exception groups. - - ## See Also * An analysis of how exception groups will likely be used in asyncio programs: https://github.com/python/exceptiongroups/issues/3#issuecomment-716203284 - -* The issue where this concept was first formalized: + * The issue where this concept was first formalized: https://github.com/python/exceptiongroups/issues/4 + + +## References + +## Copyright + +This document is placed in the public domain or under the +CC0-1.0-Universal license, whichever is more permissive.