cpython/Parser
Łukasz Langa 88f4ec88e2
[3.9] bpo-45494: Fix parser crash when reporting errors involving invalid continuation characters (GH-28993) (#29071)
There are two errors that this commit fixes:

* The parser was not correctly computing the offset and the string
  source for E_LINECONT errors due to the incorrect usage of strtok().
* The parser was not correctly unwinding the call stack when a tokenizer
  exception happened in rules involving optionals ('?', [...]) as we
  always make them return valid results by using the comma operator. We
  need to check first if we don't have an error before continuing..
(cherry picked from commit a106343f63)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>

NOTE: unlike the cherry-picked original, this commit points at a crazy location
due to a bug in the tokenizer that required a big refactor in 3.10 to fix.
We are leaving as-is for 3.9.
2021-10-20 18:51:13 +02:00
..
pegen [3.9] bpo-45494: Fix parser crash when reporting errors involving invalid continuation characters (GH-28993) (#29071) 2021-10-20 18:51:13 +02:00
pgen Fix typo in the parser generator (GH-18603) 2020-03-09 02:58:24 +00:00
acceler.c fix warnings by adding more const (GH-12924) 2019-04-23 20:39:37 +09:00
asdl.py bpo-40528: Improve and clear several aspects of the ASDL definition code for the AST (GH-19952) 2020-05-06 15:29:32 +01:00
asdl_c.py [3.9] bpo-11105: Do not crash when compiling recursive ASTs (GH-20594) (GH-26522) 2021-06-03 22:22:34 +01:00
grammar1.c bpo-39882: Add _Py_FatalErrorFormat() function (GH-19157) 2020-03-25 19:27:36 +01:00
listnode.c bpo-40268: Remove a few pycore_pystate.h includes (GH-19510) 2020-04-14 17:52:15 +02:00
myreadline.c bpo-38156: Fix compiler warning in PyOS_StdioReadline() (GH-21721) 2020-08-03 17:56:54 -07:00
node.c bpo-40502: Initialize n->n_col_offset (GH-19988) 2020-05-08 17:58:28 -03:00
parser.c bpo-39882: Py_FatalError() logs the function name (GH-18819) 2020-03-07 00:54:20 +01:00
parser.h fix warnings by adding more const (GH-12924) 2019-04-23 20:39:37 +09:00
parsetok.c bpo-40335: Correctly handle multi-line strings in tokenize error scenarios (GH-19619) 2020-04-21 01:53:04 +01:00
Python.asdl bpo-40528: Improve and clear several aspects of the ASDL definition code for the AST (GH-19952) 2020-05-06 15:29:32 +01:00
token.c bpo-35975: Support parsing earlier minor versions of Python 3 (GH-12086) 2019-03-07 12:38:08 -08:00
tokenizer.c [3.9] Update URLs in comments and metadata to use HTTPS (GH-27458) (GH-27480) 2021-07-30 16:34:04 +02:00
tokenizer.h closes bpo-39721: Fix constness of members of tok_state struct. (GH-18600) 2020-02-27 18:44:52 -08:00