Commit graph

23 commits

Author SHA1 Message Date
Lysandros Nikolaou
cfcb952e30
[3.9] bpo-42218: Correctly handle errors in left-recursive rules (GH-23065) (GH-23066)
Left-recursive rules need to check for errors explicitly, since
even if the rule returns NULL, the parsing might continue and lead
to long-distance failures.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
(cherry picked from commit 02cdfc93f8)

Automerge-Triggered-By: GH:lysnikolaou
2020-10-31 12:06:03 -07:00
Lysandros Nikolaou
24a7c298d4
[3.9] bpo-42123: Run the parser two times and only enable invalid rules on the second run (GH-22111) (GH-23011)
* Implement running the parser a second time for the errors messages

The first parser run is only responsible for detecting whether
there is a `SyntaxError` or not. If there isn't the AST gets returned.
Otherwise, the parser is run a second time with all the `invalid_*`
rules enabled so that all the customized error messages get produced.

(cherry picked from commit bca7014032)
2020-10-28 02:14:15 +02:00
Pablo Galindo
54f115dd53
[3.9] bpo-41215: Don't use NULL by default in the PEG parser keyword list (GH-21355) (GH-21356)
(cherry picked from commit 39e76c0fb0)

Co-authored-by: Pablo Galindo <pablogsal@gmail.com>

Automerge-Triggered-By: @lysnikolaou
2020-07-06 12:29:59 -07:00
Pablo Galindo
145fa86598
[3.9] Include soft keywords in keyword.py (GH-20877). (GH-20880)
(cherry picked from commit 78319e373d)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-06-15 04:31:05 +01:00
Lysandros Nikolaou
1bfe659ee5
[3.9] Backport GH-20370 and GH-20436: Soft keywords (GH-20458) 2020-05-27 21:20:07 +01:00
Miss Islington (bot)
31084be618
bpo-40750: Do not expand the new parser debug flags if Py_BUILD_CORE is not defined (GH-20393)
(cherry picked from commit deb4355a37)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-25 12:37:56 -07:00
Miss Islington (bot)
82da2c3eb4
bpo-40750: Support -d flag in the new parser (GH-20340)
(cherry picked from commit 800a35c623)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-25 10:58:03 -07:00
Miss Islington (bot)
82c274e3ba
bpo-40334: Support suppressing of multiple optional variables in Pegen (GH-20367)
(cherry picked from commit cba5031510)

Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
2020-05-24 15:38:47 -07:00
Miss Islington (bot)
427c696400
Fix debug output in PEG parser generator (GH-20308)
(cherry picked from commit b831129123)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-21 19:05:20 -07:00
Pablo Galindo
7178b1bb68
[3.9] Fix typing problems reported by mypy in pegen (GH-20297) (GH-20300)
(cherry picked from commit d10fef35c6)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-21 22:09:43 +01:00
Lysandros Nikolaou
7b7a21bc4f
bpo-40661: Fix segfault when parsing invalid input (GH-20165)
Fix segfaults when parsing very complex invalid input, like `import äˆ ð£„¯ð¢·žð±‹á”€ð””ð‘©±å®ä±¬ð©¾\nð—¶½`.

Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2020-05-18 18:32:03 +01:00
Lysandros Nikolaou
2c8cd06afe
bpo-40334: Improvements to error-handling code in the PEG parser (GH-20003)
The following improvements are implemented in this commit:
- `p->error_indicator` is set, in case malloc or realloc fail.
- Avoid memory leaks in the case that realloc fails.
- Call `PyErr_NoMemory()` instead of `PyErr_Format()`, because it requires no memory.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2020-05-17 04:19:23 +01:00
Pablo Galindo
ac7a92cc0a
bpo-40334: Avoid collisions between parser variables and grammar variables (GH-19987)
This is for the C generator:
- Disallow rule and variable names starting with `_`
- Rename most local variable names generated by the parser to start with `_`

Exceptions:
- Renaming `p` to `_p` will be a separate PR
- There are still some names that might clash, e.g.
  - anything starting with `Py`
  - C reserved words (`if` etc.)
  - Macros like `EXTRA` and `CHECK`
2020-05-09 21:34:50 -07:00
Pablo Galindo
db9163ceef
bpo-40555: Check for p->error_indicator in loop rules after the main loop is done (GH-19986) 2020-05-08 03:38:44 +01:00
Pablo Galindo
470aac4d8e
bpo-40334: Generate comments in the parser code to improve debugging (GH-19966) 2020-05-06 23:14:43 +01:00
Anthony Shaw
c95e691c90
Clean up unused imports for the peg generator module (GH-19891) 2020-05-04 03:03:05 +01:00
Pablo Galindo
7ba08ff7b4
bpo-40334: use the TOKENS file when checking dangling rules (GH-19849) 2020-05-01 23:14:12 +01:00
Pablo Galindo
b796b3fb48
bpo-40334: Simplify type handling in the PEG c_generator (GH-19818) 2020-05-01 12:32:26 +01:00
Pablo Galindo
4db245ee9d
bpo-40334: refactor and cleanup for the PEG generators (GH-19775) 2020-04-29 10:42:21 +01:00
Pablo Galindo
5b9f4988c9
bpo-40334: Refactor peg_generator to receive a Tokens file when building c code (GH-19745) 2020-04-28 13:11:55 +01:00
Pablo Galindo
2b74c835a7
bpo-40334: Support CO_FUTURE_BARRY_AS_BDFL in the new parser (GH-19721)
This commit also allows to pass flags to the new parser in all interfaces and fixes a bug in the parser generator that was causing to inline rules with actions, making them disappear.
2020-04-27 18:02:07 +01:00
Pablo Galindo
1df5a9e88c
bpo-40334: Fix build errors and warnings in test_peg_generator (GH-19672) 2020-04-23 12:42:13 +01:00
Pablo Galindo
c5fc156852
bpo-40334: PEP 617 implementation: New PEG parser for CPython (GH-19503)
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2020-04-22 23:29:27 +01:00