mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-08-01 07:33:08 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 4

bpo-44201: Avoid side effects of "invalid_*" rules in the REPL (GH-26298) (GH-26313)

Browse source 
When the parser does a second pass to check for errors, these rules can
have some small side-effects as they may advance the parser more than
the point reached in the first pass. This can cause the tokenizer to ask
for extra tokens in interactive mode causing the tokenizer to show the
prompt instead of failing instantly.

To avoid this, add a new mode to the tokenizer that is activated in the
second pass and deactivates asking for new tokens when the interactive
line is finished. As the parsing should have reached the last line in
the first pass, the second pass should not need to ask for more tokens.

(cherry picked from commit bd7476dae3)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
This commit is contained in:
Miss Islington (bot) 2021-05-22 15:23:26 -07:00 • committed by GitHub
parent 413df57968
commit 1fb6b9e91d
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
5 changed files with 26 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

9
Parser/tokenizer.c
View file

@ -85,6 +85,7 @@ tok_new(void)
tok->async_def = 0;
tok->async_def_indent = 0;
tok->async_def_nl = 0;
tok->interactive_underflow = IUNDERFLOW_NORMAL;
return tok;
}
@ -845,6 +846,10 @@ tok_underflow_string(struct tok_state *tok) {
static int
tok_underflow_interactive(struct tok_state *tok) {
if (tok->interactive_underflow == IUNDERFLOW_STOP) {
tok->done = E_INTERACT_STOP;
return 1;
}
char *newtok = PyOS_Readline(stdin, stdout, tok->prompt);
if (newtok != NULL) {
char *translated = translate_newlines(newtok, 0, tok);
@ -1399,6 +1404,10 @@ tok_get(struct tok_state *tok, const char **p_start, const char **p_end)
}
}
if (tok->done == E_INTERACT_STOP) {
return ENDMARKER;
}
/* Check for EOF and errors now */
if (c == EOF) {
if (tok->level) {