mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-08-04 17:08:35 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 13

bpo-25324: copy tok_name before changing it (#1608)

Browse source 
* add test to check if were modifying token

* copy list so import tokenize doesnt have side effects on token

* shorten line

* add tokenize tokens to token.h to get them to show up in token

* move ERRORTOKEN back to its previous location, and fix nitpick

* copy comments from token.h automatically

* fix whitespace and make more pythonic

* change to fix comments from @haypo

* update token.rst and Misc/NEWS

* change wording

* some more wording changes
This commit is contained in:
Albert-Jan Nijburg 2017-05-31 15:00:21 +01:00 • committed by Victor Stinner
parent 85aba238e4
commit fc354f0785
7 changed files with 52 additions and 21 deletions
Download patch file Download diff file Expand all files Collapse all files

3
Parser/tokenizer.c
View file

@ -106,6 +106,9 @@ const char *_PyParser_TokenNames[] = {
"AWAIT",
"ASYNC",
"<ERRORTOKEN>",
"COMMENT",
"NL",
"ENCODING"
"<N_TOKENS>"
};