mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-08-03 16:39:00 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 8

#16152: fix tokenize to ignore whitespace at the end of the code when no newline is found. Patch by Ned Batchelder.

Browse source
This commit is contained in:
Ezio Melotti 2012-11-03 17:38:43 +02:00
parent 4552e3f95c
commit 2cc3b4ba9f
4 changed files with 12 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

5
Lib/test/test_tokenize.py
View file

@ -552,6 +552,11 @@ Evil tabs
DEDENT '' (4, 0) (4, 0)
DEDENT '' (4, 0) (4, 0)
Pathological whitespace (http://bugs.python.org/issue16152)
>>> dump_tokens("@ ")
ENCODING 'utf-8' (0, 0) (0, 0)
OP '@' (1, 0) (1, 1)
Non-ascii identifiers
>>> dump_tokens("Örter = 'places'\\ngrün = 'green'")