mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-11-01 18:51:43 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 21

Issue2495: tokenize.untokenize did not insert space between two consecutive string literals:

Browse source 
"" "" => """", which is invalid code.

Will backport
This commit is contained in:
Amaury Forgeot d'Arc 2008-03-27 23:23:54 +00:00
parent 853e44ca8c
commit da0c025a43
3 changed files with 22 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

11
Lib/test/test_tokenize.py
View file

@ -487,13 +487,18 @@ Backslash means line continuation, except for comments
>>> roundtrip("# Comment \\\\nx = 0")
True
Two string literals on the same line
>>> roundtrip("'' ''")
True
Test roundtrip on random python modules.
pass the '-ucompiler' option to process the full directory.
>>>
>>> tempdir = os.path.dirname(f) or os.curdir
>>> testfiles = glob.glob(os.path.join(tempdir, "test*.py"))
XXX: tokenize doesn not support __future__.unicode_literals yet
>>> blacklist = ("test_future4.py",)
>>> testfiles = [f for f in testfiles if not f.endswith(blacklist)]
>>> if not test_support.is_resource_enabled("compiler"):
... testfiles = random.sample(testfiles, 10)
...