mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-07-08 03:45:36 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 8

bpo-44667: Treat correctly lines ending with comments and no newlines in the Python tokenizer (GH-27499)

Browse source
This commit is contained in:
Pablo Galindo Salgado 2021-07-31 02:17:09 +01:00 • committed by GitHub
parent e63e6311aa
commit b6bde9fc42
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
3 changed files with 15 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
Lib/tokenize.py
View file

@ -604,7 +604,7 @@ def _tokenize(readline, encoding):
pos += 1
# Add an implicit NEWLINE if the input doesn't end in one
if last_line and last_line[-1] not in '\r\n':
if last_line and last_line[-1] not in '\r\n' and not last_line.strip().startswith("#"):
yield TokenInfo(NEWLINE, '', (lnum - 1, len(last_line)), (lnum - 1, len(last_line) + 1), '')
for indent in indents[1:]: # pop remaining indent levels
yield TokenInfo(DEDENT, '', (lnum, 0), (lnum, 0), '')