mirrors/cpython
1
0
Fork 0
mirror of https://github.com/python/cpython.git synced 2025-07-29 14:15:07 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 9

Issue 21977: Minor improvements to the regexes in the tokenizer example.

Browse source
This commit is contained in:
Raymond Hettinger 2014-07-14 01:52:00 -07:00
parent 5283c4e108
commit 8323f68f3e
1 changed files with 2 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

4
Doc/library/re.rst
View file

@ -1340,9 +1340,9 @@ successive matches::
('ASSIGN', r':='), # Assignment operator
('END', r';'), # Statement terminator
('ID', r'[A-Za-z]+'), # Identifiers
('OP', r'[+*\/\-]'), # Arithmetic operators
('OP', r'[+\-*/]'), # Arithmetic operators
('NEWLINE', r'\n'), # Line endings
('SKIP', r'[ \t]'), # Skip over spaces and tabs
('SKIP', r'[ \t]+'), # Skip over spaces and tabs
]
tok_regex = '|'.join('(?P<%s>%s)' % pair for pair in token_specification)
get_token = re.compile(tok_regex).match