gh-104169: Fix test_peg_generator after tokenizer refactoring (#110727)

* Fix test_peg_generator after tokenizer refactoring
* Remove references to tokenizer.c in comments etc.
This commit is contained in:
Lysandros Nikolaou 2023-10-12 09:34:35 +02:00 committed by GitHub
parent 23645420dc
commit 17d65547df
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
10 changed files with 24 additions and 14 deletions

View file

@ -255,7 +255,7 @@ class UTF8ValidatorTest(unittest.TestCase):
def test_invalid_utf8(self):
# This is a port of test_utf8_decode_invalid_sequences in
# test_unicode.py to exercise the separate utf8 validator in
# Parser/tokenizer.c used when reading source files.
# Parser/tokenizer/helpers.c used when reading source files.
# That file is written using low-level C file I/O, so the only way to
# test it is to write actual files to disk.