Avoid lexer infinite loop on invalid input (#6937)

## Summary

This PR fixes a bug which sends the lexer into infinite loop for an invalid input.
The code in question is `[1` where the nesting is never finished. This means
that the lexer will keep emitting the `Err` token forever.

## Test Plan

Add a test case which collects all the tokens from the lexer. This just
makes sure that it doesn't go into infinite loop.
This commit is contained in:
Dhruv Manilawala 2023-08-28 17:21:38 +05:30 committed by GitHub
parent 99f4c6886e
commit 9c98416b96
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23

View file

@ -706,6 +706,8 @@ impl<'source> Lexer<'source> {
// We reached end of file.
// First of all, we need all nestings to be finished.
if self.nesting > 0 {
// Reset the nesting to avoid going into infinite loop.
self.nesting = 0;
return Err(LexicalError {
error: LexicalErrorType::Eof,
location: self.offset(),
@ -1680,4 +1682,12 @@ def f(arg=%timeit a = b):
]
);
}
// This test case is to just make sure that the lexer doesn't go into
// infinite loop on invalid input.
#[test]
fn test_infite_loop() {
let source = "[1";
let _ = lex(source, Mode::Module).collect::<Vec<_>>();
}
}