Commit graph

27 commits

Author SHA1 Message Date
227db6d77f adjust test cases 2024-10-16 12:21:49 -05:00
6ccdaef9b3 move this back 2024-10-16 12:20:51 -05:00
8c0d2a2b61 add a few more Django and HTML specific tokens 2024-10-16 12:19:45 -05:00
f00192a8b7
remove special case for tokenizing slashes (#22) 2024-10-16 11:30:25 -05:00
c2fb4255cd
add unit tests for lexer methods (#21) 2024-10-16 11:29:23 -05:00
8979493eac
adjust error handling in item at (#19) 2024-10-16 09:36:17 -05:00
0757f1aead
rename previous to peek previous (#18) 2024-10-16 09:34:01 -05:00
52eea54ed7
add item_at to Scanner trait and migrate to thiserror create (#17) 2024-10-15 05:49:46 -05:00
1a34ac124e
refactor method names for tokenizer trait and lexer (#16) 2024-10-14 16:15:43 -05:00
5c367ac17c
adjust scanner trait methods to return results (#15) 2024-10-14 15:47:26 -05:00
3b8ff4aba7
add error constructor methods to LexerError (#14) 2024-10-14 15:04:12 -05:00
3968104da3
borrow the string for the Token instead of allocating (#13) 2024-10-14 11:06:49 -05:00
a34ac87242
rename scan state for lexer to LexerState (#12) 2024-10-14 10:58:06 -05:00
ab09e6058b
explicitly pass char when handling whitespace (#11) 2024-10-14 06:43:34 -05:00
f59e21c958
Use borrowed &str instead of String for Lexer source (#10) 2024-10-13 20:19:23 -05:00
dce0745ce4
add stub compile function and basic integration tests (#9) 2024-10-13 18:26:49 -05:00
ab8daf12cb
move token related code to dedicated module (#8) 2024-10-13 18:14:14 -05:00
deebe6d2f8
move errors to dedicated file (#7) 2024-10-13 18:03:17 -05:00
aa7913b988
move scanner state to separate struct (#6) 2024-10-13 17:58:52 -05:00
03c0c19dd9
simplify lexer and tokenization by removing literal values (#5) 2024-10-13 17:14:15 -05:00
433c17999c rename match_char method for clarity 2024-10-13 15:14:25 -05:00
583b151c07
refactor lexer to support future parsing (#4) 2024-10-13 15:07:22 -05:00
784252e6c7
create Tokenizer trait and move Lexer::tokenize to trait implementation (#3) 2024-10-13 13:15:38 -05:00
51e00e7a5d
Remove unnecessary display from TokenType enum (#1) 2024-10-12 15:57:58 -05:00
b1bb62c086
Merge pull request #2 from joshuadavidthomas/joshuadavidthomas-patch-2
Add initial CI workflow
2024-10-12 15:39:51 -05:00
d0c4f1d02b
Add initial CI workflow 2024-10-12 15:31:30 -05:00
d69f67b74d initial commit 2024-10-12 02:20:59 -05:00