mirrors/RustPython-Parser
1
0
Fork 0
mirror of https://github.com/RustPython/Parser.git synced 2025-07-13 16:15:16 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
225 commits 5 branches 3 tags 4.3 MiB
Commit graph

170 commits

Author SHA1 Message Date
Jeong YunWon
cb8c6fb78d Flatten rustpython_parser interface 2023-02-22 20:32:31 +09:00
Jeong YunWon
8580e4ebb5 make_tokenizer -> lex to integrate terms 
we don't distinguish scanner or tokenizer from lexer
 2023-02-22 20:28:15 +09:00
Jeong YunWon
66e3080173 Fix ModeParseError message 2023-02-22 20:28:03 +09:00
Jeong YunWon
1511b6631b Break down rustpython_parser::error module 
because it doesn't share any common errors but specific error for each sub module
 2023-02-22 20:28:01 +09:00
Charlie Marsh
2a8aa6f308 Always wrap in SoftKeywordTransformer 2023-02-21 19:18:42 -05:00
Charlie Marsh
dc628cab8f Expose SoftKeywordTransformer on public API 2023-02-21 19:00:32 -05:00
Jim Fasarakis-Hilliard
c137bc9d77 Merge pull request #4519 from charliermarsh/charlie/match 
Add support for match statements to parser
 2023-02-21 19:43:28 +02:00
Jeong YunWon
60180fd54c Merge pull request #4531 from charliermarsh/charlie/exception-groups 
Implement except* syntax
 2023-02-21 13:20:18 +09:00
Charlie Marsh
c7ed645cc6 Implement except* syntax 2023-02-21 12:19:54 +09:00
Charlie Marsh
8aa3bc93f3 Allow starred expressions in subscripts 2023-02-20 17:59:35 -05:00
Charlie Marsh
f39ffef370 Update compiler/parser/src/soft_keywords.rs 
Co-authored-by: Jim Fasarakis-Hilliard <d.f.hilliard@gmail.com>
 2023-02-20 15:03:39 -05:00
Jeong YunWon
e093d2bee6 clean up soft-keyword transform 2023-02-20 15:03:39 -05:00
Charlie Marsh
ca5b474d45 Use muiltipeek 2023-02-20 15:03:39 -05:00
Charlie Marsh
2b43d45bd5 Add support for match statements to parser 2023-02-20 15:03:39 -05:00
Jeong YunWon
09b82e41d5 Merge pull request #4490 from DimitrisJim/function_parser 
Add tests, some comments, to function.rs.
 2023-02-13 20:00:35 +09:00
Jeong YunWon
cc6d8a1c58 Merge pull request #4492 from DimitrisJim/doc_parser_uno 
Document parser crate.
 2023-02-13 17:25:14 +09:00
Dimitris Fasarakis Hilliard
07918f0a9a Document parser crate. 2023-02-12 17:58:19 +02:00
Dimitris Fasarakis Hilliard
a0786ea872 Add tests, some comments, to function.rs. 2023-02-11 23:07:57 +02:00
Dimitris Fasarakis Hilliard
4713b2b3ab Refactor: Join string and string_parser. 2023-02-11 18:05:06 +02:00
Jeong YunWon
e7f14ab9b8 Add test_generator_expression_argument 2023-02-11 05:20:39 +09:00
Charlie Marsh
56c73cc63d Use entire range for generators-as-arguments 2023-02-10 10:39:40 -05:00
Dimitris Fasarakis Hilliard
659f4dd8bf Document lexer. 2023-02-07 21:43:57 +02:00
Dimitris Fasarakis Hilliard
bd158089e0 Move NewLineHandler inline, don't check each character twice. 2023-02-07 20:58:53 +02:00
Jeong YunWon
a73bee7aae use workspace dependencies 2023-02-06 15:30:38 +09:00
Dimitris Fasarakis Hilliard
1468fe46ab Hint that the unwrap should always succeed. 2023-02-01 12:17:33 +02:00
Dimitris Fasarakis Hilliard
c9364718b4 Eat for comma. 2023-01-31 12:26:05 +02:00
Dimitris Fasarakis Hilliard
38cf933bcb Add initial capacities, use u32s for indents/spaces. 2023-01-31 12:26:05 +02:00
Dimitris Fasarakis Hilliard
838990ae15 Don't call is_emoji_presentation for each invocation of consume_normal 2023-01-31 12:26:05 +02:00
Dimitris Fasarakis Hilliard
aa0290bbfc Match on ascii start/continuation characters before calling functions. 2023-01-31 12:26:05 +02:00
Aarni Koskela
f74e44d1e8 Bump phf to 0.11 series 
string_cache is still using phf_shared 0.10.0 though
 2023-01-25 19:58:43 +02:00
Jeong YunWon
d9df131720 Merge pull request #4449 from harupy/fix-dict-spread-in-dict 
Fix AST generated from a dict literal containing dict unpacking
 2023-01-22 20:44:26 +09:00
Anders Kaseorg
6dba8430be Fix end location for elif blocks 
Since we parse an `elif:` block as an `If` node, its location should
include its `orelse` node like it would for an `if:` block.

Signed-off-by: Anders Kaseorg <andersk@mit.edu>
 2023-01-17 22:39:44 -05:00
harupy
b26365b215 Remove useless String::from 
Signed-off-by: harupy <hkawamura0130@gmail.com>
 2023-01-16 21:27:57 +09:00
harupy
2d019930e9 Rename test 2023-01-15 23:36:07 +09:00
harupy
d5fc7c4c87 Improve test 2023-01-15 16:53:13 +09:00
harupy
4edd2bf78a Remove commented-out code 2023-01-15 16:10:51 +09:00
harupy
581f6e176c Fix dict spreading in dict literal 2023-01-15 13:01:59 +09:00
Ran Benita
e5fe037e38 Fixup parse_tokens after "Add NonLogicalNewline token" 
I only updated `parse()` to ignore `NonLogicalNewline`, didn't notice
it's also needed in `parse_tokens()`.
 2023-01-14 11:52:33 +02:00
Ran Benita
674eeec29c Add NonLogicalNewline token 
This token is completely ignored by the parser, but it's useful for
other users of the lexer, such as the Ruff linter. For example, the
token is helpful for a "trailing comma" lint.

The same idea exists in Python's `tokenize` module - there is a NEWLINE
token (logical newline), and a NL token (non-logical newline).

Fixes #4385.
 2023-01-12 16:47:12 +02:00
Martin Fischer
4f1e7c6291 Fix docs.rs build for rustpython-parser 
docs.rs failed to build the documentation of the recently released
rustpython-parser 0.2.0 because the build.rs script couldn't write the
parser.rs file because docs.rs builds the documentation in a sandbox
with a read-only filesystem.

This commit fixes this by writing the parser.rs file to the cargo output
directory instead, as recommended by the docs.rs documentation.[1]

Fixes #4436.

[1]: https://docs.rs/about/builds#read-only-directories
 2023-01-11 09:58:10 +01:00
Noa
884a7bdb15 Bump all crate versions to 0.2.0 2023-01-11 00:14:28 -06:00
Jeong YunWon
7885344bcf first cspell dict 2023-01-09 19:57:23 +09:00
harupy
a4a5366504 Include comment text in token 2023-01-06 23:29:20 +09:00
harupy
2dfd053bed Implement Default for Location 2023-01-05 22:48:47 +09:00
harupy
7f552e4594 Address comments 2023-01-05 18:24:54 +09:00
harupy
9efa872023 Use try_from 2023-01-05 01:18:30 +09:00
harupy
fd8468c5eb Simplify string check 2023-01-04 23:57:50 +09:00
Jeong YunWon
958c7e33ad Merge pull request #4417 from harupy/add-with-offset-methods 
Add `with_col_offset` and `with_row_offset` to `Location` for conveniece
 2023-01-04 17:07:31 +09:00
Jeong YunWon
0f311cd5e5 Merge pull request #4413 from harupy/more-generic-window-impl 
Update `CharWindow` in `compiler/parser/src/lexer.rs` to allow slicing
 2023-01-04 17:06:06 +09:00
harupy
6d140426c1 Add with_col_offset and with_row_offset to Location 2023-01-04 13:16:03 +09:00