Hypothesis found that when we have a statement like `pass\r`, we detect that
`\r` is the default and parse the trailing newline as `Newline(None)`. However, when
we render the statement back out again, since we don't have a module, we construct
a default module which treats `Newline(None)` as a `\n` not a '\r'. So, when we are
parsing statements or expressions, disable auto-inferring the default newline and always
infer the default rendered newline (`\n`) so that rendering a statement/expression back
out behaves as expected.
Because the parser conversion functions may not understand their current
position in the code, we instead need to construct a `ParserSyntaxError`
without the current line and column information, filling it in later.
Originally I did this by allowing ParserSyntaxError to be partially
initialized, but this wasn't very type-safe, and required some ugly
assertions.
Instead, I now construct and raise a PartialParserSyntaxError. The
BaseParser class is responsible for catching that partial error object
and constructing a full ParserSyntaxError.
For anything that's not an internal logic error, conversion functions
should raise a ParserSyntaxError.
Internal logic errors should probably use an AssertionError or an assert
statement, but that's not as important and is out of scope for this PR.
This removes the hard-coded logic about encountered/expected, and moves
it into a separate helper method.
Line and column can now be initialized lazily. We'll use this later to
raise `ParserSyntaxError`s inside of conversion functions, backfilling
the positions inside `_base_parser`, since conversion functions don't
always have access to position information.
Adds details about how various configuration options can be used, and
updates some details to reflect reality (e.g. we don't currently infer
the python version from your execution environment).
We only support parsing code as 3.7 right now.
This raises a descriptive error message if we receive an unsupported
version number, instead of silently trying to use it with the tokenizer
and failing in potentially strange ways.
We pass down the correct default annotation indicator to use in 100% of code rendering places, so this becomes a useless bit of initialization. We already set this to a sentinel by default, and the only thing that having an explicit str gives us is the inability to copy an annotation from a param to a return or vice versa. So, out it goes. This means we can't render Annotation by itself, so the test that was using this behavior is out too.
This adds a __repr__ to various types in the parser config, so that the
generated documentation for functions using these types renders easier
to read.
This finishes the parser! 🎉
A few related changes were made:
- `DummyNode` was removed, as it's no longer needed.
- `lpar`/`rpar` was removed from `StarredDictElement`, because I found
out that it's not valid syntax while developing the parsing logic.
