* Keep old exception messages (avoid breaking-changes for users relying on exception messages)
* Move ``get_expected_str`` out of _exceptions.py, where it does not belong, to its own file in _parser/_parsing_check.py
Summary:
This PR removes the `typing_extensions` and `typing_inspect` dependencies as we can now rely on the built-in `typing` module since Python 3.9.
Test Plan:
existing tests
which ensures we won't have inconsistent black-vs-isort errors
going forward. We can always format by running `ufmt format .`
at the root, and check with `ufmt check .` in our CI actions.
* Read install requirements from requirements.txt
* read extras_require from requirements-dev.txt
* add requirements-dev.txt to MANIFEST.in
* apply fixes for new version of Black and Flake8
* don't upgrade Pyre
* re-format
Standardize on the convention that private modules (those we don't expect people to directly import) are prefixed with an underscore. Everything under a directory/module that has an underscore is considered private, unless it is re-exported from a non-underscored module. Most things are exported from libcst directly, but there are a few things in libcst.tool, libcst.codegen and libcst.metadata that are namedspaced as such.
Fork both of these and convert internal LibCST stuff to point at them.
Removes the TODO from previous commit now that the tokenizer is in
complete agreement.
Because the parser conversion functions may not understand their current
position in the code, we instead need to construct a `ParserSyntaxError`
without the current line and column information, filling it in later.
Originally I did this by allowing ParserSyntaxError to be partially
initialized, but this wasn't very type-safe, and required some ugly
assertions.
Instead, I now construct and raise a PartialParserSyntaxError. The
BaseParser class is responsible for catching that partial error object
and constructing a full ParserSyntaxError.
- Make INDENT/DEDENT dummy tokens more readable instead of outputting an
empty string for them.
- Sort the "expected" list to make testing easier.
- Don't show the list of expected values if there are > 10
possibilities. The output becomes impossible to read at that point.
- Try to avoid using an empty line for the displayed contextual line of
source code. Instead, point at the line above it, or just don't output
any context.
This removes the hard-coded logic about encountered/expected, and moves
it into a separate helper method.
Line and column can now be initialized lazily. We'll use this later to
raise `ParserSyntaxError`s inside of conversion functions, backfilling
the positions inside `_base_parser`, since conversion functions don't
always have access to position information.
Go through and clean up where we export things from in the top level
directories. This is the first pass at cleaning up the public API. In
this diff I clean up all top-level exports that were coming out of .py
files and instead export them from libcst's base module itself.
I also didn't export a few of the things defined in newly underscored
files because they were only used internally.
