Moves PEP 621 metadata from `setup.py` and `requirements*.txt` into the
`[project]` table of `pyproject.toml`. This enables using hatch as a
task runner for the project, where previously one would need to remember
a bunch of different commands, or repeatedly consult the readme's
developer guide to find all of the relevant commands.
This creates the following hatch commands:
- docs
- fixtures
- format
- lint
- test
- typecheck
It also updates all of the github actions workflows to use the
appropriate hatch commands, and the readme's developer guide, so that
there is only one source of truth for what constitutes running tests.
The "test" workflows now drop the matrix distinction between "pure" or
"native", and run tests in both modes from a single build.
ghstack-source-id: 8834da7825
Pull Request resolved: https://github.com/Instagram/LibCST/pull/893
* Raise black's output file's target version to 3.7, which is the lowest supported Python version that libcst can be run on
* Add to, instead of override, the exclusion rules of black
* Fix the bug that files in `stubs/libcst_native/` are inadvertently ignored by black
This is due to black's file exclusion mechanism is a file-system-unaware pure-string-based pattern match. We need to prepend "^/" to specify that we are referring to the root-level "native/" folder. Yeah, I know this looks strange, but blame black for it :) . See https://black.readthedocs.io/en/stable/usage_and_configuration/the_basics.html#configuration-format for further reference.
* It's conventional to use single-quote literal string to represent regular expression in TOML format, because in this way it doesn't perform any escaping
* When codemod, specify the black formatter to use the same target Python version we use
* Fix the `test_codemod_formatter_error_input` unit test
* Remove an unused import in `test_codemod_cli` module
This massive PR implements an alternative Python parser that will allow LibCST to parse Python 3.10's new grammar features. The parser is implemented in Rust, but it's turned off by default through the `LIBCST_PARSER_TYPE` environment variable. Set it to `native` to enable. The PR also enables new CI steps that test just the Rust parser, as well as steps that produce binary wheels for a variety of CPython versions and platforms.
Note: this PR aims to be roughly feature-equivalent to the main branch, so it doesn't include new 3.10 syntax features. That will be addressed as a follow-up PR.
The new parser is implemented in the `native/` directory, and is organized into two rust crates: `libcst_derive` contains some macros to facilitate various features of CST nodes, and `libcst` contains the `parser` itself (including the Python grammar), a `tokenizer` implementation by @bgw, and a very basic representation of CST `nodes`. Parsing is done by
1. **tokenizing** the input utf-8 string (bytes are not supported at the Rust layer, they are converted to utf-8 strings by the python wrapper)
2. running the **PEG parser** on the tokenized input, which also captures certain anchor tokens in the resulting syntax tree
3. using the anchor tokens to **inflate** the syntax tree into a proper CST
Co-authored-by: Benjamin Woodruff <github@benjam.info>
which ensures we won't have inconsistent black-vs-isort errors
going forward. We can always format by running `ufmt format .`
at the root, and check with `ufmt check .` in our CI actions.
* Read install requirements from requirements.txt
* read extras_require from requirements-dev.txt
* add requirements-dev.txt to MANIFEST.in
* apply fixes for new version of Black and Flake8
* don't upgrade Pyre
* re-format
- `pyproject.toml` is supported by isort and black, and lets us call
those tools without supplying a ton of arguments.
- `.editorconfig` is supported by a wide range of editors, and can
automatically set per-project configuration.
- `.pyre_configuration` is used by pyre.
- I added test discovery to the `setup.py` file, which required some
extra `__init__.py` files.
