format doctests in docstrings (#8811)

## Summary

This PR adds opt-in support for formatting doctests in docstrings. This
reflects initial support and it is intended to add support for Markdown
and reStructuredText Python code blocks in the future. But I believe
this PR lays the groundwork, and future additions for Markdown and reST
should be less costly to add.

It's strongly recommended to review this PR commit-by-commit. The last
few commits in particular implement the bulk of the work here and
represent the denser portions.

Some things worth mentioning:

* The formatter is itself not perfect, and it is possible for it to
produce invalid Python code. Because of this, reformatted code snippets
are checked for Python validity. If they aren't valid, then we
(unfortunately silently) bail on formatting that code snippet.
* There are a couple places where it would be nice to at least warn the
user that doctest formatting failed, but it wasn't clear to me what the
best way to do that is.
* I haven't yet run this in anger on a real world code base. I think
that should happen before merging.

Closes #7146 

## Test Plan

* [x] Pass the local test suite.
* [x] Scrutinize ecosystem changes.
* [x] Run this formatter on extant code and scrutinize the results.
(e.g., CPython, numpy.)
This commit is contained in:
Andrew Gallant 2023-11-27 11:14:55 -05:00 committed by GitHub
parent 1f14d9a9f7
commit d9845a2628
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
24 changed files with 4200 additions and 97 deletions

View file

@ -43,6 +43,12 @@ pub struct PyFormatOptions {
/// in the formatted document.
source_map_generation: SourceMapGeneration,
/// Whether to format code snippets in docstrings or not.
///
/// By default this is disabled (opt-in), but the plan is to make this
/// enabled by default (opt-out) in the future.
docstring_code: DocstringCode,
/// Whether preview style formatting is enabled or not
preview: PreviewMode,
}
@ -70,6 +76,7 @@ impl Default for PyFormatOptions {
line_ending: LineEnding::default(),
magic_trailing_comma: MagicTrailingComma::default(),
source_map_generation: SourceMapGeneration::default(),
docstring_code: DocstringCode::default(),
preview: PreviewMode::default(),
}
}
@ -108,6 +115,10 @@ impl PyFormatOptions {
self.line_ending
}
pub fn docstring_code(&self) -> DocstringCode {
self.docstring_code
}
pub fn preview(&self) -> PreviewMode {
self.preview
}
@ -148,6 +159,12 @@ impl PyFormatOptions {
self
}
#[must_use]
pub fn with_docstring_code(mut self, docstring_code: DocstringCode) -> Self {
self.docstring_code = docstring_code;
self
}
#[must_use]
pub fn with_preview(mut self, preview: PreviewMode) -> Self {
self.preview = preview;
@ -284,3 +301,20 @@ impl PreviewMode {
matches!(self, PreviewMode::Enabled)
}
}
#[derive(Copy, Clone, Debug, Eq, PartialEq, Default, CacheKey)]
#[cfg_attr(feature = "serde", derive(serde::Serialize, serde::Deserialize))]
#[cfg_attr(feature = "serde", serde(rename_all = "lowercase"))]
#[cfg_attr(feature = "schemars", derive(schemars::JsonSchema))]
pub enum DocstringCode {
#[default]
Disabled,
Enabled,
}
impl DocstringCode {
pub const fn is_enabled(self) -> bool {
matches!(self, DocstringCode::Enabled)
}
}