Fix encoding mismatch between python child process and uv (#7757)

## Summary

This PR fixes #7733. According to [CPython documentation on
`sys.stdout`](https://docs.python.org/3.12/library/sys.html#sys.stdout),
when `stdout`/`stderr` is non-character device like pipe, the encoding
will be set to system locale on windows. However, on the Rust side
`stdout_reader` and `stderr_reader` expect them to be encoded in UTF-8
and will fail when child process write non-ASCII character to
stdout/stderr, e.g., build directory name containing non-ASCII
character.

Both
[CPython3](https://docs.python.org/3.12/using/cmdline.html#envvar-PYTHONIOENCODING)
and [PyPy](https://doc.pypy.org/en/default/man/pypy3.1.html#environment)
support environment variable `PYTHONIOENCODING`. When it is set to
`utf-8`, python will use UTF-8 encoding for `stdin`/`stdout`/`stderr`.
Since `stdin` is not used by the spawned python process and we expect
`stdout`/`stderr` to use UTF-8, this fix should work as expected.
<!-- What's the purpose of the change? What does it do, and why? -->

## Test Plan

I only tested it on my computer with CPython 3.12 and 3.7. With the fix
applied I confirmed that [the case I
described](https://github.com/astral-sh/uv/issues/7733#issuecomment-2380416093)
is fixed.

I'm using Windows 11 with system locale set to code page 936.
This commit is contained in:
Jiahao Yuan 2024-09-28 20:39:37 +08:00 committed by GitHub
parent 313612dad7
commit 1cae78dd03
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -939,6 +939,7 @@ impl PythonRunner {
.env("PATH", modified_path)
.env("VIRTUAL_ENV", venv.root())
.env("CLICOLOR_FORCE", "1")
.env("PYTHONIOENCODING", "utf-8")
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()