This PR tweaks uv to support reading `requirements.txt` regardless of
whether it is encoded as UTF-8 or UTF-16. This is particularly relevant
on Windows where `uv pip freeze > requirements.txt` will likely write a
UTF-16 encoded `requirements.txt` file.
There is some discussion on #1666 where it's suggested that perhaps
we should explicitly not support this. I didn't see that until I
had already put this PR together, but even so, I think it's worth
considering this. UTF-16 is predominant on Windows. It is very easy
to produce a UTF-16 encoded file. Moreover, there is an easy and well
specified way to recognize and transcode UTF-16 encoded data to UTF-8.
I think the downside of this is that it could encourage the use UTF-16
encoded `requirements.txt` files *in addition* to UTF-8 encoded
files, and it would probably be nice to converge and standardize on
one encoding. One possible alternative to this PR is that we provide
a better error message. Another alternative is to ensure that a
`-o/--output` flag exists for all commands (neither `uv pip freeze` nor
`pip freeze` have such a flag) so that users can always write output
to a file without relying on their environment's piping behavior.
(Although this last alternative seems a little sad to me.)
It's also worth noting the [PEP-0508] doesn't seem to mention file
encoding at all. So I think from a "do the standards allow this"
perspective, this change is OK.
Finally, `pip` itself seems to work with UTF-16 encoded
`requirements.txt` files.
I think I personally overall lean towards supporting UTF-16 for
`requirements.txt` files. In part because I think it smoothes out the
UX a little bit, in part because there is no obvious specification
(that I'm aware of) that mandates that these files are UTF-8, and
finally in part because `pip` supports it too.
Fixes#1666, Fixes#2276
[PEP-0508]: https://peps.python.org/pep-0508/
## Summary
This PR enables use of the Windows Store Pythons even with `py` is not
installed. Specifically, we need to ensure that the `python.exe` and
`python3.exe` executables installed into the
`C:\Users\crmar\AppData\Local\Microsoft\WindowsApp` directory _are_ used
when they're not "App execution aliases" (which merely open the Windows
Store, to help you install Python).
When `py` is installed, this isn't strictly necessary, since the
"resolved" executables are discovered via `py`. These look like
`C:\Users\crmar\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.11_qbs5n2kfra8p0\python.exe`.
Closes https://github.com/astral-sh/uv/issues/2264.
## Test Plan
- Removed all Python installations from my Windows machine.
- Uninstalled `py`.
- Enabled "App execution aliases".
- Verified that for both `cargo run venv --python python.exe` and `cargo
run venv --python python3.exe`, `uv` exited with a failure that no
Python could be found.
- Installed Python 3.10 via the Windows Store.
- Verified that the above commands succeeded without error.
- Verified that `cargo run venv --python python3.10.exe` _also_
succeeded.
Running the pep508_rs tests was failing due to uv-fs depending on
`fs_err::tokio` even when not selected. But the function that used it is
unused anyway, so i removed it.
## Summary
Allow using http(s) urls for constraints and requirements files handed
to the CLI, by handling paths starting with `http://` or `https://`
differently. This allows commands for such as: `uv pip install -c
https://raw.githubusercontent.com/apache/airflow/constraints-2.8.1/constraints-3.8.txt
requests`.
closes#1332
## Test Plan
Testing install using a `constraints.txt` file hosted on github in the
airflow repository:
fbdc2eba8e/crates/uv/tests/pip_install.rs (L1440-L1484)
## Advice Needed
- filesystem/http dispatch is implemented at a relatively low level (at
`crates/uv-fs/src/lib.rs#read_to_string`). Should I change some naming
here so it is obvious that the function is able to dispatch?
- I kept the CLI argument for -c and -r as a PathBuf, even though now it
is technically either a path or a url. We could either keep this as is
for now, or implement a new enum for this case? The enum could then
handle dispatch to files/http.
- Using another abstraction layer like
https://docs.rs/object_store/latest/object_store/ for the
files/urls/[s3] could work as well, though I ran into a bug during
testing which I couldn't debug
## Summary
After https://github.com/astral-sh/uv/pull/2121, the only remaining
issue is that calling `canonicalize` on these Pythons returns an error.
Closes https://github.com/astral-sh/uv/issues/2105.
## Test Plan
Uninstalled all python.org Pythons on my Windows machine, then created a
virtualenv. The resulting config file:
```
Using Python 3.11.8 interpreter at: C:\Users\crmar\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\python.exe
Creating virtualenv at: .venv
Activate with: .venv\Scripts\activate
PS C:\Users\crmar\workspace\puffin> cat .\.venv\pyvenv.cfg
home = C:\Users\crmar\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0
implementation = CPython
version_info = 3.11.8
include-system-site-packages = false
uv = 0.1.13
prompt = puffin
```
Prior to this PR, it would fail with a canonicalization error.
Prior to #2121, it would leave a "bad" Python in the config file:
```
Using Python 3.11.8 interpreter at: C:\Users\crmar\AppData\Local\Microsoft\WindowsApps\PythonSoftwareFoundation.Python.3.11_qbz5n2kfra8p0\python.exe
Creating virtualenv at: .venv
Activate with: .venv\Scripts\activate
PS C:\Users\crmar\workspace\puffin> cat .\.venv\pyvenv.cfg
home = C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.11_3.11.2288.0_x64__qbz5n2kfra8p0
implementation = CPython
version_info = 3.11.8
include-system-site-packages = false
uv = 0.1.13
prompt = puffin
```
Which, once activated, would fail with:
```
(venv) PS C:\Users\crmar\workspace\puffin> python
No Python at '"C:\Users\crmar\AppData\Local\Programs\Python\Python312\python.exe'
```
## Summary
We shouldn't be resolving symlinks on the provided interpreter;
otherwise we break `pyenv`, since running `cargo run pip install mypy
--python .venv/bin/python` will immediately resolve to (e.g.)
`/Users/crmarsh/.pyenv/versions/3.10.2/bin/python3.10`, and pyenv relies
on the path to do its lookups.
Instead, the canonicalizing happens when we query the interpreter
metadata.
Closes https://github.com/astral-sh/uv/issues/2068.
## Test Plan
Ran `cargo run pip install mypy --python .venv/bin/python -v -n` with a
virtualenv created using a pyenv Python; verified that Mypy was
installed into the virtual environment, rather than into the global
environment.
First, replace all usages in files in-place. I used my editor for this.
If someone wants to add a one-liner that'd be fun.
Then, update directory and file names:
```
# Run twice for nested directories
find . -type d -print0 | xargs -0 rename s/puffin/uv/g
find . -type d -print0 | xargs -0 rename s/puffin/uv/g
# Update files
find . -type f -print0 | xargs -0 rename s/puffin/uv/g
```
Then add all the files again
```
# Add all the files again
git add crates
git add python/uv
# This one needs a force-add
git add -f crates/uv-trampoline
```
