Split uv-git and uv-git-types (#11448)
Some checks are pending
CI / Determine changes (push) Waiting to run
CI / smoke test | macos (push) Blocked by required conditions
CI / lint (push) Waiting to run
CI / cargo clippy | ubuntu (push) Blocked by required conditions
CI / cargo clippy | windows (push) Blocked by required conditions
CI / cargo dev generate-all (push) Blocked by required conditions
CI / cargo shear (push) Waiting to run
CI / check windows trampoline | x86_64 (push) Blocked by required conditions
CI / cargo test | ubuntu (push) Blocked by required conditions
CI / cargo test | macos (push) Blocked by required conditions
CI / cargo test | windows (push) Blocked by required conditions
CI / check windows trampoline | aarch64 (push) Blocked by required conditions
CI / check windows trampoline | i686 (push) Blocked by required conditions
CI / test windows trampoline | i686 (push) Blocked by required conditions
CI / test windows trampoline | x86_64 (push) Blocked by required conditions
CI / smoke test | windows x86_64 (push) Blocked by required conditions
CI / smoke test | windows aarch64 (push) Blocked by required conditions
CI / integration test | conda on ubuntu (push) Blocked by required conditions
CI / integration test | deadsnakes python3.9 on ubuntu (push) Blocked by required conditions
CI / integration test | free-threaded on linux (push) Blocked by required conditions
CI / integration test | free-threaded on windows (push) Blocked by required conditions
CI / integration test | pypy on ubuntu (push) Blocked by required conditions
CI / integration test | pypy on windows (push) Blocked by required conditions
CI / integration test | graalpy on ubuntu (push) Blocked by required conditions
CI / integration test | graalpy on windows (push) Blocked by required conditions
CI / integration test | github actions (push) Blocked by required conditions
CI / integration test | determine publish changes (push) Blocked by required conditions
CI / integration test | uv publish (push) Blocked by required conditions
CI / check cache | ubuntu (push) Blocked by required conditions
CI / check cache | macos aarch64 (push) Blocked by required conditions
CI / check system | python on debian (push) Blocked by required conditions
CI / check system | python on fedora (push) Blocked by required conditions
CI / typos (push) Waiting to run
CI / mkdocs (push) Waiting to run
CI / build binary | linux libc (push) Blocked by required conditions
CI / build binary | linux musl (push) Blocked by required conditions
CI / build binary | macos aarch64 (push) Blocked by required conditions
CI / build binary | macos x86_64 (push) Blocked by required conditions
CI / build binary | windows x86_64 (push) Blocked by required conditions
CI / build binary | windows aarch64 (push) Blocked by required conditions
CI / cargo build (msrv) (push) Blocked by required conditions
CI / build binary | freebsd (push) Blocked by required conditions
CI / ecosystem test | pydantic/pydantic-core (push) Blocked by required conditions
CI / ecosystem test | prefecthq/prefect (push) Blocked by required conditions
CI / ecosystem test | pallets/flask (push) Blocked by required conditions
CI / smoke test | linux (push) Blocked by required conditions
CI / check system | python on ubuntu (push) Blocked by required conditions
CI / check system | python on opensuse (push) Blocked by required conditions
CI / check system | python on rocky linux 8 (push) Blocked by required conditions
CI / check system | python on rocky linux 9 (push) Blocked by required conditions
CI / check system | pypy on ubuntu (push) Blocked by required conditions
CI / check system | pyston (push) Blocked by required conditions
CI / check system | alpine (push) Blocked by required conditions
CI / check system | python on macos aarch64 (push) Blocked by required conditions
CI / check system | homebrew python on macos aarch64 (push) Blocked by required conditions
CI / check system | python on macos x86-64 (push) Blocked by required conditions
CI / check system | python3.10 on windows x86-64 (push) Blocked by required conditions
CI / check system | python3.10 on windows x86 (push) Blocked by required conditions
CI / check system | python3.13 on windows x86-64 (push) Blocked by required conditions
CI / check system | windows registry (push) Blocked by required conditions
CI / check system | python3.12 via chocolatey (push) Blocked by required conditions
CI / check system | python3.9 via pyenv (push) Blocked by required conditions
CI / check system | python3.13 (push) Blocked by required conditions
CI / check system | conda3.11 on macos aarch64 (push) Blocked by required conditions
CI / check system | conda3.8 on macos aarch64 (push) Blocked by required conditions
CI / check system | conda3.11 on linux x86-64 (push) Blocked by required conditions
CI / check system | conda3.8 on linux x86-64 (push) Blocked by required conditions
CI / check system | conda3.11 on windows x86-64 (push) Blocked by required conditions
CI / check system | conda3.8 on windows x86-64 (push) Blocked by required conditions
CI / check system | amazonlinux (push) Blocked by required conditions
CI / check system | embedded python3.10 on windows x86-64 (push) Blocked by required conditions
CI / benchmarks (push) Blocked by required conditions

We want to build `uv-build` without depending on the network crates. In
preparation for that, we split uv-git into uv-git and uv-git-types,
where only uv-git depends on reqwest, so that uv-build can use
uv-git-types.
This commit is contained in:
konsti 2025-02-17 10:37:55 +01:00 committed by GitHub
parent e95da5c3af
commit 248da23f6d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
32 changed files with 294 additions and 245 deletions

View file

@ -16,10 +16,11 @@ doctest = false
workspace = true
[dependencies]
uv-auth = { workspace = true }
uv-cache-key = { workspace = true }
uv-fs = { workspace = true, features = ["tokio"] }
uv-auth = { workspace = true }
uv-static = { workspace = true}
uv-git-types = { workspace = true }
uv-static = { workspace = true }
uv-version = { workspace = true }
anyhow = { workspace = true }
@ -28,7 +29,6 @@ dashmap = { workspace = true }
fs-err = { workspace = true, features = ["tokio"] }
reqwest = { workspace = true, features = ["blocking"] }
reqwest-middleware = { workspace = true }
serde = { workspace = true }
thiserror = { workspace = true }
tokio = { workspace = true }
tracing = { workspace = true }

View file

@ -15,11 +15,10 @@ use tracing::{debug, warn};
use url::Url;
use uv_fs::Simplified;
use uv_git_types::{GitHubRepository, GitOid, GitReference};
use uv_static::EnvVars;
use uv_version::version;
use crate::{GitHubRepository, GitOid};
/// A file indicates that if present, `git reset` has been done and a repo
/// checkout is ready to go. See [`GitCheckout::reset`] for why we need this.
const CHECKOUT_READY_LOCK: &str = ".ok";
@ -48,79 +47,6 @@ enum RefspecStrategy {
First,
}
/// A reference to commit or commit-ish.
#[derive(Debug, Clone, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub enum GitReference {
/// A specific branch.
Branch(String),
/// A specific tag.
Tag(String),
/// From a reference that's ambiguously a branch or tag.
BranchOrTag(String),
/// From a reference that's ambiguously a commit, branch, or tag.
BranchOrTagOrCommit(String),
/// From a named reference, like `refs/pull/493/head`.
NamedRef(String),
/// The default branch of the repository, the reference named `HEAD`.
DefaultBranch,
}
impl GitReference {
/// Creates a [`GitReference`] from an arbitrary revision string, which could represent a
/// branch, tag, commit, or named ref.
pub fn from_rev(rev: String) -> Self {
if rev.starts_with("refs/") {
Self::NamedRef(rev)
} else if looks_like_commit_hash(&rev) {
Self::BranchOrTagOrCommit(rev)
} else {
Self::BranchOrTag(rev)
}
}
/// Converts the [`GitReference`] to a `str`.
pub fn as_str(&self) -> Option<&str> {
match self {
Self::Tag(rev) => Some(rev),
Self::Branch(rev) => Some(rev),
Self::BranchOrTag(rev) => Some(rev),
Self::BranchOrTagOrCommit(rev) => Some(rev),
Self::NamedRef(rev) => Some(rev),
Self::DefaultBranch => None,
}
}
/// Converts the [`GitReference`] to a `str` that can be used as a revision.
pub fn as_rev(&self) -> &str {
match self {
Self::Tag(rev) => rev,
Self::Branch(rev) => rev,
Self::BranchOrTag(rev) => rev,
Self::BranchOrTagOrCommit(rev) => rev,
Self::NamedRef(rev) => rev,
Self::DefaultBranch => "HEAD",
}
}
/// Returns the kind of this reference.
pub(crate) fn kind_str(&self) -> &str {
match self {
Self::Branch(_) => "branch",
Self::Tag(_) => "tag",
Self::BranchOrTag(_) => "branch or tag",
Self::BranchOrTagOrCommit(_) => "branch, tag, or commit",
Self::NamedRef(_) => "ref",
Self::DefaultBranch => "default branch",
}
}
}
impl Display for GitReference {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.as_str().unwrap_or("HEAD"))
}
}
/// A Git reference (like a tag or branch) or a specific commit.
#[derive(Debug, Copy, Clone, PartialEq, Eq, Hash, PartialOrd, Ord)]
enum ReferenceOrOid<'reference> {
@ -854,11 +780,6 @@ fn github_fast_path(
})
}
/// Whether a `rev` looks like a commit hash (ASCII hex digits).
fn looks_like_commit_hash(rev: &str) -> bool {
rev.len() >= 7 && rev.chars().all(|ch| ch.is_ascii_hexdigit())
}
/// Whether `rev` is a shorter hash of `oid`.
fn is_short_hash_of(rev: &str, oid: GitOid) -> bool {
let long_hash = oid.to_string();

View file

@ -1,85 +0,0 @@
use tracing::debug;
use url::Url;
/// A reference to a repository on GitHub.
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub struct GitHubRepository<'a> {
/// The `owner` field for the repository, i.e., the user or organization that owns the
/// repository, like `astral-sh`.
pub owner: &'a str,
/// The `repo` field for the repository, i.e., the name of the repository, like `uv`.
pub repo: &'a str,
}
impl<'a> GitHubRepository<'a> {
/// Parse a GitHub repository from a URL.
///
/// Expects to receive a URL of the form: `https://github.com/{user}/{repo}[.git]`, e.g.,
/// `https://github.com/astral-sh/uv`. Otherwise, returns `None`.
pub fn parse(url: &'a Url) -> Option<Self> {
// The fast path is only available for GitHub repositories.
if url.host_str() != Some("github.com") {
return None;
};
// The GitHub URL must take the form: `https://github.com/{user}/{repo}`, e.g.,
// `https://github.com/astral-sh/uv`.
let Some(mut segments) = url.path_segments() else {
debug!("GitHub URL is missing path segments: {url}");
return None;
};
let Some(owner) = segments.next() else {
debug!("GitHub URL is missing owner: {url}");
return None;
};
let Some(repo) = segments.next() else {
debug!("GitHub URL is missing repo: {url}");
return None;
};
if segments.next().is_some() {
debug!("GitHub URL has too many path segments: {url}");
return None;
}
// Trim off the `.git` from the repository, if present.
let repo = repo.strip_suffix(".git").unwrap_or(repo);
Some(Self { owner, repo })
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_parse_valid_url() {
let url = Url::parse("https://github.com/astral-sh/uv").unwrap();
let repo = GitHubRepository::parse(&url).unwrap();
assert_eq!(repo.owner, "astral-sh");
assert_eq!(repo.repo, "uv");
}
#[test]
fn test_parse_with_git_suffix() {
let url = Url::parse("https://github.com/astral-sh/uv.git").unwrap();
let repo = GitHubRepository::parse(&url).unwrap();
assert_eq!(repo.owner, "astral-sh");
assert_eq!(repo.repo, "uv");
}
#[test]
fn test_parse_invalid_host() {
let url = Url::parse("https://gitlab.com/astral-sh/uv").unwrap();
assert!(GitHubRepository::parse(&url).is_none());
}
#[test]
fn test_parse_invalid_path() {
let url = Url::parse("https://github.com/astral-sh").unwrap();
assert!(GitHubRepository::parse(&url).is_none());
let url = Url::parse("https://github.com/astral-sh/uv/extra").unwrap();
assert!(GitHubRepository::parse(&url).is_none());
}
}

View file

@ -1,9 +1,5 @@
use url::Url;
pub use crate::credentials::{store_credentials_from_url, GIT_STORE};
pub use crate::git::{GitReference, GIT};
pub use crate::github::GitHubRepository;
pub use crate::oid::{GitOid, OidParseError};
pub use crate::git::GIT;
pub use crate::resolver::{
GitResolver, GitResolverError, RepositoryReference, ResolvedRepositoryReference,
};
@ -11,124 +7,5 @@ pub use crate::source::{Fetch, GitSource, Reporter};
mod credentials;
mod git;
mod github;
mod oid;
mod resolver;
mod source;
/// A URL reference to a Git repository.
#[derive(Debug, Clone, PartialEq, PartialOrd, Eq, Hash, Ord)]
pub struct GitUrl {
/// The URL of the Git repository, with any query parameters, fragments, and leading `git+`
/// removed.
repository: Url,
/// The reference to the commit to use, which could be a branch, tag or revision.
reference: GitReference,
/// The precise commit to use, if known.
precise: Option<GitOid>,
}
impl GitUrl {
/// Create a new [`GitUrl`] from a repository URL and a reference.
pub fn from_reference(repository: Url, reference: GitReference) -> Self {
Self {
repository,
reference,
precise: None,
}
}
/// Create a new [`GitUrl`] from a repository URL and a precise commit.
pub fn from_commit(repository: Url, reference: GitReference, precise: GitOid) -> Self {
Self {
repository,
reference,
precise: Some(precise),
}
}
/// Set the precise [`GitOid`] to use for this Git URL.
#[must_use]
pub fn with_precise(mut self, precise: GitOid) -> Self {
self.precise = Some(precise);
self
}
/// Set the [`GitReference`] to use for this Git URL.
#[must_use]
pub fn with_reference(mut self, reference: GitReference) -> Self {
self.reference = reference;
self
}
/// Return the [`Url`] of the Git repository.
pub fn repository(&self) -> &Url {
&self.repository
}
/// Return the reference to the commit to use, which could be a branch, tag or revision.
pub fn reference(&self) -> &GitReference {
&self.reference
}
/// Return the precise commit, if known.
pub fn precise(&self) -> Option<GitOid> {
self.precise
}
}
impl TryFrom<Url> for GitUrl {
type Error = OidParseError;
/// Initialize a [`GitUrl`] source from a URL.
fn try_from(mut url: Url) -> Result<Self, Self::Error> {
// Remove any query parameters and fragments.
url.set_fragment(None);
url.set_query(None);
// If the URL ends with a reference, like `https://git.example.com/MyProject.git@v1.0`,
// extract it.
let mut reference = GitReference::DefaultBranch;
if let Some((prefix, suffix)) = url
.path()
.rsplit_once('@')
.map(|(prefix, suffix)| (prefix.to_string(), suffix.to_string()))
{
reference = GitReference::from_rev(suffix);
url.set_path(&prefix);
}
Ok(Self::from_reference(url, reference))
}
}
impl From<GitUrl> for Url {
fn from(git: GitUrl) -> Self {
let mut url = git.repository;
// If we have a precise commit, add `@` and the commit hash to the URL.
if let Some(precise) = git.precise {
url.set_path(&format!("{}@{}", url.path(), precise));
} else {
// Otherwise, add the branch or tag name.
match git.reference {
GitReference::Branch(rev)
| GitReference::Tag(rev)
| GitReference::BranchOrTag(rev)
| GitReference::NamedRef(rev)
| GitReference::BranchOrTagOrCommit(rev) => {
url.set_path(&format!("{}@{}", url.path(), rev));
}
GitReference::DefaultBranch => {}
}
}
url
}
}
impl std::fmt::Display for GitUrl {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.repository)
}
}

View file

@ -1,98 +0,0 @@
use std::fmt::Display;
use std::str::{self, FromStr};
use thiserror::Error;
/// Unique identity of any Git object (commit, tree, blob, tag).
///
/// Note this type does not validate whether the input is a valid hash.
#[derive(Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord, Hash)]
pub struct GitOid {
len: usize,
bytes: [u8; 40],
}
impl GitOid {
/// Return the string representation of an object ID.
pub fn as_str(&self) -> &str {
str::from_utf8(&self.bytes[..self.len]).unwrap()
}
/// Return a truncated representation, i.e., the first 16 characters of the SHA.
pub fn as_short_str(&self) -> &str {
&self.as_str()[..16]
}
}
#[derive(Debug, Error, PartialEq)]
pub enum OidParseError {
#[error("Object ID can be at most 40 hex characters")]
TooLong,
#[error("Object ID cannot be parsed from empty string")]
Empty,
}
impl FromStr for GitOid {
type Err = OidParseError;
fn from_str(s: &str) -> Result<Self, Self::Err> {
if s.is_empty() {
return Err(OidParseError::Empty);
}
if s.len() > 40 {
return Err(OidParseError::TooLong);
}
let mut out = [0; 40];
out[..s.len()].copy_from_slice(s.as_bytes());
Ok(GitOid {
len: s.len(),
bytes: out,
})
}
}
impl Display for GitOid {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
write!(f, "{}", self.as_str())
}
}
impl serde::Serialize for GitOid {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: serde::Serializer,
{
self.as_str().serialize(serializer)
}
}
impl<'de> serde::Deserialize<'de> for GitOid {
fn deserialize<D>(deserializer: D) -> Result<GitOid, D::Error>
where
D: serde::Deserializer<'de>,
{
let value = String::deserialize(deserializer)?;
GitOid::from_str(&value).map_err(serde::de::Error::custom)
}
}
#[cfg(test)]
mod tests {
use std::str::FromStr;
use super::{GitOid, OidParseError};
#[test]
fn git_oid() {
GitOid::from_str("4a23745badf5bf5ef7928f1e346e9986bd696d82").unwrap();
assert_eq!(GitOid::from_str(""), Err(OidParseError::Empty));
assert_eq!(
GitOid::from_str(&str::repeat("a", 41)),
Err(OidParseError::TooLong)
);
}
}

View file

@ -3,17 +3,19 @@ use std::path::PathBuf;
use std::str::FromStr;
use std::sync::Arc;
use tracing::debug;
use crate::{Fetch, GitHubRepository, GitOid, GitReference, GitSource, GitUrl, Reporter};
use dashmap::mapref::one::Ref;
use dashmap::DashMap;
use fs_err::tokio as fs;
use reqwest_middleware::ClientWithMiddleware;
use tracing::debug;
use uv_cache_key::{cache_digest, RepositoryUrl};
use uv_fs::LockedFile;
use uv_git_types::{GitHubRepository, GitOid, GitReference, GitUrl};
use uv_version::version;
use crate::{Fetch, GitSource, Reporter};
#[derive(Debug, thiserror::Error)]
pub enum GitResolverError {
#[error(transparent)]

View file

@ -12,9 +12,10 @@ use tracing::{debug, instrument};
use url::Url;
use uv_cache_key::{cache_digest, RepositoryUrl};
use uv_git_types::{GitOid, GitUrl};
use crate::git::GitRemote;
use crate::{GitOid, GitUrl, GIT_STORE};
use crate::GIT_STORE;
/// A remote Git source that can be checked out locally.
pub struct GitSource {
@ -65,10 +66,10 @@ impl GitSource {
}
/// Fetch the underlying Git repository at the given revision.
#[instrument(skip(self), fields(repository = %self.git.repository, rev = ?self.git.precise))]
#[instrument(skip(self), fields(repository = %self.git.repository(), rev = ?self.git.precise()))]
pub fn fetch(self) -> Result<Fetch> {
// Compute the canonical URL for the repository.
let canonical = RepositoryUrl::new(&self.git.repository);
let canonical = RepositoryUrl::new(self.git.repository());
// The path to the repo, within the Git database.
let ident = cache_digest(&canonical);
@ -76,17 +77,17 @@ impl GitSource {
// Authenticate the URL, if necessary.
let remote = if let Some(credentials) = GIT_STORE.get(&canonical) {
Cow::Owned(credentials.apply(self.git.repository.clone()))
Cow::Owned(credentials.apply(self.git.repository().clone()))
} else {
Cow::Borrowed(&self.git.repository)
Cow::Borrowed(self.git.repository())
};
let remote = GitRemote::new(&remote);
let (db, actual_rev, task) = match (self.git.precise, remote.db_at(&db_path).ok()) {
let (db, actual_rev, task) = match (self.git.precise(), remote.db_at(&db_path).ok()) {
// If we have a locked revision, and we have a preexisting database
// which has that revision, then no update needs to happen.
(Some(rev), Some(db)) if db.contains(rev) => {
debug!("Using existing Git source `{}`", self.git.repository);
debug!("Using existing Git source `{}`", self.git.repository());
(db, rev, None)
}
@ -95,17 +96,17 @@ impl GitSource {
// situation that we have a locked revision but the database
// doesn't have it.
(locked_rev, db) => {
debug!("Updating Git source `{}`", self.git.repository);
debug!("Updating Git source `{}`", self.git.repository());
// Report the checkout operation to the reporter.
let task = self.reporter.as_ref().map(|reporter| {
reporter.on_checkout_start(remote.url(), self.git.reference.as_rev())
reporter.on_checkout_start(remote.url(), self.git.reference().as_rev())
});
let (db, actual_rev) = remote.checkout(
&db_path,
db,
&self.git.reference,
self.git.reference(),
locked_rev.map(GitOid::from),
&self.client,
self.disable_ssl,