language-servers/ruff
1
0
Fork 0
mirror of https://github.com/astral-sh/ruff.git synced 2025-08-04 10:49:50 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

[red-knot] Don't use separate ID types for each alist (#16415)

Browse source 
Regardless of whether #16408 and #16311 pan out, this part is worth
pulling out as a separate PR.

Before, you had to define a new `IndexVec` index type for each type of
association list you wanted to create. Now there's a single index type
that's internal to the alist implementation, and you use `List<K, V>` to
store a handle to a particular list.

This also adds some property tests for the alist implementation.
This commit is contained in:
Douglas Creager 2025-02-28 14:55:55 -05:00 • committed by GitHub
parent fdf0915283
commit ba44e9de13
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
7 changed files with 768 additions and 790 deletions
Download patch file Download diff file Expand all files Collapse all files

1
.github/workflows/daily_property_tests.yaml vendored
View file

@ -47,6 +47,7 @@ jobs:
run: |
export QUICKCHECK_TESTS=100000
for _ in {1..5}; do
cargo test --locked --release --package red_knot_python_semantic -- --ignored list::property_tests
cargo test --locked --release --package red_knot_python_semantic -- --ignored types::property_tests::stable
done

1
crates/red_knot_python_semantic/src/lib.rs
View file

@ -14,6 +14,7 @@ pub use semantic_model::{HasType, SemanticModel};
pub mod ast_node_ref;
mod db;
pub mod lint;
pub(crate) mod list;
mod module_name;
mod module_resolver;
mod node_key;

745
crates/red_knot_python_semantic/src/list.rs Normal file
View file

@ -0,0 +1,745 @@
//! Sorted, arena-allocated association lists
//!
//! An [_association list_][alist], which is a linked list of key/value pairs. We additionally
//! guarantee that the elements of an association list are sorted (by their keys), and that they do
//! not contain any entries with duplicate keys.
//!
//! Association lists have fallen out of favor in recent decades, since you often need operations
//! that are inefficient on them. In particular, looking up a random element by index is O(n), just
//! like a linked list; and looking up an element by key is also O(n), since you must do a linear
//! scan of the list to find the matching element. The typical implementation also suffers from
//! poor cache locality and high memory allocation overhead, since individual list cells are
//! typically allocated separately from the heap. We solve that last problem by storing the cells
//! of an association list in an [`IndexVec`] arena.
//!
//! We exploit structural sharing where possible, reusing cells across multiple lists when we can.
//! That said, we don't guarantee that lists are canonical — it's entirely possible for two lists
//! with identical contents to use different list cells and have different identifiers.
//!
//! Given all of this, association lists have the following benefits:
//!
//! - Lists can be represented by a single 32-bit integer (the index into the arena of the head of
//! the list).
//! - Lists can be cloned in constant time, since the underlying cells are immutable.
//! - Lists can be combined quickly (for both intersection and union), especially when you already
//! have to zip through both input lists to combine each key's values in some way.
//!
//! There is one remaining caveat:
//!
//! - You should construct lists in key order; doing this lets you insert each value in constant time.
//! Inserting entries in reverse order results in _quadratic_ overall time to construct the list.
//!
//! Lists are created using a [`ListBuilder`], and once created are accessed via a [`ListStorage`].
//!
//! ## Tests
//!
//! This module contains quickcheck-based property tests.
//!
//! These tests are disabled by default, as they are non-deterministic and slow. You can run them
//! explicitly using:
//!
//! ```sh
//! cargo test -p ruff_index -- --ignored list::property_tests
//! ```
//!
//! The number of tests (default: 100) can be controlled by setting the `QUICKCHECK_TESTS`
//! environment variable. For example:
//!
//! ```sh
//! QUICKCHECK_TESTS=10000 cargo test …
//! ```
//!
//! If you want to run these tests for a longer period of time, it's advisable to run them in
//! release mode. As some tests are slower than others, it's advisable to run them in a loop until
//! they fail:
//!
//! ```sh
//! export QUICKCHECK_TESTS=100000
//! while cargo test --release -p ruff_index -- \
//! --ignored list::property_tests; do :; done
//! ```
//!
//! [alist]: https://en.wikipedia.org/wiki/Association_list
use std::cmp::Ordering;
use std::marker::PhantomData;
use std::ops::Deref;
use ruff_index::{newtype_index, IndexVec};
/// A handle to an association list. Use [`ListStorage`] to access its elements, and
/// [`ListBuilder`] to construct other lists based on this one.
#[derive(Clone, Copy, Debug, Eq, Ord, PartialEq, PartialOrd)]
pub(crate) struct List<K, V = ()> {
last: Option<ListCellId>,
_phantom: PhantomData<(K, V)>,
}
impl<K, V> List<K, V> {
pub(crate) const fn empty() -> List<K, V> {
List::new(None)
}
const fn new(last: Option<ListCellId>) -> List<K, V> {
List {
last,
_phantom: PhantomData,
}
}
}
impl<K, V> Default for List<K, V> {
fn default() -> Self {
List::empty()
}
}
#[newtype_index]
#[derive(PartialOrd, Ord)]
struct ListCellId;
/// Stores one or more association lists. This type provides read-only access to the lists. Use a
/// [`ListBuilder`] to create lists.
#[derive(Debug, Eq, PartialEq)]
pub(crate) struct ListStorage<K, V = ()> {
cells: IndexVec<ListCellId, ListCell<K, V>>,
}
/// Each association list is represented by a sequence of snoc cells. A snoc cell is like the more
/// familiar cons cell `(a : (b : (c : nil)))`, but in reverse `(((nil : a) : b) : c)`.
///
/// **Terminology**: The elements of a cons cell are usually called `head` and `tail` (assuming
/// you're not in Lisp-land, where they're called `car` and `cdr`). The elements of a snoc cell
/// are usually called `rest` and `last`.
#[derive(Debug, Eq, PartialEq)]
struct ListCell<K, V> {
rest: Option<ListCellId>,
key: K,
value: V,
}
/// Constructs one or more association lists.
#[derive(Debug, Eq, PartialEq)]
pub(crate) struct ListBuilder<K, V = ()> {
storage: ListStorage<K, V>,
/// Scratch space that lets us implement our list operations iteratively instead of
/// recursively.
///
/// The snoc-list representation that we use for alists is very common in functional
/// programming, and the simplest implementations of most of the operations are defined
/// recursively on that data structure. However, they are not _tail_ recursive, which means
/// that the call stack grows linearly with the size of the input, which can be a problem for
/// large lists.
///
/// You can often rework those recursive implementations into iterative ones using an
/// _accumulator_, but that comes at the cost of reversing the list. If we didn't care about
/// ordering, that wouldn't be a problem. Since we want our lists to be sorted, we can't rely
/// on that on its own.
///
/// The next standard trick is to use an accumulator, and use a fix-up step at the end to
/// reverse the (reversed) result in the accumulator, restoring the correct order.
///
/// So, that's what we do! However, as one last optimization, we don't build up alist cells in
/// our accumulator, since that would add wasteful cruft to our list storage. Instead, we use a
/// normal Vec as our accumulator, holding the key/value pairs that should be stitched onto the
/// end of whatever result list we are creating. For our fix-up step, we can consume a Vec in
/// reverse order by `pop`ping the elements off one by one.
scratch: Vec<(K, V)>,
}
impl<K, V> Default for ListBuilder<K, V> {
fn default() -> Self {
ListBuilder {
storage: ListStorage {
cells: IndexVec::default(),
},
scratch: Vec::default(),
}
}
}
impl<K, V> Deref for ListBuilder<K, V> {
type Target = ListStorage<K, V>;
fn deref(&self) -> &ListStorage<K, V> {
&self.storage
}
}
impl<K, V> ListBuilder<K, V> {
/// Finalizes a `ListBuilder`. After calling this, you cannot create any new lists managed by
/// this storage.
pub(crate) fn build(mut self) -> ListStorage<K, V> {
self.storage.cells.shrink_to_fit();
self.storage
}
/// Adds a new cell to the list.
///
/// Adding an element always returns a non-empty list, which means we could technically use `I`
/// as our return type, since we never return `None`. However, for consistency with our other
/// methods, we always use `Option<I>` as the return type for any method that can return a
/// list.
#[allow(clippy::unnecessary_wraps)]
fn add_cell(&mut self, rest: Option<ListCellId>, key: K, value: V) -> Option<ListCellId> {
Some(self.storage.cells.push(ListCell { rest, key, value }))
}
/// Returns an entry pointing at where `key` would be inserted into a list.
///
/// Note that when we add a new element to a list, we might have to clone the keys and values
/// of some existing elements. This is because list cells are immutable once created, since
/// they might be shared across multiple lists. We must therefore create new cells for every
/// element that appears after the new element.
///
/// That means that you should construct lists in key order, since that means that there are no
/// entries to duplicate for each insertion. If you construct the list in reverse order, we
/// will have to duplicate O(n) entries for each insertion, making it _quadratic_ to construct
/// the entire list.
pub(crate) fn entry(&mut self, list: List<K, V>, key: K) -> ListEntry<K, V>
where
K: Clone + Ord,
V: Clone,
{
self.scratch.clear();
// Iterate through the input list, looking for the position where the key should be
// inserted. We will need to create new list cells for any elements that appear after the
// new key. Stash those away in our scratch accumulator as we step through the input. The
// result of the loop is that "rest" of the result list, which we will stitch the new key
// (and any succeeding keys) onto.
let mut curr = list.last;
while let Some(curr_id) = curr {
let cell = &self.storage.cells[curr_id];
match key.cmp(&cell.key) {
// We found an existing entry in the input list with the desired key.
Ordering::Equal => {
return ListEntry {
builder: self,
list,
key,
rest: ListTail::Occupied(curr_id),
};
}
// The input list does not already contain this key, and this is where we should
// add it.
Ordering::Greater => {
return ListEntry {
builder: self,
list,
key,
rest: ListTail::Vacant(curr_id),
};
}
// If this key is in the list, it's further along. We'll need to create a new cell
// for this entry in the result list, so add its contents to the scratch
// accumulator.
Ordering::Less => {
let new_key = cell.key.clone();
let new_value = cell.value.clone();
self.scratch.push((new_key, new_value));
curr = cell.rest;
}
}
}
// We made it all the way through the list without finding the desired key, so it belongs
// at the beginning. (And we will unfortunately have to duplicate every existing cell if
// the caller proceeds with inserting the new key!)
ListEntry {
builder: self,
list,
key,
rest: ListTail::Beginning,
}
}
}
/// A view into a list, indicating where a key would be inserted.
pub(crate) struct ListEntry<'a, K, V = ()> {
builder: &'a mut ListBuilder<K, V>,
list: List<K, V>,
key: K,
/// Points at the element that already contains `key`, if there is one, or the element
/// immediately before where it would go, if not.
rest: ListTail<ListCellId>,
}
enum ListTail<I> {
/// The list does not already contain `key`, and it would go at the beginning of the list.
Beginning,
/// The list already contains `key`
Occupied(I),
/// The list does not already contain key, and it would go immediately after the given element
Vacant(I),
}
impl<K, V> ListEntry<'_, K, V>
where
K: Clone,
V: Clone,
{
fn stitch_up(self, rest: Option<ListCellId>, value: V) -> List<K, V> {
let mut last = rest;
last = self.builder.add_cell(last, self.key, value);
while let Some((key, value)) = self.builder.scratch.pop() {
last = self.builder.add_cell(last, key, value);
}
List::new(last)
}
/// Inserts a new key/value into the list if the key is not already present. If the list
/// already contains `key`, we return the original list as-is, and do not invoke your closure.
pub(crate) fn or_insert_with<F>(self, f: F) -> List<K, V>
where
F: FnOnce() -> V,
{
let rest = match self.rest {
// If the list already contains `key`, we don't need to replace anything, and can
// return the original list unmodified.
ListTail::Occupied(_) => return self.list,
// Otherwise we have to create a new entry and stitch it onto the list.
ListTail::Beginning => None,
ListTail::Vacant(index) => Some(index),
};
self.stitch_up(rest, f())
}
/// Inserts a new key and the default value into the list if the key is not already present. If
/// the list already contains `key`, we return the original list as-is.
pub(crate) fn or_insert_default(self) -> List<K, V>
where
V: Default,
{
self.or_insert_with(V::default)
}
}
impl<K, V> ListBuilder<K, V> {
/// Returns the intersection of two lists. The result will contain an entry for any key that
/// appears in both lists. The corresponding values will be combined using the `combine`
/// function that you provide.
#[allow(clippy::needless_pass_by_value)]
pub(crate) fn intersect_with<F>(
&mut self,
a: List<K, V>,
b: List<K, V>,
mut combine: F,
) -> List<K, V>
where
K: Clone + Ord,
V: Clone,
F: FnMut(&V, &V) -> V,
{
self.scratch.clear();
// Zip through the lists, building up the keys/values of the new entries into our scratch
// vector. Continue until we run out of elements in either list. (Any remaining elements in
// the other list cannot possibly be in the intersection.)
let mut a = a.last;
let mut b = b.last;
while let (Some(a_id), Some(b_id)) = (a, b) {
let a_cell = &self.storage.cells[a_id];
let b_cell = &self.storage.cells[b_id];
match a_cell.key.cmp(&b_cell.key) {
// Both lists contain this key; combine their values
Ordering::Equal => {
let new_key = a_cell.key.clone();
let new_value = combine(&a_cell.value, &b_cell.value);
self.scratch.push((new_key, new_value));
a = a_cell.rest;
b = b_cell.rest;
}
// a's key is only present in a, so it's not included in the result.
Ordering::Greater => a = a_cell.rest,
// b's key is only present in b, so it's not included in the result.
Ordering::Less => b = b_cell.rest,
}
}
// Once the iteration loop terminates, we stitch the new entries back together into proper
// alist cells.
let mut last = None;
while let Some((key, value)) = self.scratch.pop() {
last = self.add_cell(last, key, value);
}
List::new(last)
}
}
// ----
// Sets
impl<K> ListStorage<K, ()> {
/// Iterates through the elements in a set _in reverse order_.
#[allow(clippy::needless_pass_by_value)]
pub(crate) fn iter_set_reverse(&self, set: List<K, ()>) -> ListSetReverseIterator<K> {
ListSetReverseIterator {
storage: self,
curr: set.last,
}
}
}
pub(crate) struct ListSetReverseIterator<'a, K> {
storage: &'a ListStorage<K, ()>,
curr: Option<ListCellId>,
}
impl<'a, K> Iterator for ListSetReverseIterator<'a, K> {
type Item = &'a K;
fn next(&mut self) -> Option<Self::Item> {
let cell = &self.storage.cells[self.curr?];
self.curr = cell.rest;
Some(&cell.key)
}
}
impl<K> ListBuilder<K, ()> {
/// Adds an element to a set.
pub(crate) fn insert(&mut self, set: List<K, ()>, element: K) -> List<K, ()>
where
K: Clone + Ord,
{
self.entry(set, element).or_insert_default()
}
/// Returns the intersection of two sets. The result will contain any value that appears in
/// both sets.
pub(crate) fn intersect(&mut self, a: List<K, ()>, b: List<K, ()>) -> List<K, ()>
where
K: Clone + Ord,
{
self.intersect_with(a, b, |(), ()| ())
}
}
// -----
// Tests
#[cfg(test)]
mod tests {
use super::*;
use std::fmt::Display;
use std::fmt::Write;
// ----
// Sets
impl<K> ListStorage<K>
where
K: Display,
{
fn display_set(&self, list: List<K, ()>) -> String {
let elements: Vec<_> = self.iter_set_reverse(list).collect();
let mut result = String::new();
result.push('[');
for element in elements.into_iter().rev() {
if result.len() > 1 {
result.push_str(", ");
}
write!(&mut result, "{element}").unwrap();
}
result.push(']');
result
}
}
#[test]
fn can_insert_into_set() {
let mut builder = ListBuilder::<u16>::default();
// Build up the set in order
let empty = List::empty();
let set1 = builder.insert(empty, 1);
let set12 = builder.insert(set1, 2);
let set123 = builder.insert(set12, 3);
let set1232 = builder.insert(set123, 2);
assert_eq!(builder.display_set(empty), "[]");
assert_eq!(builder.display_set(set1), "[1]");
assert_eq!(builder.display_set(set12), "[1, 2]");
assert_eq!(builder.display_set(set123), "[1, 2, 3]");
assert_eq!(builder.display_set(set1232), "[1, 2, 3]");
// And in reverse order
let set3 = builder.insert(empty, 3);
let set32 = builder.insert(set3, 2);
let set321 = builder.insert(set32, 1);
let set3212 = builder.insert(set321, 2);
assert_eq!(builder.display_set(empty), "[]");
assert_eq!(builder.display_set(set3), "[3]");
assert_eq!(builder.display_set(set32), "[2, 3]");
assert_eq!(builder.display_set(set321), "[1, 2, 3]");
assert_eq!(builder.display_set(set3212), "[1, 2, 3]");
}
#[test]
fn can_intersect_sets() {
let mut builder = ListBuilder::<u16>::default();
let empty = List::empty();
let set1 = builder.insert(empty, 1);
let set12 = builder.insert(set1, 2);
let set123 = builder.insert(set12, 3);
let set1234 = builder.insert(set123, 4);
let set2 = builder.insert(empty, 2);
let set24 = builder.insert(set2, 4);
let set245 = builder.insert(set24, 5);
let set2457 = builder.insert(set245, 7);
let intersection = builder.intersect(empty, empty);
assert_eq!(builder.display_set(intersection), "[]");
let intersection = builder.intersect(empty, set1234);
assert_eq!(builder.display_set(intersection), "[]");
let intersection = builder.intersect(empty, set2457);
assert_eq!(builder.display_set(intersection), "[]");
let intersection = builder.intersect(set1, set1234);
assert_eq!(builder.display_set(intersection), "[1]");
let intersection = builder.intersect(set1, set2457);
assert_eq!(builder.display_set(intersection), "[]");
let intersection = builder.intersect(set2, set1234);
assert_eq!(builder.display_set(intersection), "[2]");
let intersection = builder.intersect(set2, set2457);
assert_eq!(builder.display_set(intersection), "[2]");
let intersection = builder.intersect(set1234, set2457);
assert_eq!(builder.display_set(intersection), "[2, 4]");
}
// ----
// Maps
impl<K, V> ListStorage<K, V> {
/// Iterates through the entries in a list _in reverse order by key_.
#[allow(clippy::needless_pass_by_value)]
pub(crate) fn iter_reverse(&self, list: List<K, V>) -> ListReverseIterator<'_, K, V> {
ListReverseIterator {
storage: self,
curr: list.last,
}
}
}
pub(crate) struct ListReverseIterator<'a, K, V> {
storage: &'a ListStorage<K, V>,
curr: Option<ListCellId>,
}
impl<'a, K, V> Iterator for ListReverseIterator<'a, K, V> {
type Item = (&'a K, &'a V);
fn next(&mut self) -> Option<Self::Item> {
let cell = &self.storage.cells[self.curr?];
self.curr = cell.rest;
Some((&cell.key, &cell.value))
}
}
impl<K, V> ListStorage<K, V>
where
K: Display,
V: Display,
{
fn display(&self, list: List<K, V>) -> String {
let entries: Vec<_> = self.iter_reverse(list).collect();
let mut result = String::new();
result.push('[');
for (key, value) in entries.into_iter().rev() {
if result.len() > 1 {
result.push_str(", ");
}
write!(&mut result, "{key}:{value}").unwrap();
}
result.push(']');
result
}
}
#[test]
fn can_insert_into_map() {
let mut builder = ListBuilder::<u16, u16>::default();
// Build up the map in order
let empty = List::empty();
let map1 = builder.entry(empty, 1).or_insert_with(|| 1);
let map12 = builder.entry(map1, 2).or_insert_with(|| 2);
let map123 = builder.entry(map12, 3).or_insert_with(|| 3);
let map1232 = builder.entry(map123, 2).or_insert_with(|| 4);
assert_eq!(builder.display(empty), "[]");
assert_eq!(builder.display(map1), "[1:1]");
assert_eq!(builder.display(map12), "[1:1, 2:2]");
assert_eq!(builder.display(map123), "[1:1, 2:2, 3:3]");
assert_eq!(builder.display(map1232), "[1:1, 2:2, 3:3]");
// And in reverse order
let map3 = builder.entry(empty, 3).or_insert_with(|| 3);
let map32 = builder.entry(map3, 2).or_insert_with(|| 2);
let map321 = builder.entry(map32, 1).or_insert_with(|| 1);
let map3212 = builder.entry(map321, 2).or_insert_with(|| 4);
assert_eq!(builder.display(empty), "[]");
assert_eq!(builder.display(map3), "[3:3]");
assert_eq!(builder.display(map32), "[2:2, 3:3]");
assert_eq!(builder.display(map321), "[1:1, 2:2, 3:3]");
assert_eq!(builder.display(map3212), "[1:1, 2:2, 3:3]");
}
#[test]
fn can_intersect_maps() {
let mut builder = ListBuilder::<u16, u16>::default();
let empty = List::empty();
let map1 = builder.entry(empty, 1).or_insert_with(|| 1);
let map12 = builder.entry(map1, 2).or_insert_with(|| 2);
let map123 = builder.entry(map12, 3).or_insert_with(|| 3);
let map1234 = builder.entry(map123, 4).or_insert_with(|| 4);
let map2 = builder.entry(empty, 2).or_insert_with(|| 20);
let map24 = builder.entry(map2, 4).or_insert_with(|| 40);
let map245 = builder.entry(map24, 5).or_insert_with(|| 50);
let map2457 = builder.entry(map245, 7).or_insert_with(|| 70);
let intersection = builder.intersect_with(empty, empty, |a, b| a + b);
assert_eq!(builder.display(intersection), "[]");
let intersection = builder.intersect_with(empty, map1234, |a, b| a + b);
assert_eq!(builder.display(intersection), "[]");
let intersection = builder.intersect_with(empty, map2457, |a, b| a + b);
assert_eq!(builder.display(intersection), "[]");
let intersection = builder.intersect_with(map1, map1234, |a, b| a + b);
assert_eq!(builder.display(intersection), "[1:2]");
let intersection = builder.intersect_with(map1, map2457, |a, b| a + b);
assert_eq!(builder.display(intersection), "[]");
let intersection = builder.intersect_with(map2, map1234, |a, b| a + b);
assert_eq!(builder.display(intersection), "[2:22]");
let intersection = builder.intersect_with(map2, map2457, |a, b| a + b);
assert_eq!(builder.display(intersection), "[2:40]");
let intersection = builder.intersect_with(map1234, map2457, |a, b| a + b);
assert_eq!(builder.display(intersection), "[2:22, 4:44]");
}
}
// --------------
// Property tests
#[cfg(test)]
mod property_tests {
use super::*;
use std::collections::{BTreeMap, BTreeSet};
impl<K> ListBuilder<K>
where
K: Clone + Ord,
{
fn set_from_elements<'a>(&mut self, elements: impl IntoIterator<Item = &'a K>) -> List<K>
where
K: 'a,
{
let mut set = List::empty();
for element in elements {
set = self.insert(set, element.clone());
}
set
}
}
// For most of the tests below, we use a vec as our input, instead of a HashSet or BTreeSet,
// since we want to test the behavior of adding duplicate elements to the set.
#[quickcheck_macros::quickcheck]
#[ignore]
#[allow(clippy::needless_pass_by_value)]
fn roundtrip_set_from_vec(elements: Vec<u16>) -> bool {
let mut builder = ListBuilder::default();
let set = builder.set_from_elements(&elements);
let expected: BTreeSet<_> = elements.iter().copied().collect();
let actual = builder.iter_set_reverse(set).copied();
actual.eq(expected.into_iter().rev())
}
#[quickcheck_macros::quickcheck]
#[ignore]
#[allow(clippy::needless_pass_by_value)]
fn roundtrip_set_intersection(a_elements: Vec<u16>, b_elements: Vec<u16>) -> bool {
let mut builder = ListBuilder::default();
let a = builder.set_from_elements(&a_elements);
let b = builder.set_from_elements(&b_elements);
let intersection = builder.intersect(a, b);
let a_set: BTreeSet<_> = a_elements.iter().copied().collect();
let b_set: BTreeSet<_> = b_elements.iter().copied().collect();
let expected: Vec<_> = a_set.intersection(&b_set).copied().collect();
let actual = builder.iter_set_reverse(intersection).copied();
actual.eq(expected.into_iter().rev())
}
impl<K, V> ListBuilder<K, V>
where
K: Clone + Ord,
V: Clone + Eq,
{
fn set_from_pairs<'a, I>(&mut self, pairs: I) -> List<K, V>
where
K: 'a,
V: 'a,
I: IntoIterator<Item = &'a (K, V)>,
I::IntoIter: DoubleEndedIterator,
{
let mut list = List::empty();
for (key, value) in pairs.into_iter().rev() {
list = self
.entry(list, key.clone())
.or_insert_with(|| value.clone());
}
list
}
}
fn join<K, V>(a: &BTreeMap<K, V>, b: &BTreeMap<K, V>) -> BTreeMap<K, (Option<V>, Option<V>)>
where
K: Clone + Ord,
V: Clone + Ord,
{
let mut joined: BTreeMap<K, (Option<V>, Option<V>)> = BTreeMap::new();
for (k, v) in a {
joined.entry(k.clone()).or_default().0 = Some(v.clone());
}
for (k, v) in b {
joined.entry(k.clone()).or_default().1 = Some(v.clone());
}
joined
}
#[quickcheck_macros::quickcheck]
#[ignore]
#[allow(clippy::needless_pass_by_value)]
fn roundtrip_list_from_vec(pairs: Vec<(u16, u16)>) -> bool {
let mut builder = ListBuilder::default();
let list = builder.set_from_pairs(&pairs);
let expected: BTreeMap<_, _> = pairs.iter().copied().collect();
let actual = builder.iter_reverse(list).map(|(k, v)| (*k, *v));
actual.eq(expected.into_iter().rev())
}
#[quickcheck_macros::quickcheck]
#[ignore]
#[allow(clippy::needless_pass_by_value)]
fn roundtrip_list_intersection(
a_elements: Vec<(u16, u16)>,
b_elements: Vec<(u16, u16)>,
) -> bool {
let mut builder = ListBuilder::default();
let a = builder.set_from_pairs(&a_elements);
let b = builder.set_from_pairs(&b_elements);
let intersection = builder.intersect_with(a, b, |a, b| a + b);
let a_map: BTreeMap<_, _> = a_elements.iter().copied().collect();
let b_map: BTreeMap<_, _> = b_elements.iter().copied().collect();
let intersection_map = join(&a_map, &b_map);
let expected: Vec<_> = intersection_map
.into_iter()
.filter_map(|(k, (v1, v2))| Some((k, v1? + v2?)))
.collect();
let actual = builder.iter_reverse(intersection).map(|(k, v)| (*k, *v));
actual.eq(expected.into_iter().rev())
}
}

41
crates/red_knot_python_semantic/src/semantic_index/narrowing_constraints.rs
View file

@ -16,10 +16,10 @@
//! - Iterating through the predicates in a constraint
//!
//! In particular, note that we do not need random access to the predicates in a constraint. That
//! means that we can use a simple [_sorted association list_][ruff_index::list] as our data
//! structure. That lets us use a single 32-bit integer to store each narrowing constraint, no
//! matter how many predicates it contains. It also makes merging two narrowing constraints fast,
//! since alists support fast intersection.
//! means that we can use a simple [_sorted association list_][crate::list] as our data structure.
//! That lets us use a single 32-bit integer to store each narrowing constraint, no matter how many
//! predicates it contains. It also makes merging two narrowing constraints fast, since alists
//! support fast intersection.
//!
//! Because we visit the contents of each scope in source-file order, and assign scoped IDs in
//! source-file order, that means that we will tend to visit narrowing constraints in order by
@ -28,21 +28,15 @@
//!
//! [`Predicate`]: crate::semantic_index::predicate::Predicate
use ruff_index::list::{ListBuilder, ListSetReverseIterator, ListStorage};
use ruff_index::newtype_index;
use crate::list::{List, ListBuilder, ListSetReverseIterator, ListStorage};
use crate::semantic_index::predicate::ScopedPredicateId;
/// A narrowing constraint associated with a live binding.
///
/// A constraint is a list of [`Predicate`]s that each constrain the type of the binding's symbol.
///
/// An instance of this type represents a _non-empty_ narrowing constraint. You will often wrap
/// this in `Option` and use `None` to represent an empty narrowing constraint.
///
/// [`Predicate`]: crate::semantic_index::predicate::Predicate
#[newtype_index]
pub(crate) struct ScopedNarrowingConstraintId;
pub(crate) type ScopedNarrowingConstraint = List<ScopedNarrowingConstraintPredicate>;
/// One of the [`Predicate`]s in a narrowing constraint, which constraints the type of the
/// binding's symbol.
@ -71,7 +65,7 @@ impl From<ScopedPredicateId> for ScopedNarrowingConstraintPredicate {
/// A collection of narrowing constraints for a given scope.
#[derive(Debug, Eq, PartialEq)]
pub(crate) struct NarrowingConstraints {
lists: ListStorage<ScopedNarrowingConstraintId, ScopedNarrowingConstraintPredicate>,
lists: ListStorage<ScopedNarrowingConstraintPredicate>,
}
// Building constraints
@ -80,7 +74,7 @@ pub(crate) struct NarrowingConstraints {
/// A builder for creating narrowing constraints.
#[derive(Debug, Default, Eq, PartialEq)]
pub(crate) struct NarrowingConstraintsBuilder {
lists: ListBuilder<ScopedNarrowingConstraintId, ScopedNarrowingConstraintPredicate>,
lists: ListBuilder<ScopedNarrowingConstraintPredicate>,
}
impl NarrowingConstraintsBuilder {
@ -93,9 +87,9 @@ impl NarrowingConstraintsBuilder {
/// Adds a predicate to an existing narrowing constraint.
pub(crate) fn add_predicate_to_constraint(
&mut self,
constraint: Option<ScopedNarrowingConstraintId>,
constraint: ScopedNarrowingConstraint,
predicate: ScopedNarrowingConstraintPredicate,
) -> Option<ScopedNarrowingConstraintId> {
) -> ScopedNarrowingConstraint {
self.lists.insert(constraint, predicate)
}
@ -103,9 +97,9 @@ impl NarrowingConstraintsBuilder {
/// that appear in both inputs.
pub(crate) fn intersect_constraints(
&mut self,
a: Option<ScopedNarrowingConstraintId>,
b: Option<ScopedNarrowingConstraintId>,
) -> Option<ScopedNarrowingConstraintId> {
a: ScopedNarrowingConstraint,
b: ScopedNarrowingConstraint,
) -> ScopedNarrowingConstraint {
self.lists.intersect(a, b)
}
}
@ -113,15 +107,14 @@ impl NarrowingConstraintsBuilder {
// Iteration
// ---------
pub(crate) type NarrowingConstraintsIterator<'a> = std::iter::Copied<
ListSetReverseIterator<'a, ScopedNarrowingConstraintId, ScopedNarrowingConstraintPredicate>,
>;
pub(crate) type NarrowingConstraintsIterator<'a> =
std::iter::Copied<ListSetReverseIterator<'a, ScopedNarrowingConstraintPredicate>>;
impl NarrowingConstraints {
/// Iterates over the predicates in a narrowing constraint.
pub(crate) fn iter_predicates(
&self,
set: Option<ScopedNarrowingConstraintId>,
set: ScopedNarrowingConstraint,
) -> NarrowingConstraintsIterator<'_> {
self.lists.iter_set_reverse(set).copied()
}
@ -143,7 +136,7 @@ mod tests {
impl NarrowingConstraintsBuilder {
pub(crate) fn iter_predicates(
&self,
set: Option<ScopedNarrowingConstraintId>,
set: ScopedNarrowingConstraint,
) -> NarrowingConstraintsIterator<'_> {
self.lists.iter_set_reverse(set).copied()
}

8
crates/red_knot_python_semantic/src/semantic_index/use_def/symbol_state.rs
View file

@ -47,7 +47,7 @@ use ruff_index::newtype_index;
use smallvec::{smallvec, SmallVec};
use crate::semantic_index::narrowing_constraints::{
NarrowingConstraintsBuilder, ScopedNarrowingConstraintId, ScopedNarrowingConstraintPredicate,
NarrowingConstraintsBuilder, ScopedNarrowingConstraint, ScopedNarrowingConstraintPredicate,
};
use crate::semantic_index::visibility_constraints::{
ScopedVisibilityConstraintId, VisibilityConstraintsBuilder,
@ -189,7 +189,7 @@ pub(super) struct SymbolBindings {
#[derive(Clone, Debug, PartialEq, Eq)]
pub(super) struct LiveBinding {
pub(super) binding: ScopedDefinitionId,
pub(super) narrowing_constraint: Option<ScopedNarrowingConstraintId>,
pub(super) narrowing_constraint: ScopedNarrowingConstraint,
pub(super) visibility_constraint: ScopedVisibilityConstraintId,
}
@ -199,7 +199,7 @@ impl SymbolBindings {
fn unbound(scope_start_visibility: ScopedVisibilityConstraintId) -> Self {
let initial_binding = LiveBinding {
binding: ScopedDefinitionId::UNBOUND,
narrowing_constraint: None,
narrowing_constraint: ScopedNarrowingConstraint::empty(),
visibility_constraint: scope_start_visibility,
};
Self {
@ -218,7 +218,7 @@ impl SymbolBindings {
self.live_bindings.clear();
self.live_bindings.push(LiveBinding {
binding,
narrowing_constraint: None,
narrowing_constraint: ScopedNarrowingConstraint::empty(),
visibility_constraint,
});
}

1
crates/ruff_index/src/lib.rs
View file

@ -4,7 +4,6 @@
//! Inspired by [rustc_index](https://github.com/rust-lang/rust/blob/master/compiler/rustc_index/src/lib.rs).
mod idx;
pub mod list;
mod slice;
mod vec;

761
crates/ruff_index/src/list.rs
View file

@ -1,761 +0,0 @@
use std::cmp::Ordering;
use std::ops::Deref;
use crate::vec::IndexVec;
use crate::Idx;
/// Stores one or more _association lists_, which are linked lists of key/value pairs. We
/// additionally guarantee that the elements of an association list are sorted (by their keys), and
/// that they do not contain any entries with duplicate keys.
///
/// Association lists have fallen out of favor in recent decades, since you often need operations
/// that are inefficient on them. In particular, looking up a random element by index is O(n), just
/// like a linked list; and looking up an element by key is also O(n), since you must do a linear
/// scan of the list to find the matching element. The typical implementation also suffers from
/// poor cache locality and high memory allocation overhead, since individual list cells are
/// typically allocated separately from the heap.
///
/// We solve that last problem by storing the cells of an association list in an [`IndexVec`]
/// arena. You provide the index type (`I`) that you want to use with this arena. That means that
/// an individual association list is represented by an `Option<I>`, with `None` representing an
/// empty list.
///
/// We exploit structural sharing where possible, reusing cells across multiple lists when we can.
/// That said, we don't guarantee that lists are canonical — it's entirely possible for two lists
/// with identical contents to use different list cells and have different identifiers.
///
/// Given all of this, association lists have the following benefits:
///
/// - Lists can be represented by a single 32-bit integer (the index into the arena of the head of
/// the list).
/// - Lists can be cloned in constant time, since the underlying cells are immutable.
/// - Lists can be combined quickly (for both intersection and union), especially when you already
/// have to zip through both input lists to combine each key's values in some way.
///
/// There is one remaining caveat:
///
/// - You should construct lists in key order; doing this lets you insert each value in constant time.
/// Inserting entries in reverse order results in _quadratic_ overall time to construct the list.
///
/// This type provides read-only access to the lists. Use a [`ListBuilder`] to create lists.
#[derive(Debug, Eq, PartialEq)]
pub struct ListStorage<I, K, V = ()> {
cells: IndexVec<I, ListCell<I, K, V>>,
}
/// Each association list is represented by a sequence of snoc cells. A snoc cell is like the more
/// familiar cons cell `(a : (b : (c : nil)))`, but in reverse `(((nil : a) : b) : c)`.
///
/// **Terminology**: The elements of a cons cell are usually called `head` and `tail` (assuming
/// you're not in Lisp-land, where they're called `car` and `cdr`). The elements of a snoc cell
/// are usually called `rest` and `last`.
///
/// We use a tuple struct instead of named fields because we always unpack a cell into local
/// variables:
///
/// ```ignore
/// let ListCell(rest, last_key, last_value) = /* ... */;
/// ```
#[derive(Debug, Eq, PartialEq)]
struct ListCell<I, K, V>(Option<I>, K, V);
impl<I: Idx, K, V> ListStorage<I, K, V> {
/// Iterates through the entries in a list _in reverse order by key_.
pub fn iter_reverse(&self, list: Option<I>) -> ListReverseIterator<'_, I, K, V> {
ListReverseIterator {
storage: self,
curr: list,
}
}
}
pub struct ListReverseIterator<'a, I, K, V> {
storage: &'a ListStorage<I, K, V>,
curr: Option<I>,
}
impl<'a, I: Idx, K, V> Iterator for ListReverseIterator<'a, I, K, V> {
type Item = (&'a K, &'a V);
fn next(&mut self) -> Option<Self::Item> {
let ListCell(rest, key, value) = &self.storage.cells[self.curr?];
self.curr = *rest;
Some((key, value))
}
}
/// Constructs one or more association lists.
#[derive(Debug, Eq, PartialEq)]
pub struct ListBuilder<I, K, V = ()> {
storage: ListStorage<I, K, V>,
/// Scratch space that lets us implement our list operations iteratively instead of
/// recursively.
///
/// The snoc-list representation that we use for alists is very common in functional
/// programming, and the simplest implementations of most of the operations are defined
/// recursively on that data structure. However, they are not _tail_ recursive, which means
/// that the call stack grows linearly with the size of the input, which can be a problem for
/// large lists.
///
/// You can often rework those recursive implementations into iterative ones using an
/// _accumulator_, but that comes at the cost of reversing the list. If we didn't care about
/// ordering, that wouldn't be a problem. Since we want our lists to be sorted, we can't rely
/// on that on its own.
///
/// The next standard trick is to use an accumulator, and use a fix-up step at the end to
/// reverse the (reversed) result in the accumulator, restoring the correct order.
///
/// So, that's what we do! However, as one last optimization, we don't build up alist cells in
/// our accumulator, since that would add wasteful cruft to our list storage. Instead, we use a
/// normal Vec as our accumulator, holding the key/value pairs that should be stitched onto the
/// end of whatever result list we are creating. For our fix-up step, we can consume a Vec in
/// reverse order by `pop`ping the elements off one by one.
scratch: Vec<(K, V)>,
}
impl<I: Idx, K, V> Default for ListBuilder<I, K, V> {
fn default() -> Self {
ListBuilder {
storage: ListStorage {
cells: IndexVec::default(),
},
scratch: Vec::default(),
}
}
}
impl<I, K, V> Deref for ListBuilder<I, K, V> {
type Target = ListStorage<I, K, V>;
fn deref(&self) -> &ListStorage<I, K, V> {
&self.storage
}
}
impl<I: Idx, K, V> ListBuilder<I, K, V> {
/// Finalizes a `ListBuilder`. After calling this, you cannot create any new lists managed by
/// this storage.
pub fn build(mut self) -> ListStorage<I, K, V> {
self.storage.cells.shrink_to_fit();
self.storage
}
/// Adds a new cell to the list.
///
/// Adding an element always returns a non-empty list, which means we could technically use `I`
/// as our return type, since we never return `None`. However, for consistency with our other
/// methods, we always use `Option<I>` as the return type for any method that can return a
/// list.
#[allow(clippy::unnecessary_wraps)]
fn add_cell(&mut self, rest: Option<I>, key: K, value: V) -> Option<I> {
Some(self.storage.cells.push(ListCell(rest, key, value)))
}
/// Returns an entry pointing at where `key` would be inserted into a list.
///
/// Note that when we add a new element to a list, we might have to clone the keys and values
/// of some existing elements. This is because list cells are immutable once created, since
/// they might be shared across multiple lists. We must therefore create new cells for every
/// element that appears after the new element.
///
/// That means that you should construct lists in key order, since that means that there are no
/// entries to duplicate for each insertion. If you construct the list in reverse order, we
/// will have to duplicate O(n) entries for each insertion, making it _quadratic_ to construct
/// the entire list.
pub fn entry(&mut self, list: Option<I>, key: K) -> ListEntry<I, K, V>
where
K: Clone + Ord,
V: Clone,
{
self.scratch.clear();
// Iterate through the input list, looking for the position where the key should be
// inserted. We will need to create new list cells for any elements that appear after the
// new key. Stash those away in our scratch accumulator as we step through the input. The
// result of the loop is that "rest" of the result list, which we will stitch the new key
// (and any succeeding keys) onto.
let mut curr = list;
while let Some(curr_id) = curr {
let ListCell(rest, curr_key, curr_value) = &self.storage.cells[curr_id];
match key.cmp(curr_key) {
// We found an existing entry in the input list with the desired key.
Ordering::Equal => {
return ListEntry {
builder: self,
list,
key,
rest: ListTail::Occupied(curr_id),
};
}
// The input list does not already contain this key, and this is where we should
// add it.
Ordering::Greater => {
return ListEntry {
builder: self,
list,
key,
rest: ListTail::Vacant(curr_id),
};
}
// If this key is in the list, it's further along. We'll need to create a new cell
// for this entry in the result list, so add its contents to the scratch
// accumulator.
Ordering::Less => {
let new_key = curr_key.clone();
let new_value = curr_value.clone();
self.scratch.push((new_key, new_value));
curr = *rest;
}
}
}
// We made it all the way through the list without finding the desired key, so it belongs
// at the beginning. (And we will unfortunately have to duplicate every existing cell if
// the caller proceeds with inserting the new key!)
ListEntry {
builder: self,
list,
key,
rest: ListTail::Beginning,
}
}
}
/// A view into a list, indicating where a key would be inserted.
pub struct ListEntry<'a, I, K, V> {
builder: &'a mut ListBuilder<I, K, V>,
list: Option<I>,
key: K,
/// Points at the element that already contains `key`, if there is one, or the element
/// immediately before where it would go, if not.
rest: ListTail<I>,
}
enum ListTail<I> {
/// The list does not already contain `key`, and it would go at the beginning of the list.
Beginning,
/// The list already contains `key`
Occupied(I),
/// The list does not already contain key, and it would go immediately after the given element
Vacant(I),
}
impl<I: Idx, K, V> ListEntry<'_, I, K, V>
where
K: Clone + Ord,
V: Clone,
{
fn stitch_up(self, rest: Option<I>, value: V) -> Option<I> {
let mut result = rest;
result = self.builder.add_cell(result, self.key, value);
while let Some((key, value)) = self.builder.scratch.pop() {
result = self.builder.add_cell(result, key, value);
}
result
}
/// Inserts a new key/value into the list if the key is not already present. If the list
/// already contains `key`, we return the original list as-is, and do not invoke your closure.
pub fn or_insert_with<F>(self, f: F) -> Option<I>
where
F: FnOnce() -> V,
{
let rest = match self.rest {
// If the list already contains `key`, we don't need to replace anything, and can
// return the original list unmodified.
ListTail::Occupied(_) => return self.list,
// Otherwise we have to create a new entry and stitch it onto the list.
ListTail::Beginning => None,
ListTail::Vacant(index) => Some(index),
};
self.stitch_up(rest, f())
}
/// Inserts a new key/value into the list if the key is not already present. If the list
/// already contains `key`, we return the original list as-is.
pub fn or_insert(self, value: V) -> Option<I> {
self.or_insert_with(|| value)
}
/// Inserts a new key and the default value into the list if the key is not already present. If
/// the list already contains `key`, we return the original list as-is.
pub fn or_insert_default(self) -> Option<I>
where
V: Default,
{
self.or_insert_with(V::default)
}
/// Ensures that the list contains an entry mapping the key to `value`, returning the resulting
/// list. Overwrites any existing entry with the same key. As an optimization, if the existing
/// entry has an equal _value_, as well, we return the original list as-is.
pub fn replace(self, value: V) -> Option<I>
where
V: Eq,
{
// If the list already contains `key`, skip past its entry before we add its replacement.
let rest = match self.rest {
ListTail::Beginning => None,
ListTail::Occupied(index) => {
let ListCell(rest, _, existing_value) = &self.builder.cells[index];
if value == *existing_value {
// As an optimization, if value isn't changed, there's no need to stitch up a
// new list.
return self.list;
}
*rest
}
ListTail::Vacant(index) => Some(index),
};
self.stitch_up(rest, value)
}
/// Ensures that the list contains an entry mapping the key to the default, returning the
/// resulting list. Overwrites any existing entry with the same key. As an optimization, if the
/// existing entry has an equal _value_, as well, we return the original list as-is.
pub fn replace_with_default(self) -> Option<I>
where
V: Default + Eq,
{
self.replace(V::default())
}
}
impl<I: Idx, K, V> ListBuilder<I, K, V> {
/// Returns the intersection of two lists. The result will contain an entry for any key that
/// appears in both lists. The corresponding values will be combined using the `combine`
/// function that you provide.
pub fn intersect_with<F>(
&mut self,
mut a: Option<I>,
mut b: Option<I>,
mut combine: F,
) -> Option<I>
where
K: Clone + Ord,
V: Clone,
F: FnMut(&V, &V) -> V,
{
self.scratch.clear();
// Zip through the lists, building up the keys/values of the new entries into our scratch
// vector. Continue until we run out of elements in either list. (Any remaining elements in
// the other list cannot possibly be in the intersection.)
while let (Some(a_id), Some(b_id)) = (a, b) {
let ListCell(a_rest, a_key, a_value) = &self.storage.cells[a_id];
let ListCell(b_rest, b_key, b_value) = &self.storage.cells[b_id];
match a_key.cmp(b_key) {
// Both lists contain this key; combine their values
Ordering::Equal => {
let new_key = a_key.clone();
let new_value = combine(a_value, b_value);
self.scratch.push((new_key, new_value));
a = *a_rest;
b = *b_rest;
}
// a's key is only present in a, so it's not included in the result.
Ordering::Greater => a = *a_rest,
// b's key is only present in b, so it's not included in the result.
Ordering::Less => b = *b_rest,
}
}
// Once the iteration loop terminates, we stitch the new entries back together into proper
// alist cells.
let mut result = None;
while let Some((key, value)) = self.scratch.pop() {
result = self.add_cell(result, key, value);
}
result
}
/// Returns the union of two lists. The result will contain an entry for any key that appears
/// in either list. For keys that appear in both lists, the corresponding values will be
/// combined using the `combine` function that you provide.
pub fn union_with<F>(&mut self, mut a: Option<I>, mut b: Option<I>, mut combine: F) -> Option<I>
where
K: Clone + Ord,
V: Clone,
F: FnMut(&V, &V) -> V,
{
self.scratch.clear();
// Zip through the lists, building up the keys/values of the new entries into our scratch
// vector. Continue until we run out of elements in either list. (Any remaining elements in
// the other list will be added to the result, but won't need to be combined with
// anything.)
let mut result = loop {
let (a_id, b_id) = match (a, b) {
// If we run out of elements in one of the lists, the non-empty list will appear in
// the output unchanged.
(None, other) | (other, None) => break other,
(Some(a_id), Some(b_id)) => (a_id, b_id),
};
let ListCell(a_rest, a_key, a_value) = &self.storage.cells[a_id];
let ListCell(b_rest, b_key, b_value) = &self.storage.cells[b_id];
match a_key.cmp(b_key) {
// Both lists contain this key; combine their values
Ordering::Equal => {
let new_key = a_key.clone();
let new_value = combine(a_value, b_value);
self.scratch.push((new_key, new_value));
a = *a_rest;
b = *b_rest;
}
// a's key goes into the result next
Ordering::Greater => {
let new_key = a_key.clone();
let new_value = a_value.clone();
self.scratch.push((new_key, new_value));
a = *a_rest;
}
// b's key goes into the result next
Ordering::Less => {
let new_key = b_key.clone();
let new_value = b_value.clone();
self.scratch.push((new_key, new_value));
b = *b_rest;
}
}
};
// Once the iteration loop terminates, we stitch the new entries back together into proper
// alist cells.
while let Some((key, value)) = self.scratch.pop() {
result = self.add_cell(result, key, value);
}
result
}
}
// ----
// Sets
impl<I: Idx, K> ListStorage<I, K, ()> {
/// Iterates through the elements in a set _in reverse order_.
pub fn iter_set_reverse(&self, set: Option<I>) -> ListSetReverseIterator<'_, I, K> {
ListSetReverseIterator {
storage: self,
curr: set,
}
}
}
pub struct ListSetReverseIterator<'a, I, K> {
storage: &'a ListStorage<I, K, ()>,
curr: Option<I>,
}
impl<'a, I: Idx, K> Iterator for ListSetReverseIterator<'a, I, K> {
type Item = &'a K;
fn next(&mut self) -> Option<Self::Item> {
let ListCell(rest, key, ()) = &self.storage.cells[self.curr?];
self.curr = *rest;
Some(key)
}
}
impl<I: Idx, K> ListBuilder<I, K, ()> {
/// Adds an element to a set.
pub fn insert(&mut self, set: Option<I>, element: K) -> Option<I>
where
K: Clone + Ord,
{
self.entry(set, element).or_insert_default()
}
/// Returns the intersection of two sets. The result will contain any value that appears in
/// both sets.
pub fn intersect(&mut self, a: Option<I>, b: Option<I>) -> Option<I>
where
K: Clone + Ord,
{
self.intersect_with(a, b, |(), ()| ())
}
/// Returns the intersection of two sets. The result will contain any value that appears in
/// either set.
pub fn union(&mut self, a: Option<I>, b: Option<I>) -> Option<I>
where
K: Clone + Ord,
{
self.union_with(a, b, |(), ()| ())
}
}
// -----
// Tests
#[cfg(test)]
mod tests {
use super::*;
use std::fmt::Display;
use std::fmt::Write;
use crate::newtype_index;
// Allows the macro invocation below to work
use crate as ruff_index;
#[newtype_index]
struct TestIndex;
// ----
// Sets
impl<I, K> ListStorage<I, K>
where
I: Idx,
K: Display,
{
fn display_set(&self, list: Option<I>) -> String {
let elements: Vec<_> = self.iter_set_reverse(list).collect();
let mut result = String::new();
result.push('[');
for element in elements.into_iter().rev() {
if result.len() > 1 {
result.push_str(", ");
}
write!(&mut result, "{element}").unwrap();
}
result.push(']');
result
}
}
#[test]
fn can_insert_into_set() {
let mut builder = ListBuilder::<TestIndex, u16>::default();
// Build up the set in order
let set1 = builder.insert(None, 1);
let set12 = builder.insert(set1, 2);
let set123 = builder.insert(set12, 3);
let set1232 = builder.insert(set123, 2);
assert_eq!(builder.display_set(None), "[]");
assert_eq!(builder.display_set(set1), "[1]");
assert_eq!(builder.display_set(set12), "[1, 2]");
assert_eq!(builder.display_set(set123), "[1, 2, 3]");
assert_eq!(builder.display_set(set1232), "[1, 2, 3]");
// And in reverse order
let set3 = builder.insert(None, 3);
let set32 = builder.insert(set3, 2);
let set321 = builder.insert(set32, 1);
let set3212 = builder.insert(set321, 2);
assert_eq!(builder.display_set(None), "[]");
assert_eq!(builder.display_set(set3), "[3]");
assert_eq!(builder.display_set(set32), "[2, 3]");
assert_eq!(builder.display_set(set321), "[1, 2, 3]");
assert_eq!(builder.display_set(set3212), "[1, 2, 3]");
}
#[test]
fn can_intersect_sets() {
let mut builder = ListBuilder::<TestIndex, u16>::default();
let set1 = builder.entry(None, 1).or_insert_default();
let set12 = builder.entry(set1, 2).or_insert_default();
let set123 = builder.entry(set12, 3).or_insert_default();
let set1234 = builder.entry(set123, 4).or_insert_default();
let set2 = builder.entry(None, 2).or_insert_default();
let set24 = builder.entry(set2, 4).or_insert_default();
let set245 = builder.entry(set24, 5).or_insert_default();
let set2457 = builder.entry(set245, 7).or_insert_default();
let intersection = builder.intersect(None, None);
assert_eq!(builder.display_set(intersection), "[]");
let intersection = builder.intersect(None, set1234);
assert_eq!(builder.display_set(intersection), "[]");
let intersection = builder.intersect(None, set2457);
assert_eq!(builder.display_set(intersection), "[]");
let intersection = builder.intersect(set1, set1234);
assert_eq!(builder.display_set(intersection), "[1]");
let intersection = builder.intersect(set1, set2457);
assert_eq!(builder.display_set(intersection), "[]");
let intersection = builder.intersect(set2, set1234);
assert_eq!(builder.display_set(intersection), "[2]");
let intersection = builder.intersect(set2, set2457);
assert_eq!(builder.display_set(intersection), "[2]");
let intersection = builder.intersect(set1234, set2457);
assert_eq!(builder.display_set(intersection), "[2, 4]");
}
#[test]
fn can_union_sets() {
let mut builder = ListBuilder::<TestIndex, u16>::default();
let set1 = builder.entry(None, 1).or_insert_default();
let set12 = builder.entry(set1, 2).or_insert_default();
let set123 = builder.entry(set12, 3).or_insert_default();
let set1234 = builder.entry(set123, 4).or_insert_default();
let set2 = builder.entry(None, 2).or_insert_default();
let set24 = builder.entry(set2, 4).or_insert_default();
let set245 = builder.entry(set24, 5).or_insert_default();
let set2457 = builder.entry(set245, 7).or_insert_default();
let union = builder.union(None, None);
assert_eq!(builder.display_set(union), "[]");
let union = builder.union(None, set1234);
assert_eq!(builder.display_set(union), "[1, 2, 3, 4]");
let union = builder.union(None, set2457);
assert_eq!(builder.display_set(union), "[2, 4, 5, 7]");
let union = builder.union(set1, set1234);
assert_eq!(builder.display_set(union), "[1, 2, 3, 4]");
let union = builder.union(set1, set2457);
assert_eq!(builder.display_set(union), "[1, 2, 4, 5, 7]");
let union = builder.union(set2, set1234);
assert_eq!(builder.display_set(union), "[1, 2, 3, 4]");
let union = builder.union(set2, set2457);
assert_eq!(builder.display_set(union), "[2, 4, 5, 7]");
let union = builder.union(set1234, set2457);
assert_eq!(builder.display_set(union), "[1, 2, 3, 4, 5, 7]");
}
// ----
// Maps
impl<I, K, V> ListStorage<I, K, V>
where
I: Idx,
K: Display,
V: Display,
{
fn display(&self, list: Option<I>) -> String {
let entries: Vec<_> = self.iter_reverse(list).collect();
let mut result = String::new();
result.push('[');
for (key, value) in entries.into_iter().rev() {
if result.len() > 1 {
result.push_str(", ");
}
write!(&mut result, "{key}:{value}").unwrap();
}
result.push(']');
result
}
}
#[test]
fn can_insert_into_map() {
let mut builder = ListBuilder::<TestIndex, u16, u16>::default();
// Build up the map in order
let map1 = builder.entry(None, 1).replace(1);
let map12 = builder.entry(map1, 2).replace(2);
let map123 = builder.entry(map12, 3).replace(3);
let map1232 = builder.entry(map123, 2).replace(4);
assert_eq!(builder.display(None), "[]");
assert_eq!(builder.display(map1), "[1:1]");
assert_eq!(builder.display(map12), "[1:1, 2:2]");
assert_eq!(builder.display(map123), "[1:1, 2:2, 3:3]");
assert_eq!(builder.display(map1232), "[1:1, 2:4, 3:3]");
// And in reverse order
let map3 = builder.entry(None, 3).replace(3);
let map32 = builder.entry(map3, 2).replace(2);
let map321 = builder.entry(map32, 1).replace(1);
let map3212 = builder.entry(map321, 2).replace(4);
assert_eq!(builder.display(None), "[]");
assert_eq!(builder.display(map3), "[3:3]");
assert_eq!(builder.display(map32), "[2:2, 3:3]");
assert_eq!(builder.display(map321), "[1:1, 2:2, 3:3]");
assert_eq!(builder.display(map3212), "[1:1, 2:4, 3:3]");
}
#[test]
fn can_insert_if_needed_into_map() {
let mut builder = ListBuilder::<TestIndex, u16, u16>::default();
// Build up the map in order
let map1 = builder.entry(None, 1).or_insert(1);
let map12 = builder.entry(map1, 2).or_insert(2);
let map123 = builder.entry(map12, 3).or_insert(3);
let map1232 = builder.entry(map123, 2).or_insert(4);
assert_eq!(builder.display(None), "[]");
assert_eq!(builder.display(map1), "[1:1]");
assert_eq!(builder.display(map12), "[1:1, 2:2]");
assert_eq!(builder.display(map123), "[1:1, 2:2, 3:3]");
assert_eq!(builder.display(map1232), "[1:1, 2:2, 3:3]");
// And in reverse order
let map3 = builder.entry(None, 3).or_insert(3);
let map32 = builder.entry(map3, 2).or_insert(2);
let map321 = builder.entry(map32, 1).or_insert(1);
let map3212 = builder.entry(map321, 2).or_insert(4);
assert_eq!(builder.display(None), "[]");
assert_eq!(builder.display(map3), "[3:3]");
assert_eq!(builder.display(map32), "[2:2, 3:3]");
assert_eq!(builder.display(map321), "[1:1, 2:2, 3:3]");
assert_eq!(builder.display(map3212), "[1:1, 2:2, 3:3]");
}
#[test]
fn can_intersect_maps() {
let mut builder = ListBuilder::<TestIndex, u16, u16>::default();
let map1 = builder.entry(None, 1).or_insert(1);
let map12 = builder.entry(map1, 2).or_insert(2);
let map123 = builder.entry(map12, 3).or_insert(3);
let map1234 = builder.entry(map123, 4).or_insert(4);
let map2 = builder.entry(None, 2).or_insert(20);
let map24 = builder.entry(map2, 4).or_insert(40);
let map245 = builder.entry(map24, 5).or_insert(50);
let map2457 = builder.entry(map245, 7).or_insert(70);
let intersection = builder.intersect_with(None, None, |a, b| a + b);
assert_eq!(builder.display(intersection), "[]");
let intersection = builder.intersect_with(None, map1234, |a, b| a + b);
assert_eq!(builder.display(intersection), "[]");
let intersection = builder.intersect_with(None, map2457, |a, b| a + b);
assert_eq!(builder.display(intersection), "[]");
let intersection = builder.intersect_with(map1, map1234, |a, b| a + b);
assert_eq!(builder.display(intersection), "[1:2]");
let intersection = builder.intersect_with(map1, map2457, |a, b| a + b);
assert_eq!(builder.display(intersection), "[]");
let intersection = builder.intersect_with(map2, map1234, |a, b| a + b);
assert_eq!(builder.display(intersection), "[2:22]");
let intersection = builder.intersect_with(map2, map2457, |a, b| a + b);
assert_eq!(builder.display(intersection), "[2:40]");
let intersection = builder.intersect_with(map1234, map2457, |a, b| a + b);
assert_eq!(builder.display(intersection), "[2:22, 4:44]");
}
#[test]
fn can_union_maps() {
let mut builder = ListBuilder::<TestIndex, u16, u16>::default();
let map1 = builder.entry(None, 1).or_insert(1);
let map12 = builder.entry(map1, 2).or_insert(2);
let map123 = builder.entry(map12, 3).or_insert(3);
let map1234 = builder.entry(map123, 4).or_insert(4);
let map2 = builder.entry(None, 2).or_insert(20);
let map24 = builder.entry(map2, 4).or_insert(40);
let map245 = builder.entry(map24, 5).or_insert(50);
let map2457 = builder.entry(map245, 7).or_insert(70);
let union = builder.union_with(None, None, |a, b| a + b);
assert_eq!(builder.display(union), "[]");
let union = builder.union_with(None, map1234, |a, b| a + b);
assert_eq!(builder.display(union), "[1:1, 2:2, 3:3, 4:4]");
let union = builder.union_with(None, map2457, |a, b| a + b);
assert_eq!(builder.display(union), "[2:20, 4:40, 5:50, 7:70]");
let union = builder.union_with(map1, map1234, |a, b| a + b);
assert_eq!(builder.display(union), "[1:2, 2:2, 3:3, 4:4]");
let union = builder.union_with(map1, map2457, |a, b| a + b);
assert_eq!(builder.display(union), "[1:1, 2:20, 4:40, 5:50, 7:70]");
let union = builder.union_with(map2, map1234, |a, b| a + b);
assert_eq!(builder.display(union), "[1:1, 2:22, 3:3, 4:4]");
let union = builder.union_with(map2, map2457, |a, b| a + b);
assert_eq!(builder.display(union), "[2:40, 4:40, 5:50, 7:70]");
let union = builder.union_with(map1234, map2457, |a, b| a + b);
assert_eq!(builder.display(union), "[1:1, 2:22, 3:3, 4:44, 5:50, 7:70]");
}
}