Commit graph

112 commits

Author SHA1 Message Date
Shunsuke Shibayama
cdd0b85516
Expose the Input query Id with cycle_initial (#1015)
Some checks failed
Release-plz / Release-plz PR (push) Has been cancelled
Release-plz / Release-plz release (push) Has been cancelled
Book / Book (push) Has been cancelled
Test / Test (push) Has been cancelled
Test / Miri (push) Has been cancelled
Test / Shuttle (push) Has been cancelled
Test / Benchmarks (push) Has been cancelled
Book / Deploy (push) Has been cancelled
2025-10-27 12:27:09 +00:00
Micha Reiser
d38145c295
Expose the query ID and the last provisional value to the cycle recovery function (#1012)
Some checks failed
Test / Shuttle (push) Has been cancelled
Release-plz / Release-plz release (push) Has been cancelled
Test / Test (push) Has been cancelled
Test / Miri (push) Has been cancelled
Book / Book (push) Has been cancelled
Release-plz / Release-plz PR (push) Has been cancelled
Test / Benchmarks (push) Has been cancelled
Book / Deploy (push) Has been cancelled
* Expose the query ID and the last provisional value to the cycle recovery function

* Mark cycle as converged if fallback value is the same as the last provisional

* Make `cycle_fn` optional
2025-10-23 06:36:56 +00:00
Micha Reiser
16d51d63d5
Fix hangs in multithreaded fixpoint iteration (#1010)
* Fix race condition between releasing a transferred query's lock and the
same query blocking on the outer head in `provisional_retry`

* Fix infinite loop in `provisional_retry
2025-10-23 06:15:01 +00:00
Micha Reiser
ffa811dca2
Remove experimental parallel feature (#1013)
Some checks are pending
Book / Deploy (push) Blocked by required conditions
Release-plz / Release-plz release (push) Waiting to run
Book / Book (push) Waiting to run
Release-plz / Release-plz PR (push) Waiting to run
Test / Test (push) Waiting to run
Test / Miri (push) Waiting to run
Test / Shuttle (push) Waiting to run
Test / Benchmarks (push) Waiting to run
2025-10-22 11:41:56 +00:00
Micha Reiser
ef9f9329be
Run fixpoint per strongly connected component (#999)
Some checks failed
Book / Book (push) Has been cancelled
Release-plz / Release-plz release (push) Has been cancelled
Release-plz / Release-plz PR (push) Has been cancelled
Test / Test (push) Has been cancelled
Test / Miri (push) Has been cancelled
Test / Shuttle (push) Has been cancelled
Test / Benchmarks (push) Has been cancelled
Book / Deploy (push) Has been cancelled
* Run nested cycles in a single fixpoint iteration

Fix serde attribute

* Remove inline from `validate_same_iteration`

* Nits

* Move locking into sync table

* More trying

* More in progress work

* More progress

* Fix most parallel tests

* More bugfixes

* Short circuit in some cases

* Short circuit in drop

* Delete some unused code

* A working solution

* Simplify more

* Avoid repeated query lookups in `transfer_lock`

* Use recursion for unblocking

* Fix hang in `maybe_changed_after`

* Move claiming of transferred memos into a separate function

* More aggressive use of attributes

* Make re-entrant a const parameter

* Smaller clean-ups

* Only collect cycle heads one level deep

* More cleanups

* More docs

* More comments

* More documentation, cleanups

* More documentation, cleanups

* Remove inline attribute

* Fix failing tracked structs test

* Fix panic

* Fix persistence test

* Add test for panic in nested cycle

* Allow cycle initial values same-stack

* Try inlining fetch

* Remove some inline attributes

* Add safety comment

* Clippy

* Panic if `provisional_retry` runs too many times

* Better handling of panics in cycles

* Don't use const-generic for `REENTRANT`

* More nit improvements

* Remove `IterationCount::panicked`

* Prefer outer most cycles in `outer_cycle`

* Code review feedback

* Iterate only once in panic test when running with miri
2025-10-16 09:23:03 +00:00
Micha Reiser
22a4d9932b
test: add parallel maybe changed after test (#963)
* Add parallel `maybe_changed_after` shuttle test

* Update tests/parallel/cycle_nested_deep_conditional_changed.rs

Co-authored-by: Carl Meyer <carl@astral.sh>

---------

Co-authored-by: Carl Meyer <carl@astral.sh>
2025-08-08 08:32:37 +00:00
Ibraheem Ahmed
dba66f1a37
Use inventory for static ingredient registration (#934)
Some checks failed
Book / Book (push) Has been cancelled
Release-plz / Release-plz release (push) Has been cancelled
Release-plz / Release-plz PR (push) Has been cancelled
Test / Test (push) Has been cancelled
Test / Miri (push) Has been cancelled
Test / Shuttle (push) Has been cancelled
Test / Benchmarks (push) Has been cancelled
Book / Deploy (push) Has been cancelled
* use `inventory` for static ingredient registration

* remove unnecessary synchronization from memo tables

* use global ingredient caches for database-independent ingredients

* add manual ingredient registration API

* remove static ingredient index optimization when manual registration is in use

* fix atomic imports

* simplify ingredient caches
2025-07-18 04:55:50 +00:00
Lukas Wirth
2c57c5628b
Fix flaky parallel_join test (#900)
Some checks are pending
Book / Book (push) Waiting to run
Book / Deploy (push) Blocked by required conditions
Release-plz / Release-plz release (push) Waiting to run
Release-plz / Release-plz PR (push) Waiting to run
Test / Test (push) Waiting to run
Test / Miri (push) Waiting to run
Test / Shuttle (push) Waiting to run
Test / Benchmarks (push) Waiting to run
2025-06-04 12:24:15 +00:00
Micha Reiser
2b5188778e
fix: multithreaded nested fixpoint iteration (#882)
Some checks are pending
Test / Test (push) Waiting to run
Test / Miri (push) Waiting to run
Test / Shuttle (push) Waiting to run
Test / Benchmarks (push) Waiting to run
Book / Book (push) Waiting to run
Book / Deploy (push) Blocked by required conditions
Release-plz / Release-plz release (push) Waiting to run
Release-plz / Release-plz PR (push) Waiting to run
* Set `validate_final` in `execute` after removing the last cycle head

* Add runaway query repro

* Add tracing

* Fix part 1

* Fix `cycle_head_kinds` to always return provisional for memos that aren't verified final (They should be validated by `validate_same_iteration` or wait for the cycle head

* Fix cycle error

* Documentation

* Fix await for queries depending on initial value

* correctly initialize queued

* Cleanup

* Short circuit if entire query runs on single thread

* Move parallel code into its own method

* Rename method, add self_key to queued

* Revert self-key changes

* Move check *after* `deep_verify_memo`

* Add a test for a cycle with changing cycle heads

* Short circuit more often

* Consider iteration in `validate_provisional`

* Only yield if all heads result in a cycle. Retry if even just one inner cycle made progress (in which case there's a probably a new memo)

* Fix hangs

* Cargo fmt

* clippy

* Fix hang if cycle initial panics

* Rename `cycle_head_kind` enable `cycle_a_t1_b_t2_fallback` shuttle test

* Cleanup

* Docs
2025-06-01 08:45:37 +00:00
Micha Reiser
e1fe3698ce
Fix hang in nested fixpoint iteration (#871) 2025-05-27 17:06:34 +00:00
Ibraheem Ahmed
0414d89327
Replace loom with shuttle (#876)
* replace loom with shuttle

* inline `empty_cycle_heads`

* ignore failing shuttle test
2025-05-23 15:28:51 +00:00
Lukas Wirth
cfa88e9a4d
Move salsa event system into Zalsa (#849)
* Move salsa event system into `Zalsa`

* Encode `None` into ShallowUpdate

This shrinks the return value from 16 to 8 bytes
2025-05-07 11:45:49 +00:00
Chayim Refael Friedman
05b4faddb5
Add a third cycle mode, equivalent to old Salsa cycle behavior (#801)
That is, directly set a value for all queries that have fallbacks, and ignore all other queries in the cycle.

Unlike old Salsa, we still need all cycle heads to be marked, and we still execute the queries to completion, but we throw their result.
2025-04-22 10:35:43 +00:00
Lukas Wirth
3820cac9a4
Remove incorrect parallel_scope API (#797)
Rayon does not allow joining on the spawns, meaning we might end up with results in the tests before reading out the panics
2025-04-16 14:09:35 +00:00
Lukas Wirth
a89e3d2357
chore: Normalize imports style (#779)
* Default impl some ingredient functions

* chore: Normalize imports style

Effectively reformatted everything with
```toml
imports_granularity = "Module"
group_imports = "StdExternalCrate"
reorder_imports = true
```
2025-03-28 05:11:40 +00:00
Lukas Wirth
5ee3bdde58
refactor: Clean up par_map a bit (#742)
* Adjust safety argument of `par_map`

* More parallel APIs

* assert `ParallelDb` `Send` promise

* fix: Fix `par_map` unsoundness

* Add more parallel API tests
2025-03-24 19:45:12 +00:00
Lukas Wirth
1bbf4c2be5
feat: Drop Debug requirements and flip implementation defaults (#756) 2025-03-15 15:14:34 +00:00
Lukas Wirth
1dc24388d1
chore: Use nextest for miri test runs (#758) 2025-03-15 10:30:23 +00:00
Carl Meyer
095d8b2b81
rewrite cycle handling to support fixed-point iteration (#603) 2025-03-10 22:25:47 +00:00
Lukas Wirth
a8b659d1c8 Tidy up Cargo.toml dependencies 2025-02-22 10:05:08 +01:00
Bruce Mitchener
511aabac97 Reduce typo count 2025-02-20 12:06:12 +07:00
Lukas Wirth
3468e8ea62 Drop unnecssary usages of AtomicCell 2025-02-11 11:51:13 +01:00
Lukas Wirth
87fdc13d4f Fix parallel_map::execute_cancellation test 2024-12-23 17:11:46 +01:00
David Barsky
db7274ecb0 internal: ignore parallel tests under miri 2024-11-25 11:46:03 -05:00
David Barsky
5f0904ae4a introduce parallel salsa 2024-11-25 11:29:24 -05:00
Niko Matsakis
a27befeeba run tests with UPDATE_EXPECT=1 2024-08-20 10:09:49 -04:00
Niko Matsakis
4d2ccffddc return to the database-wrapping-storage setup 2024-08-04 02:18:54 -04:00
Niko Matsakis
138ca4b1f3 merge handle into the database
Separate handles are no longer needed.
2024-07-28 12:47:50 +00:00
Niko Matsakis
daaa78056a switch to new database design
Under this design, *all* databases are a
`DatabaseImpl<U>`, where the `U` implements
`UserData` (you can use `()` if there is none).

Code would default to `&dyn salsa::Database` but
if you want to give access to the userdata, you
can define a custom database trait
`MyDatabase: salsa::Databse` so long as you

* annotate `MyDatabase` trait definition of
  impls of `MyDatabase` with `#[salsa::db]`
* implement `MyDatabase` for `DatabaseImpl<U>`
  where `U` is your userdata (this could be a
  blanket impl, if you don't know the precise
  userdata type).

The `tests/common/mod.rs` shows the pattern.
2024-07-28 12:47:50 +00:00
Niko Matsakis
64556e9d28 make event generation lazy
Creating events if nobody is listening has
always bugged me.
2024-07-28 12:35:33 +00:00
Niko Matsakis
8a3cc6e404 add a test for handle cancellation
I realized there weren't any!
2024-07-24 10:36:07 +00:00
Niko Matsakis
470c5b3d78 restore parallel_cycle_one_recover 2024-07-24 09:54:30 +00:00
Niko Matsakis
ac474a9c8d restore parallel_cycle_mid_recover 2024-07-24 09:53:24 +00:00
Niko Matsakis
82d37de105 start restoring parallel tests 2024-07-24 09:46:19 +00:00
Niko Matsakis
92eee06596 delete a bunch of dead code 2024-07-19 05:55:40 -04:00
Niko Matsakis
b7e6bc62af make methods work 2024-07-18 06:03:34 -04:00
Niko Matsakis
043922d34f wip 2024-07-17 09:18:43 -04:00
Niko Matsakis
8a39bf029b wip 2024-07-16 06:04:01 -04:00
Matthijs Brobbel
c7851112a5
Rename salsa-2022 to salsa 2024-06-18 09:40:21 +02:00
Matthijs Brobbel
db75ae00a2
Remove old salsa 2024-06-18 09:25:26 +02:00
Niko Matsakis
afd9b72203 pacify the merciless clippy 2024-04-02 06:51:39 -04:00
andrzej.gluszak
73102b1e8e Fix clippy issues 2022-08-24 18:45:53 +02:00
Bernardo Uriarte
927d905e58 test-env-log has been renamed to test-log 2022-08-22 11:00:47 +02:00
Niko Matsakis
1e3c2f22aa Expose the ability to remove the value from an input query, taking ownership of it
Co-authored-by: Tim Robinson <tim.g.robinson@gmail.com>
2022-06-03 05:52:00 -04:00
Niko Matsakis
356392578b new parallel friendly algorithm 2022-01-21 13:58:13 -05:00
Niko Matsakis
cb658b9b89 enable partial recovery across threads
Including the corner case where the active thread does not have
recovery.
2021-11-11 09:07:45 -05:00
Niko Matsakis
bfa74bc865 spread parallel tests into their own files
I was finding the parallel test setup hard to read,
so everything relating to one test is now in a single
file, with shorter names.
2021-11-11 06:54:52 -05:00
Niko Matsakis
45434cfa93 rework cycle to permit partial recovery (wip)
This is "wip" because it does not yet handle cross-thread recovery
correctly. That is coming in a later commit.
2021-11-11 06:54:32 -05:00
Niko Matsakis
33d47cc747 throw Cycle value directly
Do not wrap in Cancelled.
2021-11-02 12:45:42 -04:00
Niko Matsakis
75ee3edd2e introduce Cycle type and use in recovery, errors
The Cycle type gives more structured information and ensures
deterministic ordering of participants within any particular
execution.
2021-10-31 07:21:07 -04:00