Instead of simple numbering, we hash important bits, like the name of the item.
This will allow for much better incrementality, e.g. when you add an item. Currently, this invalidates the IDs of all following items, which invalidates pretty much everything.
More correctly, *also* per-token. Because as it turns out, while the top-level edition affects parsing (I think), the per-token edition affects escaping of identifiers/keywords.
I expected this to be faster (due to less allocations and better cache locality), but benchmarked it is not (neither it is slower). Memory usage, however, drops by ~50mb (of `analysis-stats .`). I guess tt construction is just not hot.
This also simplifies using even less memory for token trees by compressing equal span, which I plan to do right after.
Some workflows are more easily expressed with a flat tt, while some are better expressed with a tree. With the right helpers, though (which was mostly a matter of trial and error), even the worst workflows become very easy indeed.
proc-macro ABI breakage still affects the tests when a new stable version
releases. Ideally we'd still be able to run the tests on the rust-analyzer
repo without having to update the proc-macro ABI, but for now just to
unblock CI we will ignore them here, as they are still run in upstream.
