Instead of simple numbering, we hash important bits, like the name of the item.
This will allow for much better incrementality, e.g. when you add an item. Currently, this invalidates the IDs of all following items, which invalidates pretty much everything.
Now that they're const it's no longer needed.
Nothing manual was performed: only a regexp search of `sym::([\w][\w\d]*)\.clone\(\)` and replace by `sym::$1`.
More correctly, *also* per-token. Because as it turns out, while the top-level edition affects parsing (I think), the per-token edition affects escaping of identifiers/keywords.
I expected this to be faster (due to less allocations and better cache locality), but benchmarked it is not (neither it is slower). Memory usage, however, drops by ~50mb (of `analysis-stats .`). I guess tt construction is just not hot.
This also simplifies using even less memory for token trees by compressing equal span, which I plan to do right after.
Some workflows are more easily expressed with a flat tt, while some are better expressed with a tree. With the right helpers, though (which was mostly a matter of trial and error), even the worst workflows become very easy indeed.
