If an entry in the layout cache contains recursive structures, the entry
is not reusable if the recursive structure is currently in the "seen"
set. The example elucidated in the source code is as follows:
Suppose we are constructing the layout of
```
[A, B (List r)] as r
```
and we have already constructed and cached the layout of `List r`, which would
be
```
List (Recursive [Unit, List RecursivePointer])
```
If we use the cached entry of `List r`, we would end up with the layout
```
Recursive [Unit, (List (Recursive [Unit, List RecursivePointer]))]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cached layout for `List r`
```
but this is not correct; the canonical layout of `[A, B (List r)] as r` is
```
Recursive [Unit, (List RecursivePointer)]
```
However, the current implementation only preserves this behavior for
structures that contain one recursive structure under them. In practice,
there can be structures that contain multiple recursive structures under
them, and we must be sure to record all those structures in the
layout-cache.
If a lambda set is non-recursive, but contains naked recursion pointers,
we should not fill those naked pointers in with the slot of the lambda
set during interning. Such naked pointers must belong to an encompassing
lambda set that is in fact recursive, and will be filled in later.
For example, `LambdaSet([Foo, LambdaSet(Bar, [<rec>])] as <rec>)` should
not have the inner lambda set's capture be filled in with itself.
Also, during reification of recursion pointers, we do not need to
traverse re-inserted lambda sets again, since they were just fixed-up.
Closes#5026
Adding a cache layer can only be done with a snapshot and rollback.
This is necessary to prevent extra variables just lying around on the
toplevel of the layout cache.
After unification, variable roots can change. So, when we invalidate
entries in the layout cache, we must compare for variable equivalence
relative to the current state of subs.
