If an entry in the layout cache contains recursive structures, the entry
is not reusable if the recursive structure is currently in the "seen"
set. The example elucidated in the source code is as follows:
Suppose we are constructing the layout of
```
[A, B (List r)] as r
```
and we have already constructed and cached the layout of `List r`, which would
be
```
List (Recursive [Unit, List RecursivePointer])
```
If we use the cached entry of `List r`, we would end up with the layout
```
Recursive [Unit, (List (Recursive [Unit, List RecursivePointer]))]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ cached layout for `List r`
```
but this is not correct; the canonical layout of `[A, B (List r)] as r` is
```
Recursive [Unit, (List RecursivePointer)]
```
However, the current implementation only preserves this behavior for
structures that contain one recursive structure under them. In practice,
there can be structures that contain multiple recursive structures under
them, and we must be sure to record all those structures in the
layout-cache.
Suppose we have a when expression
```
15 if foo -> <b1>
b if bar -> <b2>
_ -> <b3>
```
that may have a decision tree like
```
15?
\true => foo?
\true => <b1>
\false => bar?
\true => <b2>
\false => <b3>
\false => bar?
\true => <b2>
\false => <b3>
```
In this case, the guard "bar?" appears twice in the compiled decision
tree. We need to materialize the guard expression in both locations in
the compiled tree, which means we cannot as-is stamp a compiled `bar?`
twice in each location. The reason is that
- the compiled joinpoint for each `bar?` guard needs to have a unique ID
- the guard expression might have call which needs unique call spec IDs,
or other joins that need unique joinpoint IDs.
So, save the expression as we build up the decision tree and materialize
the guard each time we need it. In practice the guards should be quite
small, so duplicating should be fine. We could avoid duplication, but
it's not clear to me how to do that exactly since the branches after the
guard might end up being different.
If a lambda set is non-recursive, but contains naked recursion pointers,
we should not fill those naked pointers in with the slot of the lambda
set during interning. Such naked pointers must belong to an encompassing
lambda set that is in fact recursive, and will be filled in later.
For example, `LambdaSet([Foo, LambdaSet(Bar, [<rec>])] as <rec>)` should
not have the inner lambda set's capture be filled in with itself.
Also, during reification of recursion pointers, we do not need to
traverse re-inserted lambda sets again, since they were just fixed-up.
Closes#5026
