# Testing

RCL is tested extensively. The two primary means of testing are golden tests
and fuzzing.

## Golden tests

Golden tests are located in the `golden` directory as files ending in `.test`.
Every file consists of two sections: an input, then a line

    # output:

and then an expected output. The test runner `golden/run.py` takes these files,
executes <abbr>RCL</abbr>, and compares the actual output against the expected
output. The mode in which `run.py` executes <abbr>RCL</abbr> depends on the
subdirectory that the tests are in. See the docstring in `run.py` for more
information.

The goal of the golden tests is to cover all relevant branches of the code. For
example, every error message that <abbr>RCL</abbr> can generate should be
accompanied by a test that triggers it. It is instructive to inspect the
coverage report to see what tests are missing, or to discover pieces of dead
code. The easiest way to generate the report is with Nix:

    nix build .#coverage --out-link result
    xdg-open result/index.html

Alternatively, you can build with coverage support and run `grcov` manually,
see `flake.nix` for the exact flags and commands.

## Fuzz tests

The other means of testing are fuzz tests. RCL contains one primary fuzzer
called `main`, whose fuzz inputs are valid <abbr>RCL</abbr> files that contain a
little bit of metadata about what to exercise. Putting everything in one fuzzer
enables sharing the corpus across the different modes.

The basic modes of the fuzzer just test that the lexer, parser, and evaluator
do not crash. They are useful for finding mistakes such as slicing a string
across a code point binary, or trying to consume tokens past the end of the
document. However the more interesting fuzzers are the formatter, and the value
pretty-printer. They verify the following properties:

 * The formatter is idempotent. Running `rcl fmt` on an already-formatted file
   should not change it. This sounds obvious, but the fuzzer caught a few
   interesting cases related to trailing whitespace and multiline strings.
 * The evaluator is idempotent. RCL can evaluate to json, and is itself a
   superset of json, so evaluating the same input again should not change it.

## Unit tests

There is less of a focus on unit tests. Constructing all the right environment
and values for even a very basic test quickly becomes unwieldy; even the
<abbr>AST</abbr> of a few lines of code, when constructed directly in Rust,
becomes hundreds of lines of code. Rather than constructing the <abbr>AST</abbr>
by hand, we can just use the parser to construct one. Similarly, for expected
output values, instead of constructing these in Rust, we can format them, and
express the entire process as a golden test instead.