rcl/docs/generating_files.md

Generating files

RCL can abstract away repetition, for example in GitHub Actions workflows and Kubernetes manifests. It also enables sharing configuration between systems that do not natively share data. For example, you can have one file to define users and groups, and import it into both your Tailscale configuration and into your Hashicorp Vault configuration. However, none of these tools natively read RCL. You still need to run rcl to generate the required .yml, .tf.json, .json, and .toml files that can be consumed by your existing tools.

If you have just a few files, it’s not so bad to run rcl yourself:

rcl evaluate --format=json policies.rcl --output=policies.json

In a larger repository with many generated files, this gets tedious. It’s not very discoverable either. Ideally we would have one command to update all generated files. This is typically the job of a build system. RCL offers two solutions here:

• The built-in rcl build command acts as a simple build system. It has limitations, but avoids the need to bring in more tools, and it suffices for many use cases.
• For more advanced use cases, RCL can integrate with external build systems through e.g. depfile support. See the Ninja chapter for an example of how to integrate with the Ninja build system.

In this chapter we will look at the first case, rcl build.

Using rcl build

To start using rcl build, create a file named build.rcl:

{
  "output.txt": {
    format = "raw",
    contents = "Hello, world.",
  }
}

Then, in the same directory, run rcl build. This will create the file output.txt next to build.rcl, with the following contents:

Hello, world.

The format corresponds to one of the output formats supported by rcl evaluate, and the contents can be an arbitrary value. For example, we can output as JSON:

{
  "output.json": {
    format = "json",
    contents = {
      is-example = true,
      name = "rcl-build demo",
    }
  }
}

This would create output.json with the following contents:

{"is-example": true, "name": "rcl-build demo"}

Adding a banner

The files generated by RCL may be used in places where it is not obvious that they are generated. To clarify that a file is generated, we can add a banner. This is a short message that gets prepended to the output. For example:

{
  "Cargo.toml": {
    format = "toml",
    banner = "# This file is generated from build.rcl.",
    contents = { package = { name = "rcl", edition = "2021" } },
  }
}

Running rcl build then creates the following Cargo.toml:

# This file is generated from build.rcl.
[package]
edition = "2021"
name = "rcl"

Inside build.rcl, the banner message is enclosed as a string literal, which makes it clear that the message refers to the generated file and not to build.rcl itself. This is unlike e.g. standalone Jinja template files, where if you want the banner in the output file, putting it in the input template can create confusion about whether the comment refers to the template or to the rendered output.

Loading external data

The build.rcl file is a regular RCL document, which means it can import other documents. For example, say we have users.rcl with the following contents:

[
  {
    uid = 0,
    name = "Eldon Tyrell",
    email = "eldon@tyrell.com",
  },
  {
    uid = 7,
    name = "Rachael Tyrell",
    email = "rachael@tyrell.com",
  },
]

Then we can convert that to YAML using the following build.rcl:

{
  "users.yaml": {
    // JSON documents are valid YAML.
    format = "json",
    banner = "# This file is generated from users.rcl.",
    contents = { users = import "users.rcl" },
  },
}

Which produces the following users.yaml:

# This file is generated from users.rcl.
{
  "users": [
    {"email": "eldon@tyrell.com", "name": "Eldon Tyrell", "uid": 0},
    {"email": "rachael@tyrell.com", "name": "Rachael Tyrell", "uid": 7}
  ]
}

Dynamic build targets

Because build.rcl is a regular RCL document, we can use the normal features to generate build targets. For example, let’s take the same users.rcl as before, and generate one output file per user:

let users = import "users.rcl";
{
  for user in users:
  let username = user.email.remove_suffix("@tyrell.com");
  f"users/{username}.toml": {
    format = "toml",
    banner = "# This file is generated from build.rcl.",
    contents = user,
  }
}

This will create a directory users with two files in it: eldon.toml and rachael.toml. The contents of eldon.toml are as follows:

# This file is generated from build.rcl.
email = "eldon@tyrell.com"
name = "Eldon Tyrell"
uid = 0

Recursing into subdirectories

In a larger repository, you may have several independent subdirectories that each contain a build.rcl file. It would be nice to be able to build everything with a single command, instead of having to run rcl build in every subdirectory. We can do this through regular imports. Let's say we have subdirectory a and b, then the top-level build.rcl could look like this:

{
  for fname, target in import "a/build.rcl": f"a/{fname}": target,
  for fname, target in import "b/build.rcl": f"b/{fname}": target,
}

Further reading

For full details, see the rcl build docs, in particular the section with the build file specification. For integration with an external build system, continue to the next chapter about Ninja integration.