In the expression `{ key: true }` (json syntax) `key` is interpreted as an
identifier whereas in `{ key = true }` (record syntax) `key` is a string literal.
This hint should highlight that possible syntax mixup.
When I initially converted from the "if-then-else" keyword syntax to the
colon-based one, during development I put the colon in, then later I
removed it again, but I was still unsure. Now, after having used the
syntax for some time, my feeling is that there should be a colon after
all. Nim got it right. So put it back.
Because it's easy to stay backwards compatible here, make the colon
optional. We can make it mandatory at some point in the future, but even
making the autoformatter put it there is probably a strong enough push.
Report errors properly. This is quite verbose, I should make some
shorthands for formatting paths. Also check the sandbox policy while
we create the output path.
As expected, the golden tests fail to run under Nix because the test
directory is not writable. And it's better to not write in my opinion,
let's not hack that and have a dry run output mode.
For now the output format is not structured, this is good enough for the
thests. It could be nice to do structured output in RCL format, but we
can do that later if needed.
I don't like this distinction that in the build file, we don't
implicitly add a newline, but on the command line we do. But making
users specify a trailing newline on the command line is annoying. So
then the RCL build spec will have to match the CLI, and append the
newline. But then empty string cannot be the default, so let's accept
null too.
I couldn't add a banner to the GitHub Actions yaml because the output
format is json which does not support comments. By making the banner
just a string that is up to the user to fill, everything becomes both
simpler and more general.
Classic Unix tools are silent on success, but I do like to know how much
it wrote. I can imagine the output would become verbose and I'd rather
have one status line like Ninja. But for now this is okay.
Wow, writing a proper deserializer for an RCL value is still a lot of
work, even in Rust! I should write a #[derive] macro for it. But having
record types would make this a lot easier already, so let's wait with
that. For now it's hand-written with a dozen error messages that I will
need to add goldens for ...
When using RCL as a jq replacement, often I have some pipeline and I
want to edit the last part of the query on the command line. I don't
want to have to move my cursor all the way back to wrap the entire
expression in braces. So even though comprehensions can already do map
and filter and flatmap, I still want to add those.
This is step one, adding map.
So I can highlight stuff on my blog as long as there is no highlighting
in Pandoc/Skylighting. With the change to MarkupString, this was really
easy to do!
This will enable it to be used from the webassembly module, and also
gets rid of the duplication for the ansi escape codes. I tried hard to
get the Tree-sitter highlight crate to work with webassembly, but it's
just too much of a mess and debugging linker errors is one of those
things that always takes an entire night, where you feel on the verge
of a breakthrough, but in the end it fails and you end up feeling
miserable. So no Tree-sitter wasm, maybe I can just use this
highlighter, it is less robust and less fancy, but it's better than
nothing.
This temporarily removes highlighting built-ins, maybe I can restore it
again later. Or maybe I can just highlight any function call by looking
ahead for a paren.
Respecting the trailing comma only for collections of >= 2 elements
previously masked this issue. One special case in the formatter is that
singleton list comprehensions don't get the trailing comma. But that
means that we should ignore it for the purpose of wide/tall.
This fixes the regression introduced in a070021. The problem was
discovered by the fuzzer.
This fixes a longstanding issue where reporting errors that we have to
blame on just the document's result in general got blamed on its full
span, which is often a comment and not the offending value. Now we blame
it on the inner body expression, which is more natural.
At first I thought, “but a single-element collection should always fit,
right?” And sure it fits, but then I reformatted a larger experimental
config I have, including some GitHub Actions, and it turns out that
sometimes I prefer even single-element litst to be tall. Black is right
about this. I should be less opinionated, leave it to the user.
Maybe I am nitpicking here, but I think this will benefit readability.
It's cool that you can have let bindings anywhere, but it doesn't
necessarily make things more readable when you cram everything on one
line.
Now that full expressions are no longer allowed in some places, the
changelog mentions how to fix it, but we can actually put the help
straight into the code. And then it's helpful in all cases where you
try to use an expression and the keyword would be valid if you added
paretheses.
This change is similar to the one for conditionals. Like them, it stems
from the formatter printing a raw space before the collection, which is
problematic for non-code prefixes. Only here it did not surface as a non-
idempotency in the formatter because no non-code is allowed before the
collection. Still I think if you want to write this:
[for x in let y = [1, 2, 3]; y: y * 2]
Then instead you should write this:
[for x in (let y = [1, 2, 3]; y): y * 2]
It's much clearer and it creates fewer problems with formatting.
This is again a non-idempotency discovered by the fuzzer. Now an
expression in a hole can have blank lines or comments preceding it.
I am starting to see a pattern: all expressions (that can contain non-
code) need to be preceded by a separator. When they are preceded by
nothing (like was the case for the hole) or by a hard-coded space (like
was the case for the conditional), we may put a break or comment
directly after some content while it should go on its own line.
Are there more cases where an expression is not preceded by a separator?
I guess the fuzzer will find out!
The fuzzer now discovered a non-idempotency in the formatter, in case
there is a non-code prefix for the condition. This has something to do
with the space between "if" and the condition, there is no separator
there, whatever follows goes on the same line, which is usually not the
case.
For evaluation everything works fine, but how do you format this? We can
try to repair it, but it's hard. A solution that sidesteps all this is
to restrict what kind of expressions we can have after an if. Just don't
allow statements and ifs there. We don't lose any expressivity, if you
want that it still works, just put parens around it. With parens it is
also possible to format the expression properly, e.g.
if (
// Comments are fine, everything is indented here, etc.
condition
):
then_value
else
else_value
Oh, and unrelated, I think I am convinced that I want the colon after
"else" back. But let's do that in a follow-up. Or maybe it can be
optional but the formatter always puts it there?
The implementation of this forces propagating spans in more places,
which ended up being a drive-by fix for one place where spans were
computed incorrectly. This fix shows up in the golden tests.
Hmm, at this point its not unanimously more elegant. When parsing a
sequence of things that may have trailing non-code, it was nice to parse
the non-code and then look at the separator. Now we have to peek over it
instead. Alternatively, I could parse it, and have a way to pass it in
to parse_expr as "seed non-code", but that is also a bit clumsy. For now
the "peek past" will do.
Wow! My change to make a statement expr be the top-level one that
optionally includes a prefix was a great discovery! Everything becomes
much simpler now! No need to store those prefixes separately everywhere
any more! I should have done this much earlier. And all of the golden
tests still pass with this change, it's almost magical. Also a good
demonstration of how something that ends up looking simple may not be
simple to discover.
I suspect that a similar transformation is possible with what is
currently Prefixed<Seq>, I'll look into that next.
This changes the CST to keep a list of statements instead of making it
a degenerate tree that nests deeper for every statement. The primary
reason for doing this is to enable better pretty-printing by formatting
either the entire chain as wide or tall, and not breaking up a chain of
let bindings or assertions where some are on a line and some are not.
This is quite a deep change in one sense, but the code changes ended
up being smaller than I expected. It also ends up enabling non-code
prefixes in more places, so I think this is a good change in general.
I need to audit the parser and CST because I think there are now a few
places where a prefix is stored separately that would now be parsed into
an Expr::Statements node instead.
But what surprises me most, after I got everything to compile, all the
golden tests still pass aside from a few reformattings, and the new
format looks universally better than the old one. Wow! I think I really
discovered the "right" way to implement this!
