## Description
Whilst finishing "const generics" I spent a LOT of time trying to find
why some cases where not working. Until I found the issue on a "indirect
input" for the method `type_check_method_application`.
The issue is that some other method was returning a wrong value, and
although the solution is not hard, there is no way to fix just this
problem, and there is no way to assert this returned value to guarantee
that the problem will not return.
To start to improve these kind of tests, and help debug problems, this
PR start to create a "callback" mechanism that allows anyone using the
compiler as a library, this will not be available at the CLI, to inspect
and control logging.
The integration in out e2e test harness is done by the `test.toml`.
```
category = "run"
expected_result = { action = "return", value = 1 }
expected_result_new_encoding = { action = "return_data", value = "0000000000000001" }
validate_abi = true
logs = """
if pkg != "std" && event == "on_before_method_resolution" && method == "len_xxx" {
print_args();
trace(true);
}
if pkg != "std" && event == "on_after_method_resolution" && method == "len_xxx" {
print_args();
trace(false);
}
"""
```
The `harness` framework will not call "logs" for every callback that
happens outside of the "std" library. To allow interaction with the
harness itself, some commands are available:
- `print_args` print all the available arguments for that event on the
"snapshot";
- `trace` enables/disables trace level logging, that also end up in the
"snapshot".
In the example above, we will generate the following snapshot:
```
---
source: test/src/e2e_vm_tests/harness_callback_handler.rs
assertion_line: 14
---
on_before_method_resolution: FromModule { method_name: len_xxx }; Regular([]); S<unresolved T -> u64, N -> 3>
before get_method_safe_to_unify: S<unresolved T -> u64, N -> 3> len_xxx<N -> None>(self: unresolved Self -> S<unresolved T -> T, N -> None>): u64 -> u64
after get_method_safe_to_unify: S<unresolved T -> u64, N -> 3>; len_xxx<N -> 3>(self: unresolved Self -> S<unresolved T -> u64, N -> None>): u64 -> u64
on_after_method_resolution: FromModule { method_name: len_xxx }; Regular([]); S<unresolved T -> u64, N -> 3>; len_xxx<N -> 3>(self: unresolved Self -> S<unresolved T -> u64, N -> None>): u64 -> u64; S<unresolved T -> u64, N -> 3>
on_before_method_resolution: FromModule { method_name: len_xxx }; Regular([]); S<unresolved T -> u64, N -> 3>
before get_method_safe_to_unify: S<unresolved T -> u64, N -> 3> len_xxx<N -> None>(self: unresolved Self -> S<unresolved T -> T, N -> None>): u64 -> u64
after get_method_safe_to_unify: S<unresolved T -> u64, N -> 3>; len_xxx<N -> 3>(self: unresolved Self -> S<unresolved T -> u64, N -> None>): u64 -> u64
```
The important part is that the problem I want to fix it's there. On the
line of "get_method_safe_to_unify", the "self" parameter const generic
argument "N" is not being materialized, although the method resolution
pass started with a fully materialized type.
With this callback machinery it becomes easier to debug the problems,
and possible to generate asserts to internal methods.
## Checklist
- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [ ] I have requested a review from the relevant team or maintainers.
## Description
Bumps fuel-vm, fuel-core fuels-rs and forc-wallet to latest versions.
Upgrades Rust to 2021 edition.
## Checklist
- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [x] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [x] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- I believe this is non-breaking from user code perspective but I'm not
too sure about that
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
---------
Co-authored-by: z <zees-dev@users.noreply.github.com>
Co-authored-by: zees-dev <63374656+zees-dev@users.noreply.github.com>
Co-authored-by: JoshuaBatty <joshpbatty@gmail.com>
Co-authored-by: IGI-111 <igi-111@protonmail.com>
## Description
This PR is the final step in implementing the `panic` expression and the
ABI errors, as defined in the [ABI Errors
RFC](https://github.com/FuelLabs/sway-rfcs/blob/master/rfcs/0014-abi-errors.md).
The PR generates the `errorCodes` section in the ABI, like, e.g.:
```
"errorCodes": {
"18446744069414584320": {
"pos": {
"pkg": "my_lib@1.2.3",
"file": "src/lib.rs",
"line": 42,
"column": 13
},
"logId": null,
"msg": "Error message."
},
"18446744069414584321": {
"pos": {
"pkg": "my_contract",
"file": "src/main.rs",
"line": 21,
"column": 34
},
"logId": "4933727799282657266",
"msg": null
}
}
```
If the `panic` argument is a string slice that can be const-evaluated,
the `msg` field of the corresponding error code is generated, and a log
is not emitted. The string itself is stored only within the ABI JSON,
and is not included into the generated bytecode.
Additionally, the PR:
- provides a common infrastructure for obtaining a `SourceLocation` from
a `Span`. This is currently used in the implementation of the `__dbg`
intrinsics and the `panic` expression.
- improves filtering of `forc` outputs in snapshot tests by supporting
regular expressions. The `grep` command is renamed to `regex` to avoid
confusion with piping the actual `grep` CLI command.
Partially addresses #6765.
## Checklist
- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
---------
Co-authored-by: hal3e <git@hal3e.io>
## Description
This PR implements the `__dbg(...)` intrinsic, which is very similar to
Rust `dbg!(...)` macro.
Up until now, it has being the norm to use `__log` to debug values.
Given that this is NOT the first use case for log, we have always found
some issues with it: log does not work on predicates, log does not show
"private" fields like `Vec::capacity` and others.
To solve these problems `__dbg` is being introduced:
1 - it will work on all program types, including predicates;
2 - it also prints the file name, line and column;
3 - All types will have an automatic implementation of Debug if
possible, which can still be customized.
4 - Even `raw_ptr` and other non "loggable" types, have `Debug` impls.
5 - `__dbg` will be completely stripped in the release build by default.
It can be turned on again if needed.
So this:
```
// Aggregates
let _ = __dbg((1u64, 2u64));
let _ = __dbg([1u64, 2u64]);
// Structs and Enums
let _ = __dbg(S { });
let _ = __dbg(E::None);
let _ = __dbg(E::Some(S { }));
```
will generate this:
```
[src/main.sw:19:13] = (1, 2)
[src/main.sw:20:13] = [1, 2]
[src/main.sw:23:13] = S { }
[src/main.sw:24:13] = None
[src/main.sw:25:13] = E(S { })
```
How does this work?
`__dbg(value)` intrinsic is desugared into `{ let f = Formatter{};
f.print_str(...); let value = value; value.fmt(f); value }`.
`Formatter` is similar to Rust's one. The difference is that we still do
not support string formatting, so the `Formatter` has a lot of `print_*`
functions.
And each `print` function calls a "syscall". This `syscall` uses `ecal`
under the hood and it follows unix write syscall schema.
```sway
// ssize_t write(int fd, const void buf[.count], size_t count);
fn syscall_write(fd: u64, buf: raw_ptr, count: u64) {
asm(id: 1000, fd: fd, buf: buf, count: count) {
ecal id fd buf count;
}
}
```
For that to work, the VM interpreter must have its `EcalState` setup and
interpret syscall number 1000 as `write`. This PR does this for `forc
test` and our `e2e test suite`.
Each test in `forc test` will capture these calls and only print to the
terminal when requested with the `--log` flag.
## Garbage Collector and auto generated
Before, we were associating all auto-generated code with a pseudo file
called "<autogenerated>.sw" that was never garbage collected.
This generated a problem inside the LSP when the `auto_impl.rs` ran a
second time because of a collision in the "shareable type" map. When we
try to solve this collision, choosing to keep the old value or to insert
the new, the type inside the map points to already collected types and
the compiler panics. This is a known problem.
The workaround for this is to break the auto-generated code into
multiple files. Now they are named "main.autogenerated.sw", for example.
We create one pseudo-file for each real file that needs one.
When we garbage collect one file, `main.sw`, for example, we also
collect its associated auto-generated file.
## Checklist
- [ ] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [x] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
## Description
This PR is a prequel to https://github.com/FuelLabs/sway/pull/7015,
trying to optimize the compiler to alleviate the hit of having more
items, `impl` etc...
We have here two optimizations:
1 - We spend a lot of time counting newlines when converting byte
offsets to `LineCol`. Now, we calculate line offsets just once, and use
binary search later to find which line a byte offset is at.
2 - `QueryEngine::get_programs_cache_entry` was cloning the whole
`TyProgram`. That is why `did_change_with_caching` was always getting
worse, as the program was increasing in size. Now all compilation stages
are behind `Arc`, which makes the `clone` free.
## Analysis
Putting a `dbg!(...)` like the image below, and calling `counts`
(https://github.com/nnethercote/counts).
```
cargo bench -- traverse 2>&1 | grep "sway-types/src/span.rs:29:9" | counts
```
I get the following results:
```
972102 counts
( 1) 156720 (16.1%, 16.1%): [sway-types/src/span.rs:29:9] self.pos = 0
( 2) 15900 ( 1.6%, 17.8%): [sway-types/src/span.rs:29:9] self.pos = 104
( 3) 15840 ( 1.6%, 19.4%): [sway-types/src/span.rs:29:9] self.pos = 107
( 4) 2280 ( 0.2%, 19.6%): [sway-types/src/span.rs:29:9] self.pos = 19281
( 5) 2280 ( 0.2%, 19.9%): [sway-types/src/span.rs:29:9] self.pos = 19285
( 6) 2280 ( 0.2%, 20.1%): [sway-types/src/span.rs:29:9] self.pos = 19287
( 7) 2280 ( 0.2%, 20.3%): [sway-types/src/span.rs:29:9] self.pos = 19292
( 8) 2280 ( 0.2%, 20.6%): [sway-types/src/span.rs:29:9] self.pos = 19323
( 9) 2280 ( 0.2%, 20.8%): [sway-types/src/span.rs:29:9] self.pos = 19327
( 10) 2280 ( 0.2%, 21.0%): [sway-types/src/span.rs:29:9] self.pos = 19329
( 11) 2280 ( 0.2%, 21.3%): [sway-types/src/span.rs:29:9] self.pos = 19334
( 12) 870 ( 0.1%, 21.4%): [sway-types/src/span.rs:29:9] self.pos = 4285
...
```
This means that `line_col` is being called 972k times. 16% is for
position zero, which should be trivial. The rest will iterate the whole
file source code to count number of lines. Making the real complexity of
the work here something like `O(qty * self.pos)`. And some values of
`self.pos` are not trivial at all.
## Checklist
- [ ] I have linked to any relevant issues.
- [ ] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [ ] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [ ] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [ ] I have requested a review from the relevant team or maintainers.
## Description
This PR reimplements how `#[attribute]`s are parsed, defined, and
checked. It fixes the following issues we had with attributes:
- attributes were in general not checked for their expected and allowed
usage.
- when checked, checks were fixing particular reported issues and were
scattered through various compilation phases: lexing, parsing, and tree
conversion.
- when checked, reported errors in some cases had bugs and were
overalapping with other errors and warnings.
- attribute handling was not centralized, but rahter scattered throught
the whole code base, and mostly done at use sites.
For concrete examples of these issues, see the list of closed issues
below.
The PR:
- encapsulates the attributes handling within a single abstraction:
`Attributes`.
- assigns the following properties to every attribute:
- _target_: what kind of elements an attribute can annotate.
- _multiplicity_: if more then one attribute of the same kind can be
applied to an element.
- _arguments multiplicity_: a range defining how many arguments an
attribute can or must have.
- _arguments value expectation_: if attribute arguments are expected to
have values.
- validates those properties in a single place, during the
`convert_parse_tree`. Note that this means that modules with attribute
issues will now consistently not reach the type checking phase. This was
previously the case for some of the issues with attributes where custom
checks where done during the type checking phase. #6987 proposes to move
those checks after the tree conversion phase, allowing the continuation
of compilation.
- adds the `AttributeKind::Unknow` to preserve unknown attributes in the
typed tree.
- removes redundant code related to attributes: specific warnings and
errors, specialized parsing, repetitive and error-prone at use site
checks, etc.
- removes the unused `Doc` attribute.
The PR also introduces a new pattern in emitting errors. The
`attr_decls_to_attributes` function will always return `Attributes` even
if they have errors, because:
- we expect the compilation to continue even in the case of errors.
- we expect those errors to be ignored if a valid `#[cfg]` attribute
among those evaluates to false.
- we expect them to be emitted with eventual other errors, or alone, at
the end of the tree conversion of the annotated element.
For more details, see the comment on `attr_decls_to_attributes` and
`cfg_eval`.
Closes#6880; closes#6913; closes#6914; closes#6915; closes#6916;
closes#6917; closes#6918; closes#6931; closes#6981; closes#6983;
closes#6984; closes#6985.
## Breaking changes
Strictly seen, this PR can cause breaking changes, but only in the case
of invalid existing attribute usage. We treat those breaking changes as
bug fixes in the compiler and, thus, do not have them behind a feature
flag.
E.g., this kind of code was possible before, but will now emit and
error:
```sway
#[storage(read, write, read, write)]
struct Struct {}
```
## Checklist
- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [x] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
## Description
Merges the two libraries. They were initially separate to separate the
core logic and fuel vm specific functionality, but that separation is no
longer maintained so having a merged library is better.
Closes#6708
## Checklist
- [ ] I have linked to any relevant issues.
- [ ] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [ ] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [ ] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [ ] I have requested a review from the relevant team or maintainers.
---------
Co-authored-by: Sophie <47993817+sdankel@users.noreply.github.com>
Co-authored-by: Igor Rončević <ironcev@hotmail.com>
## Description
This PR:
- extends infrastructure for writing migrations that modify individual
expressions and do not need access to a broader scope. It does that by
providing lexed and typed tree visitors. Visiting lexed and typed tree
is implemented to the level needed to write the migration for `impl
TryFrom<Bytes> for b256`.
- implements the migration for `impl TryFrom<Bytes> for b256` as
explained in #6994.
Adding `#[cfg(experimental_try_from_bytes_for_b256 = true)]` to the
`impl TryFrom<Bytes> for b256` revealed that there existed already an
`impl TryFrom<Bytes> for b256` in `std::primitive_conversions::b256`.
Due the lack of trait coherence there was no any errors when it was
re-implemented in `std::bytes`.
This PR marks the existing impl in `std::primitive_conversions::b256`
with `#[cfg(experimental_try_from_bytes_for_b256 = false)]` so that it
will be removed in the next breaking release, in favor of the new impl
in `std::bytes`.
## Checklist
- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [ ] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
## Description
This PR introduces `forc-migrate`, a Forc tool for migrating Sway
projects to the next breaking change version of Sway.
The tool addresses two points crucial for code updates caused by
breaking changes:
- it informs developers about the breaking changes and **assists in
planing and executing** the migration.
- it **automatically changes source code** where possible, reducing the
manual effort needed for code changes.
Besides adding the `forc-migrate` tool, the PR:
- extends `Diagnostic` to support migration diagnostics aside with
errors and warnings.
- changes `swayfmt` to support generating source code from arbitrary
lexed trees. The change is a minimal one done only in the parts of
`swayfmt` that are executed by migration steps written in this PR.
Adapting `swayfmt` to fully support arbitrary lexed trees will be done
in #6779.
The migration for the `references` feature, migrating `ref mut` to
`&mut`, is developed only partially, to demonstrate the development and
usage of automatic migrations that alter the original source code.
The intended usage of the tool is documented in detail in the "forc
migrate" chapter of The Sway Book: _Forc reference > Plugins >
forc_migrate_. (The generated documentation has issues that are caused
by the documentation generation bug explained in #6792. These issues
will be fixed in a separate PR that will fix it for all the plugins.)
We expect the `forc-migrate` to evolve based on the developer's
feedback. Some of the possible extensions of the tool are:
- adding additional CLI options, e.g., for executing only specific
migration steps, or ignoring them.
- passing parameters to migration steps from the CLI.
- not allowing updates by default, if the repository contains modified
or untracked files.
- migrating workspaces.
- migrating other artifacts, e.g., Forc.toml files or contract IDs.
- migrating between arbitrary versions of Sway.
- migrating SDK code.
- etc.
`forc-migrate` also showed a clear need for better infrastructure for
writing static analyzers and transforming Sway code. The approach used
in the implementation of this PR should be seen as a pragmatic
beginning, based on the reuse of what we currently have. Some future
options are discussed in #6836.
## Demo
### `forc migrate show`
Shows the breaking change features and related migration steps. This
command can be run anywhere and does not require a Sway project.
```
Breaking change features:
- storage_domains (https://github.com/FuelLabs/sway/issues/6701)
- references (https://github.com/FuelLabs/sway/issues/5063)
Migration steps (1 manual and 1 semiautomatic):
storage_domains
[M] Review explicitly defined slot keys in storage declarations (`in` keywords)
references
[S] Replace `ref mut` function parameters with `&mut`
Experimental feature flags:
- for Forc.toml: experimental = { storage_domains = true, references = true }
- for CLI: --experimental storage_domains,references
```
### `forc migrate check`
Performs a dry-run of the migration on a concrete Sway project. It
outputs all the occurrences in code that need to be reviewed or changed,
as well as the migration time effort:
```
info: [storage_domains] Review explicitly defined slot keys in storage declarations (`in` keywords)
--> /home/kebradalaonda/Desktop/M Forc migrate tool/src/main.sw:19:10
|
...
19 | y in b256::zero(): u64 = 0,
| ------------
20 | z: u64 = 0,
21 | a in calculate_slot_address(): u64 = 0,
| ------------------------
22 | b in 0x0102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f20: u64 = 0,
| ------------------------------------------------------------------
|
= help: If the slot keys used in `in` keywords represent keys generated for `storage` fields
= help: by the Sway compiler, those keys might need to be recalculated.
= help:
= help: The previous formula for calculating storage field keys was: `sha256("storage.<field name>")`.
= help: The new formula is: `sha256((0u8, "storage.<field name>"))`.
= help:
= help: For a detailed migration guide see: https://github.com/FuelLabs/sway/issues/6701
____
Migration effort:
storage_domains
[M] Review explicitly defined slot keys in storage declarations (`in` keywords)
Occurrences: 3 Migration effort (hh::mm): ~00:06
references
[S] Replace `ref mut` function parameters with `&mut`
Occurrences: 0 Migration effort (hh::mm): ~00:00
Total migration effort (hh::mm): ~00:06
```
### `forc migrate run`
Runs the migration steps and guides developers through the migration
process.
## Checklist
- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [x] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [x] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [ ] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
## Description
Fixes#5498.
This PR contains a major refactoring of the namespace module, which is
responsible for keeping track of the symbol bindings in the program. The
refactoring is intended to reduce memory allocation during compilation,
and also fixes a number of minor bugs related to path resolution.
### Advice to the reviewer:
I recommend reviewing this PR in 3 steps:
1. Review the PR description. If something in the description is
unclear, then leave a review comment, and I will try to address the
issue. The code changes will be significantly easier to review once you
get the overall idea of what is happening.
2. Review the files `sway-core/src/semantic_analysis/namespace/root.rs`,
`sway-core/src/semantic_analysis/namespace/namespace.rs` and
`sway-core/src/language/call_path.rs`. These files contain the major
datastructure changes, and if anything in those files needs to be
addressed, then there is limited value in reviewing the rest of the code
changes.
3. Review the remaining files. These code changes are all consequences
of the changes in step 2, so it shouldn't take too much time, even
though there are quite a few changes.
### Remaining issues
No remaining issues except for what reviwers bring up.
### Breaking changes
As illustrated by one of the test cases the PR makes paths to external
libraries such as `::core::codec::*` illegal. This is a bugfix rather
than a change in behavior (the correct path is `core::codec::*`), but
may require updates of existing programs.
Additionally, the visibility check for modules was broken, so some
modules declared using `mod x` would have incorrectly been considered
public even though it should have been declared `pub mod x`. Again, this
is a bugfix rather than a change of behavior, but may require updates of
existing programs.
### Changes made by this PR
_Representation of the dependency graph_ (main file:
`sway-core/src/semantic_analysis/namespace/root.rs`)
Previous: External libraries were represented as submodules of every
module in the program. This meant that every module in every external
package was cloned for every module in the program, and recursively so,
since external libraries could also have dependencies that required
clones for every module. In particular, `std` has `core` as a
dependency:
```
core:
"" (lib.sw)
/ / | \ \
primitives raw_ptr raw_slice slice ...
std:
"" (lib.sw)
/ / | \ \
core constants vec primitive_conversions ...
/ / | \ \ | | | | \
... core core core b256 ...
/ / | \ \ / / | \ \ / / | \ \ |
... ... ... core
/ / | \ \
...
External library:
"" (lib.sw)
/ / | \
core std submodule ...
/ / | \ \ / \ / \ \
... core ... core std ...
/ / | \ \ / / | \ \ / \
... ... core ...
/ / | \ \
Main program:
"" (main.sw)
/ / | \
core std external library submodule
/ / | \ \ / \ / \ \ / | \
... core ... core std submodule core std external library
/ / | \ \ / / | \ \ / \ / \ / \ / \ / | \
... ... core ... ... ... ... core ... core std ...
/ / | \ \ / \ / \ / \
... ... core ...
/ \
...
```
Changes in this PR: External libraries are now represented only at the
package root. The root module is given the same name as the package so
that paths beginning with the package name can be resolved as a module
path. If multiple libraries depend on the same external library, then
they each contain a clone (this can be resolved a at later stage by
introducing a global table of libraries).
```
core:
core (lib.sw)
/ / | \ \
primitives raw_ptr raw_slice slice ...
std:
std (lib.sw) ====== externals =====> core
/ / | \ \
/ | | \ ...
constants vec primitive_conversions ...
/ \
b256 ...
External library:
external_lib (lib.sw) === externals ===> core
/ \ / / | \ \
submodule ... ...
|
... std == externals => core
/ / | \ \ / / | \ \
... ...
Main program:
main_program (main.sw) === externals ===> core
/ \ / / | \ \
submodule ... ...
|
... std == externals => core
/ / | \ \ / / | \ \
... ...
extenral_lib == externals => core
/ / | \ \ / / | \ \
... ...
std == externals => core
/ / | \ \ / / | \ \
... ...
```
Reasoning: This change dramatically reduces the amount of cloning of
external libraries. The cloning is not eliminated completely, since
multiple libraries may have the same library as a dependency (e.g.,
`core` is used as a dependency for both the program and `std`, so if the
program depends on `std`, then there will be two clones of `core` in the
tree). The remaining cloning can be eliminated by introducing a global
table of libraries, which should be doable without making significant
changes to the path resolution algorithm.
_Callpath representation_ (main file:
sway-core/src/language/call_path.rs)
Previous: Paths to externals were considered relative to current module.
This worked because all externals existed a submodules of every module.
Paths inside the current package were ambiguous, and were either
interpreted relative to current module or relative to the root module.
The root module did not have a name, so the empty path was used to
indicate the root module.
Now: All full paths start with the name of the package, which is also
the name of the root module. If the first identifier is not the name of
the current package, then we look in the externals of that package, and
if the name isn't found there we throw an error. Note that the name of
an external is defined in Forc.toml, and may be different from the name
used inside the external library (e.g., a library may call itself `X`,
but a project may use `X` as an external under the name `Y`). This means
that any path that escapes from a library must have its package name
changed.
Reasoning: This is all a conseqence of the change to the representation
of the dependency graph. We need to be able to the rely on the package
name in order to resolve the path within the correct pacakge.
Additionally, if a path escapes from a dependency to a dependent, then
the package name must be changed to what the package is called in the
dependent.
_CallPath.callpath_type_ (main file:
sway-core/src/language/call_path.rs)
Previous: The CallPath struct contained a flag `is_absolute`, which, if
set, was used to mean two different things. For parsed paths it was used
to signal that the path started with `::`, i.e., that the path should be
resolved relative to the current package root. For paths generated
during typechecking the flag was used to indicate that the path was a
full path, i.e., that it should either be resolved relative to the
current package root or that it was a path to an external library.
Now: `is_absolute` has been replaced with `callpath_type`, which has 3
possible values: `RelativeToPackageRoot`, which indicates that the path
started with `::`, `Full`, which indicates that the path starts with the
package name, and `Ambiguous`, which indicates that the path should
either be resolved relative to the current module, or that it is a full
path referring to an external library.
Reasoning: A path has 3 different states, so a boolean flag is not
sufficient to represent all states.
_Path representation_ (main file: sway-core/src/language/call_path.rs)
Previous: Full paths always pointed to the module containing the
declaration of the thing that the path resolves to.
Now: Full paths point to a module where the suffix is bound. This may
not be the module containing the declaration of the thing the suffix is
bound to, because the binding may be the result of an import. The trait
map is special in that the path to the actual declaration is used as a
key in the map, so for traits we use the canonical path, which point to
the module where the declaration resides.
Reasoning: The previous definition of full paths required a duplication
of the symbol resolution algorithm in order to construct the full path,
and it was getting very difficult to maintain. In the vast majority of
cases we only need the full path to indicate somewhere where the symbol
can be resolved, and don't actually need to know where the declaration
resides. In the few cases where we do need to know the path to the
declaration (the trait map uses these paths as keys) we can generate the
canonical path by first converting to the full path, and then resolving
the path and using the path stored with the binding in the lexical
scope, since this is the path to the actual declaration. Additionally,
the previous solution was only able to generate full paths for paths
where the suffix is an identifier - for other suffix types we could not
generate a full path. This has now been changed so that we can generate
full paths for all paths, although canonical paths are still restricted
to paths with an identifier as the suffix. This also moves us closer to
a situation where we can represent semantic (canonical) paths separately
from syntactic callpaths.
_Namespace.init removed_ (main file:
sway-core/src/semantic_analysis/namespace/namespace.rs)
Previous: The `Namespace` struct contained an `init` module which was
used as a template for new modules, a clone of which was created every
time the collection pass or the typechecker entered a submodule. This
template contained all external libraries as submodules, as well as (for
contract pacakges) a (new) declaration of `CONTRACT_ID`.
Now: The `init` module has been removed. When the collection pass or the
typechecker enters a new submodule, `Namespace` creates a new `Module`
(if none exists yet for the relevant submodule), and populates it with
implicit imports (`core::prelude::*` and `std::prelude::*`, and for
contract packages `CONTRACT_ID`).
Reasoning: The current solution makes it significantly clearer what is
in a new `Module` object. Additionally, the declaration of `CONTRACT_ID`
is no longer duplicated in each module - it is now declared once in the
root module, and imported everywhere else.
_namespace encapsulation_ (main file:
sway-core/src/semantic_analysis/namespace/namespace.rs)
Previous: The `Namespace` struct contained methods to extract the
underlying datastructures in the `namespace` module.
Now: An attempt has been made to encapsulate the `namespace` module a
little bit more, so that changes and lookups to the datastructures in
the `namespace` module are done by calling methods on the Namespace
struct rather than letting the caller extract the datastructures and
manipulating them directly.
Reasoning: When the datastructures are manipulated in many different
places it makes it very hard to track down all the places that need to
be changed when making changes to the datastructure. The current
solution is not perfect, but it does encapsulate things a bit better
than before.
### Minor changes and improvements
A few minor improvements have been made:
- A compilation unit (`core`, `std`, etc.) is now referred to as a
_package_. (Note: The `Root` struct should probably be renamed to
`Package` for consistency)
- The visibility check for path resolution in `type_resolve.rs` had to
be updated because path resolution was changed. The current version is
not perfect, but it does improve the faulty check we had before.
- `TyModule` so far contained a clone of the `Namespace` struct in the
state it was in when typechecking of the relevant module was done. The
`Namespace` object was never used except for the one in the root module,
so I have moved the `Namespace` clone to `TyProgram`, thus eliminating
quite a bit of unnecessary cloning (the Namespace struct owns the
current package `Root` object, and thus cloning the `Namespace` struct
clones the entire dependency graph). For error handling purposes in
sway-lsp we save the current namespace in `TypeCheckFailed` rather than
in `TyModule`, which also significantly reduces memory allocation during
compilation.
### Updated tests
A number of tests have been updated for various reasons:
- Improved or changed error messages
- Module visibility was not checked correctly, so some tests had to have
certain submodules made public.
- In once case, `core::str_slice` was referenced using the path
`::core::str_slice`. This is invalid, but since `core` used to be
treated as a submodule of every module in the program, the path resolved
anyway. (This also shows that the invalid paths in issue #5498 are now
actually treated as invalid - I have not added regression tests of that
issue, since it seems unlikely that we will reintroduce the bug).
### Unresolved issues
- The `Namespace` struct is cloned before the symbol collection phase.
This is expensive, since the `Namespace` struct at that point contains
the entire dependency graph (minus the modules of the current package).
Ideally we would like the symbol collection phase and the typechecker to
use the same `Namespace` struct, but at the moment the Module struct is
unable to perform updates from parsed to typechecked declarations, so
this isn't possible. The cloning happens in `sway-core/src/lib.rs` and
`sway-core/src/semantic_analysis/namespace/contract_helpers.rs`. Once we
represent externals as a global table we should be able to change the
`Root.externals` map to only use references into the global table, which
will make the cloning step much cheaper.
- `contract_helpers.rs` throws a regular compiler error if typechecking
fails, but that should be an ICE, since it is generated code that is
failing to typecheck.
- Paths may in some cases escape from external packages to the current
package. If this happens across multiple packages, then this may result
in the path no longer being valid. For instance, if A depends on B,
which in turn depends on C, then a path may escape from C via B into A,
but since A does not have C as a direct dependency, then the path can no
longer be resolved. This again can be fixed once we introduce a global
table of externals.
- The visibility check in `type_resolve.rs` is insufficient, because it
doesn't handle enum variants and associated types correctly.
## Checklist
- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [x] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [x] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [x] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
---------
Co-authored-by: João Matos <joao@tritao.eu>
Co-authored-by: Joshua Batty <joshpbatty@gmail.com>
## Description
Today, contract call uses the method name to identify which contract
method to call. That means that a contract cannot have multiple methods
with the same name.
That can happen when a contract implements traits:
A lot of the lifetime changes were caused by clippy.
## Checklist
- [ ] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [ ] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [ ] I have requested a review from the relevant team or maintainers.
---------
Co-authored-by: Joshua Batty <joshpbatty@gmail.com>
## Description
This PR refactors `swayfmt` to be able generate code from arbitrary
lexed trees. Arbitrary means a lexed tree that can be fully or partially
created in-memory by manipulating the tree structure in code. It does
not need to necessarily be backed by source code. This is needed to be
able to reuse `swayfmt` for generating source code from tools that
analyze and modify Sway code, as explained in #6779.
This PR makes the `swayfmt` independent of spans backed by the source
code, by doing the following changes:
- Keywords are rendered by using theirs `AS_STR` associated constants.
- The `Token` trait is extended with the `AS_STR` associated constant,
and tokens are rendered by using that constant.
- `Ident`s are rendered by using theirs `as_str()` methods. This method
takes the textual implementation from `Ident::name_override_opt` if it
is provided, and ignores the span in that case.
- `Literal`s are rendered based on the literal value and not their
spans. The exception are numeric literals that do not have empty spans.
Those are considered to be backed by source code and are rendered as
written in code, e.g., `1_000_000u64`.
The PR also fixes some existing bugs in `swayfmt`:
- `use path::{single_import,};` will be formatted as `use
path::single_import;`
- `use path::{a,b,};` will be formatted as `use path::{a, b};`
- partial addresses #6802 by rendering annotations for final values.
- partial addresses #6805 by properly rendering final values, but still
not using the expected field alignment.
The PR also removes the `path` parameter from the `parse_file`. It was
used only to provide an unnecessary dummy `source_id` which was always
pointing to the package root file.
Closes#6779.
## Checklist
- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
## Description
This PR extends Serde support across all AstNode-related types, building
upon the partial implementation from #4193. Key points:
* Adds both `Serialize` and `Deserialize` trait implementations to all
relevant types.
* Prepares for upcoming language server features that will
serialize/deserialize types between keystrokes.
* Important: The type interface in the compiler is not yet stable. This
implementation is intended for temporary serialization/deserialization
work, not for persistent storage.
## Checklist
- [x] I have linked to any relevant issues.
- [ ] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [ ] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
## Description
This PR implements a major rewrite of the `TypeEngine`. The rewrite:
- significantly improves overall compilation performance. The
compilation time of the real-world [Spark Orderbook
workspace](https://github.com/compolabs/orderbook-contract) got
**reduced from 8.99 to 5.94 seconds**, and the memory consumption from
**3.32 to 1.78 GB**.
- provides robust API for inserting types, related to the usage of the
`source_id` (see: #5991). The new API forbids not providing `source_id`s
or providing semantically questionable or non-optimal ones.
- introduces simpler, much less verbose, API for inserting types.
The PR also removes the obsolete and unused `TypeInfo::Storage`.
The PR **does not address** the following:
- The `TypeEngine`'s "garbage-collection-(un)friendlines" (see: #6603).
- The handling of the `TypeInfo::Custom` and `TypeInfo::TraitType` types
within the `TypeEngine` (see: #6601).
- The number of interactions with the `TypeEngine`. E.g., removing
unnecessary insertions after `resolve()`ing or `monomorphyze()`ing types
will be done as a part of `DeclEngine` optimization.
Closes#5991.
## Shareable types
The PR formalizes the notion of a _shareable type_ within the
`TypeEngine` (strictly speaking, a shareable `TypeInfo`). A shareable
type is a type that is both:
- _unchangeable_: it, or any of its parts, cannot be replaced during the
unification or monomorphization.
- and _undistinguishable by annotations_: it doesn't carry any
additional information (annotations) that could differ it from another
instance of `TypeInfo` that would, purely from the type perspective, be
a same type.
E.g., `u64` or `MyStruct<u64, bool>` are unchangeable while `Numeric`,
`Unknown` or `MyStruct<T1, T1>` are changeable.
E.g., in this example, `a` and `b` have the same type, `[u64; 2]` but
those two types differ in spans assigned to the "u64"s and the "2"s, and
are treated as different within the `TypeEngine`, and thus as
non-shareable.
```
let a: [u64; 2] = [1, 1];
let b: [u64; 2] = [2, 2];
```
Shareability of a type is crucial for reusing the `TypeSourceInfo`
instances within the type engine. Shareable types can be given different
`TypeId`s without the need to newly allocate a `TypeSourceInfo`.
## Performance improvements
The cummulative effect of the performance improvements on the
compilataion of the real-world [Spark Orderbook
workspace](https://github.com/compolabs/orderbook-contract) is given
below. The compilation means the frontend compilation, up to the IR
generation (`forc check`).
**Compilation time**
```
Before:
Time (mean ± σ): 8.995 s ± 0.297 s [User: 7.126 s, System: 1.289 s]
Range (min … max): 8.675 s … 9.262 s 3 runs
After:
Time (mean ± σ): 5.945 s ± 0.517 s [User: 4.749 s, System: 0.768 s]
Range (min … max): 5.349 s … 6.280 s 3 runs
```
**Memory consumption**
```
---------------------------------------------------------
total(B) useful-heap(B) extra-heap(B)
---------------------------------------------------------
Before: 3,316,786,808 3,237,317,383 79,469,425
After: 1,784,467,376 1,743,772,406 40,694,970
```
### Applied optimizations:
- **Replacement of expensive `insert` calls with compile time constants
for built-in types.** Built-in types like `!`, `bool`, `()`, `u8`, etc.
are inserted into the engine at its creation at predefined slots within
the `slab`. The `id_of_<built-in-type>` methods just return those
predefined `TypeId`s, effectively being compiled down to constants. The
calls like `type_engine.insert(engines, TypeInfo::Boolean, None)` are
replaced with maximally optimized and non-verbose
`type_engine.id_of_bool()`.
- **Elimination of extensive creation of `TypeSourceInfo`s for
`TypeInfo::Unknown/Numeric`s.** `Unknown` and `Numeric` are inserted
into the engine ~50.000 times. Each insert used to create a new instance
of `TypeSourceInfo` with the `source_id` set to `None`. The optimization
replaces those ~50.000 instances with two predefined singleton
instances, one for `Unknown` + `None` and one for `Numeric` + `None`.
(Note that when implementing #6603, we will want to bind also `Unknown`s
and `Numeric`s to `source_id`s different then `None`, but we will still
want to reuse the `TypeInfo` instances.)
- **Elimination of extensive temporary heap-allocations during hash
calculation.** The custom hasher obtained by `make_hasher` required a
`TypeSourceInfo` to calculate the hash and it was called every time the
`insert` was called, ~530.000 times. Getting the `TypeSourceInfo`
originally required cloning the `TypeInfo`, which depending on the
concrete `TypeInfo` instance could cause heap allocations, and also
heap-allocating that copy within an `Arc`. Hash was calculated
regardless of the possibility for the type to be stored in the hash map
of reusable types. The optimization removed the hashing step if the type
is not shareable, removed the cloning of the `TypeInfo` and introduced a
custom `compute_hash_without_heap_allocation` method that produced the
same hash as the `make_hasher` but without unnecessary temporary
heap-allocations of `TypeSourceInfo`s.
- **Replacement of `TypeSourceInfo`s within the hash map with
`Arc<TypeSourceInfo>`s.** The hash map unnecessarily required making
copies of reused `TypeSourceInfo`s.
- **Introducing the concept of a _shareable type_ and rolling it out for
all types.** Previously, the engine checked only for changeability of
types by using the `TypeInfo::is_changeable` method. The implementation
of that method was extremely simplified, e.g. treating all structs and
enums with generic arguments as changeable even if they were fully
monomorphized. This resulted in over-bloating the engine with type
instances that were actually unchangeable, but considered to be
changeable. Also, strictly seen, the unchangeability (during unification
and monomorphization) is not the only necessary criteria for reuse (or
sharing) a type within the engine. Another important aspect is, as
explained above, the _differentiability by annotations_. The PR
introduces the notion of a _shareable type_ which is both unchangeable
and not differentiable by annotations. The optimization takes advantage
of such types by storing them only once per `source_id`.
- **Elimination of extensive unnecessary inserts of new types during
`replace()` calls.** When `replace()`ing types during unifications, a
new `TypeSourceInfo` instance was created for every replacement, ~46.000
times. This meant a heap-allocation of the `TypeSourceInfo` (16 bytes)
and the `Arc`-contained `TypeInfo` (232 bytes), even if the replaced
type was shareable and already available in the type engine. The
optimization now reuses an already existing shareable type if available.
## Robustness related to `source_id`s
The issues we had with properly providing the `source_id` in the
`insert` method are explained in #5991. This PR removes the `source_id`
from the new public API and calculates it internally within the engine,
based on the type being inserted. This makes inserting of types both
more robust and less verbose and eliminates the possibility of providing
a semantically wrong `source_id`.
Note that the calculation of an optimal `source_id` done within the
engine fully corresponds to the "calculations" we currently have at call
sites.
E.g., when inserting enums, previously we always had to write
`type_engine.insert(engines, TypeInfo::Enum(decl_id),
enum_decl.span.source_id())`. Fetching the `source_id` from the enum
declaration is now done within the `insert_enum` method:
`type_engine.insert_enum(engines, decl_id)`.
Note that for certain types we will want to change the current behavior
and either provide a `source_id` or pick a more suitable one. E.g, even
when inserting `Unknown`s, we will want to have `source_id`s, if
possible. This will be done in #6603, but again, together with providing
a robust API that will be difficult to misuse.
## Simplicity
As already mentioned in some of the examples above, the new
`id_of_<type>()` and `insert_<type>` methods provide much simpler and
less verbose API to use. Introducing those methods remove a lot of
boilerplate code from the callers. Also, the individual `insert_<type>`
methods are additionally optimized for inserting the particular type.
The common `insert` method is intended to be used only in cases where
the inserted `TypeInfo` is not known at the call site.
Here are some examples of the new API versus the existing one:
|Before|After|
|------|-----|
| `type_engine.insert(engines, TypeInfo::Tuple(vec![]), None)` |
`type_engine.id_of_unit()` |
| `type_engine.insert(engines, TypeInfo::Unknown, None)` |
`type_engine.new_unknown()` |
| `type_engine.insert(engines,
TypeInfo::UnsignedInteger(IntegerBits::SixtyFour), None)` |
`type_engine.id_of_u64()` |
For more complex types, like, e.g., `TypeInfo::Tuple`, the difference in
inserting is even more prominent, as can be seen from the diffs of
numerous code lines deleted in this PR.
## Checklist
- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
---------
Co-authored-by: Joshua Batty <joshpbatty@gmail.com>
Co-authored-by: João Matos <joao@tritao.eu>
## Description
This PR starts the implementation of
https://github.com/FuelLabs/sway-rfcs/blob/master/rfcs/0013-changes-lifecycle.md.
## CLI flags
Now, all `CLI`s, including our tests, support two new arguments
```
...
--experimental <EXPERIMENTAL>
Comma separated list of all experimental features that will be enabled [possible values: new_encoding, storage_domains]
--no-experimental <NO_EXPERIMENTAL>
Comma separated list of all experimental features that will be disabled [possible values: new_encoding, storage_domains]
...
```
Experimental features can also be enabled inside `Forc.toml` for
workspaces and individual projects.
```
experimental = { encoding-v1 = true }
```
And can be enabled using environment variables:
```
FORC_NO_EXPERIMENTAL=feature_a,feature_b FORC_EXPERIMENTAL=feature_c forc build ...
```
These configurations are applied in a very specific order to allow them
to be overwritten. The order from the weakest to the strongest is:
1 - Workspace `TOML`
2 - Project `TOML`
3 - CLI
4 - Environment variables.
The rationale is:
1 - We want an easy way to enable and/or disable a feature for the whole
workspace;
2 - But we want to allow projects to overwrite these features, when
needed.
These two are the suggested approach to configure experimental features,
but we also want to allow someone to easily turn on or off features to
test something in particular. For that one can use the CLI flags; and in
the specific case one does not have access to the CLI, for example, when
one of the `SDK` is compiling the code, one can still completely
overwrite the `TOML` files using environment variables.
We are also applying the "no experimental" first. This is to allow the
use case of exactly controlling which features will be enabled and
disabled. In this case one can use the "meta-feature" `all` to disable
everything and enable only the desired features. For example:
```
FORC_NO_EXPERIMENTAL=all FORC_EXPERIMENTAL=new_encoding your_command....
```
## Test for "-h"
This PR also introduces snapshot tests for "-h" for all forc commands at
https://github.com/FuelLabs/sway/pull/6586/files#diff-20a5e677d2ae9266a2247739b6a473d2ad0c627955187d668822e7d194f8e940
## Multiple test.toml
This PR also enables the use of multiple `test.toml`. Now each
`test.toml` will be a different test, allowing the same code to be
compiled and asserted differently.
For example, `main_args_empty` has two files: the normal `test.toml` and
a new `test.encoding_v1.toml`. One has `new_encoding` enabled and the
other disabled.
When a `test.toml` file has the `experimental` field, we do not use
fields with `_new_encoding` suffix anymore, making these files simpler:
test.toml:
```toml
category = "run"
# (1336, 1)
script_data = "0000000000000538 0000000000000001"
expected_result = { action = "return", value = 1337 }
validate_abi = true
experimental = { encoding_v1 = false }
```
test.encoding_v1.toml
```
script_data = "0000000000000538 0000000000000001"
expected_result = { action = "return_data", value = "0000000000000539" }
experimental = { new_encoding = true }
```
Test tomls with a suffix use `test.toml` as a base. So we do not need to
repeat every field in them.
Now when we run a test, we will see two tests instead of just one:
```
...
Testing should_pass/language/main_args/main_args_empty (test.encoding_v1.toml) ... ok
Testing should_pass/language/main_args/main_args_empty (test.toml) ... ok
...
```
## Conditional compilation
It is also possible to use conditional compilation using these
experimental features. Examples can be found inside `sway-lib-std`.
```sway
#[cfg(experimental_new_encoding = true)]
pub fn assert_eq<T>(v1: T, v2: T) {
...
}
```
## Checklist
- [ ] I have linked to any relevant issues.
- [ ] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [ ] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [ ] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [ ] I have requested a review from the relevant team or maintainers.
---------
Co-authored-by: IGI-111 <igi-111@protonmail.com>
## Description
For #6179, PR #6501 kept bumping into errors as it was doing too many
things, so I've split that PR into multiple PRs. This is the first, and
the only thing it does is move all of the various `Cargo.toml`
dependencies into the single workspace `Cargo.toml`.
Future PRs will:
- Update dependency versions
- Update code that breaks from the version bumps
## Checklist
- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [x] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [x] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [x] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
## Description
To fix the Numeric type propagation we now do the type checking of code
blocks twice, first to collect all the unification and second to unify
the types of a previous namespace with the variable declarations types
of the current namespace.
This change incurs a performance drop in the compilation time. The
benchmark `compile` takes 25% more time to run.
This can be improved significantly by using LexicalScopes instead of
cloning the whole namespace on each scope.
Fixes#6371
## Checklist
- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [x] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
---------
Co-authored-by: IGI-111 <igi-111@protonmail.com>
## Description
To be merged after https://github.com/FuelLabs/sway/pull/6250
## Checklist
- [ ] I have linked to any relevant issues.
- [ ] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [ ] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [ ] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [ ] I have requested a review from the relevant team or maintainers.
---------
Co-authored-by: Brandon Kite <brandonkite92@gmail.com>
Co-authored-by: IGI-111 <igi-111@protonmail.com>
## Description
Fix https://github.com/FuelLabs/sway/issues/6342 and
https://github.com/FuelLabs/sway/issues/6370.
https://github.com/FuelLabs/sway/issues/6342 occurs because when the
type checker unifies `Numeric` with an unsigned integer, it never checks
if the original literal actually fits into the new "unified" type.
https://github.com/FuelLabs/sway/issues/6370 happens because we never go
back to the array and check if every item actually is valid.
This PR is now doing these two tests at `type_check_analyze`.
## Checklist
- [ ] I have linked to any relevant issues.
- [ ] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [ ] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [ ] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [ ] I have requested a review from the relevant team or maintainers.
---------
Co-authored-by: Joshua Batty <joshpbatty@gmail.com>
## Description
Namespace annotation is no longer supported and has been replaced by
namespacing syntax.
## Checklist
- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [x] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [x] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [x] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
## Description
Includes critical fix https://github.com/FuelLabs/sway/pull/6212
## Checklist
- [ ] I have linked to any relevant issues.
- [ ] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [ ] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [ ] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [ ] I have requested a review from the relevant team or maintainers.
## Description
With this change storage keys will have a hash based on:
`sha256("storage::<storage_namespace_name1>::<storage_namespace_name2>.<storage_field_name>"`
or a given key.
A key can be provided by using the `in` keyword in a storage field as
such:
```
storage {
a in 0x02dac99c283f16bc91b74f6942db7f012699a2ad51272b15207b9cc14a70dbae: u64
}
```
Or by using a const expression:
```
const C1 = 0x02dac99c283f16bc91b74f6942db7f012699a2ad51272b15207b9cc14a70dbae;
storage {
a in C1: u64
}
```
StorageKey.field_id is now based on:
`sha256("storage::<storage_namespace_name1>::<storage_namespace_name2>.<storage_field_name>.<struct_field_name1>.<struct_field_name2>")`
Nested storage namespaces such as the following are now supported.
```
storage {
my_namespace1 {
my_namespace2 {
b:u64 = 1,
}
}
}
```
A deprecated warning is now thrown when the attribute `#[namespace(ns)]`
is used.
A warning is thrown when storage fields use the same storage slot key.
The warning looks like this:
```
Two storage fields are using the same storage key.
First field: storage::ns1.f3
Second field: storage::ns2.f4
Key: 5F4C20CE4BD128E5393A4C2B82007DAC795FA0006D01ACF8DB4C42632BC680CA
```
This PR does not include yet the necessary documentation updates.
Created dev-rel documentation request:
https://github.com/FuelLabs/devrel-requests/issues/14
## Checklist
- [ ] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [x] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [x] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
---------
Co-authored-by: IGI-111 <igi-111@protonmail.com>
## Description
This PR fixes a problem with constant folding optimization pass with the
mod operator. Unfortunately, we were not checking that the RHS was not
zero. So expressions like `mod 1 0` were packing the compiler.
## Checklist
- [ ] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [x] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [x] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [ ] I have requested a review from the relevant team or maintainers.
## Description
Currently we have the ability to cache programs and also clear these
programs from the engines using a garbage collector. The current
terminology refers to modules rather than programs. I want to implement
more granular caching and GC at the module level rather than just at the
program level. This helps clear up what is actually being cached and
cleared from the GC.
## Checklist
- [ ] I have linked to any relevant issues.
- [ ] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [ ] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [ ] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [ ] I have requested a review from the relevant team or maintainers.
Co-authored-by: IGI-111 <igi-111@protonmail.com>
## Description
This PR is part of https://github.com/FuelLabs/sway/issues/5727 and
fixes a problem with AbiEncode/AbiDecode code generation. The problem
lies in the fact that to encode/decode structs we need to "touch" all
fields. Which makes all fields to pass DCA.
Now, when a `StructFieldAccess` lives inside an autogenerated source,
the "code flow graph" will be built as usual, but no edge will be
created to the struct field, nor to the field owner struct declaration.
## Checklist
- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [x] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [x] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [x] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
---------
Co-authored-by: Joshua Batty <joshpbatty@gmail.com>
## Description
This PR implements `Display` and `DisplayWithEngines` for some more
types to help with debugging.
## Checklist
- [ ] I have linked to any relevant issues.
- [ ] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [ ] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
## Description
Switching from `std::sync::RwLock` to `parking_lot::RwLock` is giving us
a 10% further performance increase.
before: `13.459551958s`
after: `12.125325542s`
## Checklist
- [ ] I have linked to any relevant issues.
- [ ] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [ ] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [ ] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [ ] I have requested a review from the relevant team or maintainers.
## Description
There's still a ton more to do but going to open this to get merged to
avoid conflicts. We can possibly add in some of these rules into CI in a
future PR.
## Checklist
- [ ] I have linked to any relevant issues.
- [ ] I have commented my code, particularly in hard-to-understand
areas.
- [ ] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [ ] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [ ] I have added tests that prove my fix is effective or that my
feature works.
- [ ] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [ ] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [ ] I have requested a review from the relevant team or maintainers.
## Description
Fixes a critical bug where the server would crash if compilation was
triggered from anything other than the root `main.sw` file. This was due
to the metrics_map using the `SourceId` rather then the `ModuleId` as
the key.
This was introduced last week in #5813 as we now need to metrics to
decide if the engines should be mem swapped or discarded.
closes#5870
## Checklist
- [x] I have linked to any relevant issues.
- [x] I have commented my code, particularly in hard-to-understand
areas.
- [x] I have updated the documentation where relevant (API docs, the
reference, and the Sway book).
- [x] If my change requires substantial documentation changes, I have
[requested support from the DevRel
team](https://github.com/FuelLabs/devrel-requests/issues/new/choose)
- [x] I have added tests that prove my fix is effective or that my
feature works.
- [x] I have added (or requested a maintainer to add) the necessary
`Breaking*` or `New Feature` labels where relevant.
- [x] I have done my best to ensure that my PR adheres to [the Fuel Labs
Code Review
Standards](https://github.com/FuelLabs/rfcs/blob/master/text/code-standards/external-contributors.md).
- [x] I have requested a review from the relevant team or maintainers.
