Fixes two issues:
- If a cached packument was out of date and missing a version from the
lockfile, we would fail. Instead we should try again with a forced
re-fetch
- We weren't threading through the workspace patch packages correctly
These tests are failing randomly on Windows at the moment. The PR that
added them was released in Deno v2.2.10 and users haven't reported any
problems so far. So temporarily ignoring these tests to unblock another
patch release.
This PR updates the behavior of `deno test --coverage` option. Now if
`--coverage` option is specified, `deno test` command automatically
shows summary report in the terminal, and generates the lcov report in
`$coverage_dir/lcov.info` and html report in `$coverage_dir/html/`
This change also adds `--coverage-raw-data-only` flag, which prevents
the above reports generated, instead only generates the raw json
coverage data (which is the same as current behavior)
This extracts out the shared libraries and `.node` native modules to a
temp file and opens them from there. **This means that this
implementation will not work in every scenario.** For example, a library
could require other files that only exist in the in-memory file system.
To solve that, we'll introduce
https://github.com/denoland/deno/issues/28918 later or adapt this
solution to solve more issues.
Additionally, this will not work when run on readonly file systems.
The bug has been fixed by updating the regex to ensure that the shebang
characters (#!) are no longer excluded. This allows the shebang to be
retained and parsed correctly.
---------
Co-authored-by: David Sherret <dsherret@users.noreply.github.com>
Fixes#28891
We were checking if the node_modules entry for the package was present,
but then reading from the global cache.
Instead, read from the package.json in node_modules. As a fallback(which
in theory should only really happen if the node_modules dir is somehow
messed up), take the more expensive (but likely to work) path of reading
from the registry.json.
Ref https://github.com/denoland/deno/issues/28836
This PR replaces the _stream.mjs bundle with a file-by-file port instead. A codemod transpiles Node.js internals to ESM. The codemod performs three tasks: translating CJS to ESM, remapping internal dependencies, and hoisting lazy requires as imports.
The process is fully automated through the `update_node_stream.ts` script, simplifying future internal updates. The script checks out Node.js from a specific tag defined in the `tests/node_compat/runner`.
Additionally, the update enables new tests in our Node test runner and adds features (like compose()) that were missing from the outdated bundle.
## Performance
There is a 140KB+ binary size increase on aarch64-apple-darwin and nop startup time stays the same.
Apparently things like the `bin` field can appear in the version info
from the registry, but not the package's `package.json`. I'm still not
sure how you actually achieve this, but it's the case for
`esbuild-wasm`. This fixes the following panic:
```
❯ deno i --node-modules-dir npm:esbuild-wasm
Add npm:esbuild-wasm@0.25.2
Initialize ⣯ [00:00]
- esbuild-wasm@0.25.2
============================================================
Deno has panicked. This is a bug in Deno. Please report this
at https://github.com/denoland/deno/issues/new.
If you can reliably reproduce this panic, include the
reproduction steps and re-run with the RUST_BACKTRACE=1 env
var set and include the backtrace in your report.
Platform: macos aarch64
Version: 2.2.8+58c6c0b
Args: ["deno", "i", "--node-modules-dir", "npm:esbuild-wasm"]
View stack trace at:
https://panic.deno.com/v2.2.8+58c6c0bc9c1b4ee08645be936ff9268f17028f0f/aarch64-apple-darwin/g4h6Jo393pB4k4kqBo-3kqBg6klqBogtyLg13yLw_t0Lw549Hgj8-Hgw__H428-F4yv_HgjkpKww7gIon4gIw54rKwi5MorzMw5y7G42g7Iw---I40s-I4vu4Jw2rEw8z7Dwnr6J4tp7Bo_vvK
thread 'main' panicked at cli/npm/installer/common/bin_entries.rs:108:30:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
This commit enabled [explicit resource management]
(https://github.com/tc39/proposal-explicit-resource-management)
proposal for JavaScript code.
This is done by upgrading `deno_ast` to a version that no longer
transpiles TS files with `using` keyword, and instead enables
a V8 flag that provides native support.
Closes https://github.com/denoland/deno/issues/20821
---------
Co-authored-by: Luca Casonato <hello@lcas.dev>
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
Fixes#27264. Fixes https://github.com/denoland/deno/issues/28161.
Currently the new lockfile version is gated behind an unstable flag
(`--unstable-lockfile-v5`) until the next minor release, where it will
become the default.
The main motivation here is that it improves startup performance when
using the global cache or `--node-modules-dir=auto`.
In a create-next-app project, running an empty file:
```
❯ hyperfine --warmup 25 -N --setup "rm -f deno.lock" "deno run --node-modules-dir=auto -A empty.js" "deno-this-pr run --node-modules-dir=auto -A empty.js" "deno-this-pr run --node-modules-dir=auto --unstable-lockfile-v5 empty.js" "deno run --node-modules-dir=manual -A empty.js" "deno-this-pr run --node-modules-dir=manual -A empty.js"
Benchmark 1: deno run --node-modules-dir=auto -A empty.js
Time (mean ± σ): 247.6 ms ± 1.7 ms [User: 228.7 ms, System: 19.0 ms]
Range (min … max): 245.5 ms … 251.5 ms 12 runs
Benchmark 2: deno-this-pr run --node-modules-dir=auto -A empty.js
Time (mean ± σ): 169.8 ms ± 1.0 ms [User: 152.9 ms, System: 17.9 ms]
Range (min … max): 168.9 ms … 172.5 ms 17 runs
Benchmark 3: deno-this-pr run --node-modules-dir=auto --unstable-lockfile-v5 empty.js
Time (mean ± σ): 16.2 ms ± 0.7 ms [User: 12.3 ms, System: 5.7 ms]
Range (min … max): 15.2 ms … 19.2 ms 185 runs
Benchmark 4: deno run --node-modules-dir=manual -A empty.js
Time (mean ± σ): 16.2 ms ± 0.8 ms [User: 11.6 ms, System: 5.5 ms]
Range (min … max): 14.9 ms … 19.7 ms 187 runs
Benchmark 5: deno-this-pr run --node-modules-dir=manual -A empty.js
Time (mean ± σ): 16.0 ms ± 0.9 ms [User: 12.0 ms, System: 5.5 ms]
Range (min … max): 14.8 ms … 22.3 ms 190 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Summary
deno-this-pr run --node-modules-dir=manual -A empty.js ran
1.01 ± 0.08 times faster than deno run --node-modules-dir=manual -A empty.js
1.01 ± 0.07 times faster than deno-this-pr run --node-modules-dir=auto --unstable-lockfile-v5 empty.js
10.64 ± 0.60 times faster than deno-this-pr run --node-modules-dir=auto -A empty.js
15.51 ± 0.88 times faster than deno run --node-modules-dir=auto -A empty.js
```
When using the new lockfile version, this leads to a 15.5x faster
startup time compared to the current deno version.
Install times benefit as well, though to a lesser degree.
`deno install` on a create-next-app project, with everything cached
(just setting up node_modules from scratch):
```
❯ hyperfine --warmup 5 -N --prepare "rm -rf node_modules" --setup "rm -rf deno.lock" "deno i" "deno-this-pr i" "deno-this-pr i --unstable-lockfile-v5"
Benchmark 1: deno i
Time (mean ± σ): 464.4 ms ± 8.8 ms [User: 227.7 ms, System: 217.3 ms]
Range (min … max): 452.6 ms … 478.3 ms 10 runs
Benchmark 2: deno-this-pr i
Time (mean ± σ): 368.8 ms ± 22.0 ms [User: 150.8 ms, System: 198.1 ms]
Range (min … max): 344.8 ms … 397.6 ms 10 runs
Benchmark 3: deno-this-pr i --unstable-lockfile-v5
Time (mean ± σ): 211.9 ms ± 17.1 ms [User: 7.1 ms, System: 177.2 ms]
Range (min … max): 191.3 ms … 233.4 ms 10 runs
Summary
deno-this-pr i --unstable-lockfile-v5 ran
1.74 ± 0.17 times faster than deno-this-pr i
2.19 ± 0.18 times faster than deno i
```
With lockfile v5, a 2.19x faster install time compared to the current
deno.
This improves peer dependency resolution to be more relaxed and resolve
non-version matching ancestors similar to pnpm rather than introducing
duplicate dependencies. Deno will warn when this occurs. In the future,
we should look into introducing an option to have Deno error in this
scenario.
`import.meta.log` enables basic log filtering through
`env_logger`/`DENO_LOG`. Log levels are supported, and filenames can
also be used. for example: `DENO_LOG=ext:deno_http::00_serve.ts=warn`
This adds support for using a local copy of an npm package.
```js
// deno.json
{
"patch": [
"../path/to/local_npm_package"
],
// required until Deno 2.3, but it will still be considered unstable
"unstable": ["npm-patch"]
}
```
1. Requires using a node_modules folder.
2. When using `"nodeModulesDir": "auto"`, it recreates the folder in the
node_modules directory on each run which will slightly increase startup
time.
3. When using the default with a package.json (`"nodeModulesDir":
"manual"`), updating the package requires running `deno install`. This
is to get the package into the node_modules directory of the current
workspace. This is necessary instead of linking because packages can
have multiple "copy packages" due to peer dep resolution.
Caveat: Specifying a local copy of an npm package or making changes to
its dependencies will purge npm packages from the lockfile. This might
cause npm resolution to resolve differently and it may end up not using
the local copy of the npm package. It's very difficult to only
invalidate resolution midway through the graph and then only rebuild
that part of the resolution, so this is just a first pass that can be
improved in the future. In practice, this probably won't be an issue for
most people.
Another limitation is this also requires the npm package name to exist
in the registry at the moment.
