This is the release commit being forwarded back to main for 2.2.11
---------
Co-authored-by: bartlomieju <bartlomieju@users.noreply.github.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This is the release commit being forwarded back to main for 2.2.10
Co-authored-by: nathanwhit <nathanwhit@users.noreply.github.com>
Co-authored-by: Nathan Whitaker <nathan@deno.com>
Fixes#27264. Fixes https://github.com/denoland/deno/issues/28161.
Currently the new lockfile version is gated behind an unstable flag
(`--unstable-lockfile-v5`) until the next minor release, where it will
become the default.
The main motivation here is that it improves startup performance when
using the global cache or `--node-modules-dir=auto`.
In a create-next-app project, running an empty file:
```
❯ hyperfine --warmup 25 -N --setup "rm -f deno.lock" "deno run --node-modules-dir=auto -A empty.js" "deno-this-pr run --node-modules-dir=auto -A empty.js" "deno-this-pr run --node-modules-dir=auto --unstable-lockfile-v5 empty.js" "deno run --node-modules-dir=manual -A empty.js" "deno-this-pr run --node-modules-dir=manual -A empty.js"
Benchmark 1: deno run --node-modules-dir=auto -A empty.js
Time (mean ± σ): 247.6 ms ± 1.7 ms [User: 228.7 ms, System: 19.0 ms]
Range (min … max): 245.5 ms … 251.5 ms 12 runs
Benchmark 2: deno-this-pr run --node-modules-dir=auto -A empty.js
Time (mean ± σ): 169.8 ms ± 1.0 ms [User: 152.9 ms, System: 17.9 ms]
Range (min … max): 168.9 ms … 172.5 ms 17 runs
Benchmark 3: deno-this-pr run --node-modules-dir=auto --unstable-lockfile-v5 empty.js
Time (mean ± σ): 16.2 ms ± 0.7 ms [User: 12.3 ms, System: 5.7 ms]
Range (min … max): 15.2 ms … 19.2 ms 185 runs
Benchmark 4: deno run --node-modules-dir=manual -A empty.js
Time (mean ± σ): 16.2 ms ± 0.8 ms [User: 11.6 ms, System: 5.5 ms]
Range (min … max): 14.9 ms … 19.7 ms 187 runs
Benchmark 5: deno-this-pr run --node-modules-dir=manual -A empty.js
Time (mean ± σ): 16.0 ms ± 0.9 ms [User: 12.0 ms, System: 5.5 ms]
Range (min … max): 14.8 ms … 22.3 ms 190 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Summary
deno-this-pr run --node-modules-dir=manual -A empty.js ran
1.01 ± 0.08 times faster than deno run --node-modules-dir=manual -A empty.js
1.01 ± 0.07 times faster than deno-this-pr run --node-modules-dir=auto --unstable-lockfile-v5 empty.js
10.64 ± 0.60 times faster than deno-this-pr run --node-modules-dir=auto -A empty.js
15.51 ± 0.88 times faster than deno run --node-modules-dir=auto -A empty.js
```
When using the new lockfile version, this leads to a 15.5x faster
startup time compared to the current deno version.
Install times benefit as well, though to a lesser degree.
`deno install` on a create-next-app project, with everything cached
(just setting up node_modules from scratch):
```
❯ hyperfine --warmup 5 -N --prepare "rm -rf node_modules" --setup "rm -rf deno.lock" "deno i" "deno-this-pr i" "deno-this-pr i --unstable-lockfile-v5"
Benchmark 1: deno i
Time (mean ± σ): 464.4 ms ± 8.8 ms [User: 227.7 ms, System: 217.3 ms]
Range (min … max): 452.6 ms … 478.3 ms 10 runs
Benchmark 2: deno-this-pr i
Time (mean ± σ): 368.8 ms ± 22.0 ms [User: 150.8 ms, System: 198.1 ms]
Range (min … max): 344.8 ms … 397.6 ms 10 runs
Benchmark 3: deno-this-pr i --unstable-lockfile-v5
Time (mean ± σ): 211.9 ms ± 17.1 ms [User: 7.1 ms, System: 177.2 ms]
Range (min … max): 191.3 ms … 233.4 ms 10 runs
Summary
deno-this-pr i --unstable-lockfile-v5 ran
1.74 ± 0.17 times faster than deno-this-pr i
2.19 ± 0.18 times faster than deno i
```
With lockfile v5, a 2.19x faster install time compared to the current
deno.
This is the release commit being forwarded back to main for 2.2.7
Signed-off-by: Divy Srivastava <dj.srivastava23@gmail.com>
Co-authored-by: Divy Srivastava <dj.srivastava23@gmail.com>
Fixes#28517.
The npm package info gets requested a bunch of times by deno_npm. Before
this PR, we were loading it from the FS and parsing it each and every
time. With a lot of dependencies (and large `registry.json` files), this
can lead to massive blowups in install times.
From the repro in #28517
before this PR:
```
Command being timed: "deno i"
User time (seconds): 538.54
System time (seconds): 56.49
Percent of CPU this job got: 198%
Elapsed (wall clock) time (h:mm:ss or m:ss): 4:59.45
Maximum resident set size (kbytes): 378976
```
this PR:
```
Command being timed: "deno-this-pr i"
User time (seconds): 1.29
System time (seconds): 1.56
Percent of CPU this job got: 68%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:04.16
Maximum resident set size (kbytes): 500864
```
So roughly an improvement from 339s to 4s. You can see that the max RSS
does increase a decent amount, which is the main downside. However, this
in memory cache is cleared once we're done caching npm packages, and IMO
the performance tradeoff is well worth it.
This also has a very noticable, though less drastic, effect on fresh
installs (no deno.lock) for smaller projects. Here's a clean nextJS
template project:
```
❯ hyperfine --warmup 5 --prepare "rm -rf node_modules deno.lock" "deno i" "deno-this-pr i"
Benchmark 1: deno
Time (mean ± σ): 765.0 ms ± 10.1 ms [User: 622.3 ms, System: 216.4 ms]
Range (min … max): 749.0 ms … 783.6 ms 10 runs
Benchmark 2: deno-this-pr
Time (mean ± σ): 357.2 ms ± 9.4 ms [User: 193.2 ms, System: 198.2 ms]
Range (min … max): 346.4 ms … 374.1 ms 10 runs
Summary
deno-this-pr ran
2.14 ± 0.06 times faster than deno
```
NOTE: Commit 27363d389 was incorrectly landed in main before the release
completed and is not included in v2.2.5. The official v2.2.5 release was made
from the v2.2 branch.
This adds support for using a local copy of an npm package.
```js
// deno.json
{
"patch": [
"../path/to/local_npm_package"
],
// required until Deno 2.3, but it will still be considered unstable
"unstable": ["npm-patch"]
}
```
1. Requires using a node_modules folder.
2. When using `"nodeModulesDir": "auto"`, it recreates the folder in the
node_modules directory on each run which will slightly increase startup
time.
3. When using the default with a package.json (`"nodeModulesDir":
"manual"`), updating the package requires running `deno install`. This
is to get the package into the node_modules directory of the current
workspace. This is necessary instead of linking because packages can
have multiple "copy packages" due to peer dep resolution.
Caveat: Specifying a local copy of an npm package or making changes to
its dependencies will purge npm packages from the lockfile. This might
cause npm resolution to resolve differently and it may end up not using
the local copy of the npm package. It's very difficult to only
invalidate resolution midway through the graph and then only rebuild
that part of the resolution, so this is just a first pass that can be
improved in the future. In practice, this probably won't be an issue for
most people.
Another limitation is this also requires the npm package name to exist
in the registry at the moment.
This is the release commit being forwarded back to main for 2.1.9
Co-authored-by: bartlomieju <bartlomieju@users.noreply.github.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
This is the release commit being forwarded back to main for 2.1.8
Co-authored-by: bartlomieju <bartlomieju@users.noreply.github.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Allows easily constructing a `DenoResolver` using the exact same logic
that we use in the CLI (useful for dnt and for external bundlers). This
code is then used in the CLI to ensure the logic is always up-to-date.
```rs
use std::rc::Rc;
use deno_resolver:🏭:ResolverFactory;
use deno_resolver:🏭:WorkspaceFactory;
use sys_traits::impls::RealSys;
let sys = RealSys;
let cwd = sys.env_current_dir()?;
let workspace_factory = Rc::new(WorkspaceFactory::new(sys, cwd, Default::default()));
let resolver_factory = ResolverFactory::new(workspace_factory.clone(), Default::default());
let deno_resolver = resolver_factory.deno_resolver().await?;
```
This is the release commit being forwarded back to main for 2.1.7
Please ensure:
- [x] Everything looks ok in the PR
- [x] The release has been published
To make edits to this PR:
```shell
git fetch upstream forward_v2.1.7 && git checkout -b forward_v2.1.7 upstream/forward_v2.1.7
```
Don't need this PR? Close it.
cc @crowlKats
---------
Co-authored-by: crowlKats <crowlKats@users.noreply.github.com>
Co-authored-by: crowlkats <crowlkats@toaxl.com>
This slightly degrades the performance of CJS export analysis on
subsequent runs because I changed it to no longer cache in the DENO_DIR
with this PR (denort now properly has no idea about the DENO_DIR). We'll
have to change it to embed this data in the binary and that will also
allow us to get rid of swc in denort (will do that in a follow-up PR).
This is the release commit being forwarded back to main for 2.1.6
Co-authored-by: bartlomieju <bartlomieju@users.noreply.github.com>
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Extracting out more code from the CLI for reuse elsewhere (still more
work to do, but this is a start).
This is the code for extracting npm tarballs and saving information in
the npm cache in the global deno_dir.
