We were only doing the new output on the top level install (`deno
install` with no args or flags), we now do it on `deno install
npm:chalk`, `deno add npm:chalk`, `deno cache ./foo.ts`, "deno install
--entrypoint ./foo.ts"`.
Additionally the scripts and deprecation warnings were printing above
the output, now they're deferred and displayed below
Fixes https://github.com/denoland/deno/issues/29707.
Not particularly happy with this one, but the only other idea I had was
to just ignore errors about non-existent exports when we build the graph
for a top level install. That might end up being the best solution.
For background, when you do a top level install that includes a jsr
dependency, we fetch the exports for the jsr package and then use those
exports as roots in the module graph (which triggers caching those
files).
To find those exports correctly, we need to know what version will end
up being cached, otherwise we may end up trying to reference exports
that don't exist on the actual package.
Previously, we could just look up the version in the lockfile and that
worked, but it turns out that was depending on the behavior that was
reverted in https://github.com/denoland/deno/pull/29642.
These tests are failing randomly on Windows at the moment. The PR that
added them was released in Deno v2.2.10 and users haven't reported any
problems so far. So temporarily ignoring these tests to unblock another
patch release.
Fixes#28891
We were checking if the node_modules entry for the package was present,
but then reading from the global cache.
Instead, read from the package.json in node_modules. As a fallback(which
in theory should only really happen if the node_modules dir is somehow
messed up), take the more expensive (but likely to work) path of reading
from the registry.json.
Apparently things like the `bin` field can appear in the version info
from the registry, but not the package's `package.json`. I'm still not
sure how you actually achieve this, but it's the case for
`esbuild-wasm`. This fixes the following panic:
```
❯ deno i --node-modules-dir npm:esbuild-wasm
Add npm:esbuild-wasm@0.25.2
Initialize ⣯ [00:00]
- esbuild-wasm@0.25.2
============================================================
Deno has panicked. This is a bug in Deno. Please report this
at https://github.com/denoland/deno/issues/new.
If you can reliably reproduce this panic, include the
reproduction steps and re-run with the RUST_BACKTRACE=1 env
var set and include the backtrace in your report.
Platform: macos aarch64
Version: 2.2.8+58c6c0b
Args: ["deno", "i", "--node-modules-dir", "npm:esbuild-wasm"]
View stack trace at:
https://panic.deno.com/v2.2.8+58c6c0bc9c1b4ee08645be936ff9268f17028f0f/aarch64-apple-darwin/g4h6Jo393pB4k4kqBo-3kqBg6klqBogtyLg13yLw_t0Lw549Hgj8-Hgw__H428-F4yv_HgjkpKww7gIon4gIw54rKwi5MorzMw5y7G42g7Iw---I40s-I4vu4Jw2rEw8z7Dwnr6J4tp7Bo_vvK
thread 'main' panicked at cli/npm/installer/common/bin_entries.rs:108:30:
called `Option::unwrap()` on a `None` value
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
```
Fixes#27264. Fixes https://github.com/denoland/deno/issues/28161.
Currently the new lockfile version is gated behind an unstable flag
(`--unstable-lockfile-v5`) until the next minor release, where it will
become the default.
The main motivation here is that it improves startup performance when
using the global cache or `--node-modules-dir=auto`.
In a create-next-app project, running an empty file:
```
❯ hyperfine --warmup 25 -N --setup "rm -f deno.lock" "deno run --node-modules-dir=auto -A empty.js" "deno-this-pr run --node-modules-dir=auto -A empty.js" "deno-this-pr run --node-modules-dir=auto --unstable-lockfile-v5 empty.js" "deno run --node-modules-dir=manual -A empty.js" "deno-this-pr run --node-modules-dir=manual -A empty.js"
Benchmark 1: deno run --node-modules-dir=auto -A empty.js
Time (mean ± σ): 247.6 ms ± 1.7 ms [User: 228.7 ms, System: 19.0 ms]
Range (min … max): 245.5 ms … 251.5 ms 12 runs
Benchmark 2: deno-this-pr run --node-modules-dir=auto -A empty.js
Time (mean ± σ): 169.8 ms ± 1.0 ms [User: 152.9 ms, System: 17.9 ms]
Range (min … max): 168.9 ms … 172.5 ms 17 runs
Benchmark 3: deno-this-pr run --node-modules-dir=auto --unstable-lockfile-v5 empty.js
Time (mean ± σ): 16.2 ms ± 0.7 ms [User: 12.3 ms, System: 5.7 ms]
Range (min … max): 15.2 ms … 19.2 ms 185 runs
Benchmark 4: deno run --node-modules-dir=manual -A empty.js
Time (mean ± σ): 16.2 ms ± 0.8 ms [User: 11.6 ms, System: 5.5 ms]
Range (min … max): 14.9 ms … 19.7 ms 187 runs
Benchmark 5: deno-this-pr run --node-modules-dir=manual -A empty.js
Time (mean ± σ): 16.0 ms ± 0.9 ms [User: 12.0 ms, System: 5.5 ms]
Range (min … max): 14.8 ms … 22.3 ms 190 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Summary
deno-this-pr run --node-modules-dir=manual -A empty.js ran
1.01 ± 0.08 times faster than deno run --node-modules-dir=manual -A empty.js
1.01 ± 0.07 times faster than deno-this-pr run --node-modules-dir=auto --unstable-lockfile-v5 empty.js
10.64 ± 0.60 times faster than deno-this-pr run --node-modules-dir=auto -A empty.js
15.51 ± 0.88 times faster than deno run --node-modules-dir=auto -A empty.js
```
When using the new lockfile version, this leads to a 15.5x faster
startup time compared to the current deno version.
Install times benefit as well, though to a lesser degree.
`deno install` on a create-next-app project, with everything cached
(just setting up node_modules from scratch):
```
❯ hyperfine --warmup 5 -N --prepare "rm -rf node_modules" --setup "rm -rf deno.lock" "deno i" "deno-this-pr i" "deno-this-pr i --unstable-lockfile-v5"
Benchmark 1: deno i
Time (mean ± σ): 464.4 ms ± 8.8 ms [User: 227.7 ms, System: 217.3 ms]
Range (min … max): 452.6 ms … 478.3 ms 10 runs
Benchmark 2: deno-this-pr i
Time (mean ± σ): 368.8 ms ± 22.0 ms [User: 150.8 ms, System: 198.1 ms]
Range (min … max): 344.8 ms … 397.6 ms 10 runs
Benchmark 3: deno-this-pr i --unstable-lockfile-v5
Time (mean ± σ): 211.9 ms ± 17.1 ms [User: 7.1 ms, System: 177.2 ms]
Range (min … max): 191.3 ms … 233.4 ms 10 runs
Summary
deno-this-pr i --unstable-lockfile-v5 ran
1.74 ± 0.17 times faster than deno-this-pr i
2.19 ± 0.18 times faster than deno i
```
With lockfile v5, a 2.19x faster install time compared to the current
deno.
This improves peer dependency resolution to be more relaxed and resolve
non-version matching ancestors similar to pnpm rather than introducing
duplicate dependencies. Deno will warn when this occurs. In the future,
we should look into introducing an option to have Deno error in this
scenario.
Fixes https://github.com/denoland/deno/issues/28223
This is kind of an ugly fix, but it works, and I think is the easiest
way to handle the fact that when caching the module graph we might
encounter imports that won't actually error at runtime (for instance in
files that will be bundled).
Currently deno eagerly caches all npm packages in the workspace's npm
resolution. So, for instance, running a file `foo.ts` that imports
`npm:chalk` will also install all dependencies listed in `package.json`
and all `npm` dependencies listed in the lockfile.
This PR refactors things to give more control over when and what npm
packages are automatically cached while building the module graph.
After this PR, by default the current behavior is unchanged _except_ for
`deno install --entrypoint`, which will only cache npm packages used by
the given entrypoint. For the other subcommands, this behavior can be
enabled with `--unstable-npm-lazy-caching`
Fixes#25782.
---------
Signed-off-by: Nathan Whitaker <17734409+nathanwhit@users.noreply.github.com>
Co-authored-by: Luca Casonato <hello@lcas.dev>
This will respect `"type": "commonjs"` in a package.json to determine if
`.js`/`.jsx`/`.ts`/.tsx` files are CJS or ESM. If the file is found to
be ESM it will be loaded as ESM though.
Fixes https://github.com/denoland/deno/issues/26509.
Ended up being a `deno_graph` bug causing the error to surface. This PR
updates `deno_graph` to pick up the fix and reverts the temporary
workaround that skipped JSON exports.
Fixes#26498.
This was a sort of intentional decision originally, as I wanted to avoid
caching extra files that may not be needed. It seems like that behavior
is unintuitive, so I propose we cache all of the exports of listed jsr
packages when you run a bare `deno install`.
1. Respects the formatting of the file (ex. keeps four space indents or
tabs).
2. Handles editing of comments.
3. Handles trailing commas.
4. Code is easier to maintain.
Fixes#25998. Fixes https://github.com/denoland/deno/issues/25928.
Originally I was just going to make this an error message instead of a
panic, but once I got to a minimal repro I felt that this really should
work.
The panic occurs when you have `nodeModulesDir: manual` (or a
package.json present), and you have an npm package with a tag in your
deno.json (see the spec test that illustrates this).
This code path only actually executes when trying to choose an
appropriate package version from `node_modules/.deno`, so we should be
able to fix it by storing some extra data at install time.
The fix proposed here is to repurpose the `.initialized` file that we
store in `node_modules` to store the tags associated with a package.
Basically, if you have a version requirement with a tag (e.g.
`npm:chalk@latest`), when we set up the node_modules folder for that
package, we store the tag (`latest`) in `.initialized`. Then, when doing
BYONM resolution, if we have a version requirement with a tag, we read
that file and check if the tag is present.
The downside is that we do more work when setting up `node_modules`. We
_could_ do this only when BYONM is enabled, but that would have the
downside of needing to re-run `deno install` when you switch from auto
-> manual, though maybe that's not a big deal.
Fixes#25861.
Previously we were attempting to match the version requirement against
the version already present in `node_modules` root, and if they didn't
match we would create a node_modules dir in the workspace member's
directory with the dependency.
Aside from the fact that this caused the panic, on second thought it
just doesn't make sense in general. We shouldn't be semver matching, as
resolution has already occurred and decided what package versions are
required. Instead, we can just compare the versions directly.
This replaces `--allow-net` for import permissions and makes the
security sandbox stricter by also checking permissions for statically
analyzable imports.
By default, this has a value of
`--allow-import=deno.land:443,jsr.io:443,esm.sh:443,raw.githubusercontent.com:443,gist.githubusercontent.com:443`,
but that can be overridden by providing a different set of hosts.
Additionally, when no value is provided, import permissions are inferred
from the CLI arguments so the following works because
`fresh.deno.dev:443` will be added to the list of allowed imports:
```ts
deno run -A -r https://fresh.deno.dev
```
---------
Co-authored-by: David Sherret <dsherret@gmail.com>
Refactors the lifecycle scripts code to extract out the common
functionality and then uses that to provide a warning in the global
resolver.
While ideally we would still support them with the global cache, for now
a warning is at least better than the status quo (where people are
unaware why their packages aren't working).
Partially addresses https://github.com/denoland/deno/issues/25648.
This allows packages that use `crossws` to be installed with `deno
install`. `crossws` specifies an optional peer dependency on
`uWebSockets`, but `uWebSockets` is not on npm (it is used with `git:`
or `github:` specifiers). Previously we would error on this, now we
don't error on non-existent optional peer dependencies.
This was initially added in #25399 in order to make transitioning over
from package.json to deno.json more easy, but it causes some problems
that are shown in the issue and it also means that the output of `deno
install` would have different resolution than `npm install`. Overall, I
think it's too much complexity to be smarter about this and it's
probably best to not do it. If someone needs an aliased folder then they
should keep using a package.json
Closes#25538
This allows using npm deps of jsr deps without having to add them to the
root package.json.
Works by taking the package requirement and scanning the
`node_modules/.deno` directory for the best matching package, so it
relies on deno's node_modules structure.
Additionally to make the transition from package.json to deno.json
easier, Deno now:
1. Installs npm deps in a deno.json at the same time as installing npm
deps from a package.json.
2. Uses the alias in the import map for `node_modules/<alias>` for
better package.json compatiblity.
