This was actually fixed via the deno_npm and lockfile updates, but it
would be good to have an explicit entry for this in the release notes so
labeling this as a "fix"
Closes https://github.com/denoland/deno/issues/27758
Fixes#27264. Fixes https://github.com/denoland/deno/issues/28161.
Currently the new lockfile version is gated behind an unstable flag
(`--unstable-lockfile-v5`) until the next minor release, where it will
become the default.
The main motivation here is that it improves startup performance when
using the global cache or `--node-modules-dir=auto`.
In a create-next-app project, running an empty file:
```
❯ hyperfine --warmup 25 -N --setup "rm -f deno.lock" "deno run --node-modules-dir=auto -A empty.js" "deno-this-pr run --node-modules-dir=auto -A empty.js" "deno-this-pr run --node-modules-dir=auto --unstable-lockfile-v5 empty.js" "deno run --node-modules-dir=manual -A empty.js" "deno-this-pr run --node-modules-dir=manual -A empty.js"
Benchmark 1: deno run --node-modules-dir=auto -A empty.js
Time (mean ± σ): 247.6 ms ± 1.7 ms [User: 228.7 ms, System: 19.0 ms]
Range (min … max): 245.5 ms … 251.5 ms 12 runs
Benchmark 2: deno-this-pr run --node-modules-dir=auto -A empty.js
Time (mean ± σ): 169.8 ms ± 1.0 ms [User: 152.9 ms, System: 17.9 ms]
Range (min … max): 168.9 ms … 172.5 ms 17 runs
Benchmark 3: deno-this-pr run --node-modules-dir=auto --unstable-lockfile-v5 empty.js
Time (mean ± σ): 16.2 ms ± 0.7 ms [User: 12.3 ms, System: 5.7 ms]
Range (min … max): 15.2 ms … 19.2 ms 185 runs
Benchmark 4: deno run --node-modules-dir=manual -A empty.js
Time (mean ± σ): 16.2 ms ± 0.8 ms [User: 11.6 ms, System: 5.5 ms]
Range (min … max): 14.9 ms … 19.7 ms 187 runs
Benchmark 5: deno-this-pr run --node-modules-dir=manual -A empty.js
Time (mean ± σ): 16.0 ms ± 0.9 ms [User: 12.0 ms, System: 5.5 ms]
Range (min … max): 14.8 ms … 22.3 ms 190 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Summary
deno-this-pr run --node-modules-dir=manual -A empty.js ran
1.01 ± 0.08 times faster than deno run --node-modules-dir=manual -A empty.js
1.01 ± 0.07 times faster than deno-this-pr run --node-modules-dir=auto --unstable-lockfile-v5 empty.js
10.64 ± 0.60 times faster than deno-this-pr run --node-modules-dir=auto -A empty.js
15.51 ± 0.88 times faster than deno run --node-modules-dir=auto -A empty.js
```
When using the new lockfile version, this leads to a 15.5x faster
startup time compared to the current deno version.
Install times benefit as well, though to a lesser degree.
`deno install` on a create-next-app project, with everything cached
(just setting up node_modules from scratch):
```
❯ hyperfine --warmup 5 -N --prepare "rm -rf node_modules" --setup "rm -rf deno.lock" "deno i" "deno-this-pr i" "deno-this-pr i --unstable-lockfile-v5"
Benchmark 1: deno i
Time (mean ± σ): 464.4 ms ± 8.8 ms [User: 227.7 ms, System: 217.3 ms]
Range (min … max): 452.6 ms … 478.3 ms 10 runs
Benchmark 2: deno-this-pr i
Time (mean ± σ): 368.8 ms ± 22.0 ms [User: 150.8 ms, System: 198.1 ms]
Range (min … max): 344.8 ms … 397.6 ms 10 runs
Benchmark 3: deno-this-pr i --unstable-lockfile-v5
Time (mean ± σ): 211.9 ms ± 17.1 ms [User: 7.1 ms, System: 177.2 ms]
Range (min … max): 191.3 ms … 233.4 ms 10 runs
Summary
deno-this-pr i --unstable-lockfile-v5 ran
1.74 ± 0.17 times faster than deno-this-pr i
2.19 ± 0.18 times faster than deno i
```
With lockfile v5, a 2.19x faster install time compared to the current
deno.
This adds support for using a local copy of an npm package.
```js
// deno.json
{
"patch": [
"../path/to/local_npm_package"
],
// required until Deno 2.3, but it will still be considered unstable
"unstable": ["npm-patch"]
}
```
1. Requires using a node_modules folder.
2. When using `"nodeModulesDir": "auto"`, it recreates the folder in the
node_modules directory on each run which will slightly increase startup
time.
3. When using the default with a package.json (`"nodeModulesDir":
"manual"`), updating the package requires running `deno install`. This
is to get the package into the node_modules directory of the current
workspace. This is necessary instead of linking because packages can
have multiple "copy packages" due to peer dep resolution.
Caveat: Specifying a local copy of an npm package or making changes to
its dependencies will purge npm packages from the lockfile. This might
cause npm resolution to resolve differently and it may end up not using
the local copy of the npm package. It's very difficult to only
invalidate resolution midway through the graph and then only rebuild
that part of the resolution, so this is just a first pass that can be
improved in the future. In practice, this probably won't be an issue for
most people.
Another limitation is this also requires the npm package name to exist
in the registry at the moment.
This adds support for installing `file:` dependencies in a local
package.json.
In order to use these, you must not set `--node-modules-dir=...` when
using a package.json and it should use the default of
`--node-modules-dir=manual`.
Closes https://github.com/denoland/deno/issues/18701
Shows directory import and missing extension suggestions in error
messages similar but not exactly to node.
Closes#26802
Co-authored-by: Hajime-san <Hajime-san@users.noreply.github.com>
This PR resolves 2 issues of Socket class of node compat (both are
related to playwright)
Currently `browser.launch()` of playwright is not working.
`browser.launch` opens PipeTransport (which is based on Pipe/IPC socket)
with the browser process. But that pipe doesn't start reading the data
because of the workaround #27662 (which pauses the socket at the
beginning if it's from playwright-core). This PR fixes this issue by
checking whether the given handle is `ipc` handle or not.
Another issue is that sock-init-workaround for TLS connection stopped
working at #27707 because of the changes of TLS socket initialization
steps. This change fixes the issue by correctly returning the function
in workaround path.
The added case `specs::npm::playwright_compat` checks both fixes with
actual playwright and playwright-core packages.
`browser.launch` issues
closes#16899closes#27623
`https.request` issue
closes#27658
See the comment
https://github.com/denoland/deno/pull/25470#issuecomment-2435077722 for
the reason why we do this workaround to make `make-fetch-happen` work in
Deno
This PR applies the same workaround to `npm-check-updates` package.
`npm-check-updates` internally uses
[`npm-registry-fetch`](https://www.npmjs.com/package/npm-registry-fetch)
which uses
[`make-fetch-happen`](https://www.npmjs.com/package/make-fetch-happen)
(the problematic package) for making http request to npm registry.
The detection of `make-fetch-happen` doesn't work for
`npm-check-updates` because we use call stack at `net.Socket`
constructor to check if it's called from `make-fetch-happen`, but
`npm-check-updates` bundles its dependency and the check doesn't work.
This PR adds the check of `npm-check-updates` string in call stack in
net.Socket constructor to trigger the workaroud.
closes#27629
This commit adds support for understanding "workpace:^"
and "workspace:~" version constraints in npm/pnpm workspaces.
This is done by upgrading various crates to their latest versions.
Closes https://github.com/denoland/deno/issues/26726
---------
Co-authored-by: David Sherret <dsherret@gmail.com>
Ensures a dynamic import in a CJS file will consider the referrer as an import for node resolution.
Also adds fixes (adds) support for `"resolution-mode"` in TypeScript.
The issue was this package had an import like: `".//index.js"` and we
resolved that as specified, but node normalizes it to `"./index.js"` so
we have to copy node.
This will respect `"type": "commonjs"` in a package.json to determine if
`.js`/`.jsx`/`.ts`/.tsx` files are CJS or ESM. If the file is found to
be ESM it will be loaded as ESM though.
Fixes#26677
Some packages (like supabase) declare bin entries that don't exist until
lifecycle scripts are run. For instance, the lifecycle script downloads
a binary file which serves as a bin entrypoint.
Unfortunately you can't just defer setting up the bin entries until
after lifecycle scripts have run, because the scripts may rely on them.
I looked into this, and PNPM just re-links bin entries after running
lifecycle scripts. I think that's about the best we can do as well.
Note that we'll only re-setup bin entries for packages whose lifecycle
scripts we run. This should limit the performance cost, as typically a
given project will not have many lifecycle scripts (and of those, many
of them probably don't have bin entries to set up).
* cts support
* better cjs/cts type checking
* deno compile cjs/cts support
* More efficient detect cjs (going towards stabilization)
* Determination of whether .js, .ts, .jsx, or .tsx is cjs or esm is only
done after loading
* Support `import x = require(...);`
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Fixes#25342.
Still not sure on the exact user agent to set (should it include
`node`?).
After this PR, here's the state of running some `create-*` packages
(just ones I could think of off the top of my head):
| package | prints/runs/suggests deno install | notes |
| ---------------- | ------------- | ------ |
| `create-next-app` | ❌ | falls back to npm, needs a PR
([code](c32e280209/packages/create-next-app/helpers/get-pkg-manager.ts (L3)))
| `sv create` | ❌ | uses `package-manager-detector`, needs a PR
([code](https://github.com/antfu-collective/package-manager-detector/tree/main))
| `create-qwik` | ✅ | runs `deno install` but suggests `deno start`
which doesn't work (should be `deno task start` or `deno run start`)
| `create-astro` | ✅ | runs `deno install` but suggests `npm run dev`
later in output, probably needs a PR
| `nuxi init` | ❌ | deno not an option in dialog, needs a PR
([code](f04e2e8944/src/commands/init.ts (L96-L102)))
| `create-react-app` | ❌ | uses npm
| `ng new` (`@angular/cli`) | ❌ | uses npm
| `create-vite` | ✅ | suggests working deno commands 🎉
| `create-solid` | ❌ | suggests npm commands, needs PR
It's possible that fixing `package-manager-detector` or other packages
might make some of these just work, but haven't looked too carefully at
each
Spend some time stepping through the npm client code and noticed that
the bearer token was different from ours. They do some double encoding
and @dsherret helped me in matching the encoding behavior.
Fixes https://github.com/denoland/deno/issues/26033
Currently we only warn once. With this PR, we continue to warn about
not-run scripts on explicit `deno install` (or cache). For `run` (or
other subcommands) we only warn the once, as we do currently.
Fixes https://github.com/denoland/deno/issues/25862.
npm only makes bin entries executable if they get linked into `.bin`, as
we did before this PR. So this PR actually deviates from npm, because
it's the only reasonable way to fix this that I can think of.
---
The reason this was broken in moment is the following:
Moment has dependencies on two typescript versions: 1.8 and 3.1
If you have two packages with conflicting bin entries (i.e. two
typescript versions which both have a bin entry `tsc`), in npm it is
non-deterministic and undefined which one will end up in `.bin`.
npm, due to implementation differences, chooses to put typescript 1.8
into the `.bin` directory, and so `node_modules/typescript/bin/tsc` ends
up getting marked executable. We, however, choose typescript 3.2, and so
we end up making `node_modules/typescript3/bin/tsc` executable.
As part of its tests, moment executes `node_modules/typescript/bin/tsc`.
Because we didn't make it executable, this fails.
Since the conflict resolution is undefined in npm, instead of trying to
match it, I think it makes more sense to just make bin entries
executable even if they aren't chosen in the case of a conflict.
This replaces `--allow-net` for import permissions and makes the
security sandbox stricter by also checking permissions for statically
analyzable imports.
By default, this has a value of
`--allow-import=deno.land:443,jsr.io:443,esm.sh:443,raw.githubusercontent.com:443,gist.githubusercontent.com:443`,
but that can be overridden by providing a different set of hosts.
Additionally, when no value is provided, import permissions are inferred
from the CLI arguments so the following works because
`fresh.deno.dev:443` will be added to the list of allowed imports:
```ts
deno run -A -r https://fresh.deno.dev
```
---------
Co-authored-by: David Sherret <dsherret@gmail.com>
Refactors the lifecycle scripts code to extract out the common
functionality and then uses that to provide a warning in the global
resolver.
While ideally we would still support them with the global cache, for now
a warning is at least better than the status quo (where people are
unaware why their packages aren't working).
