1. Renames `DenoResolver` to `RawDenoResolver`
2. Renames `DenoGraphResolver` to `DenoResolver`
3. Shifts down creation of `DenoGraphResolver` and
`HasPackageJsonDepFlag` to `ResolverFactory`
Fixes two issues:
- If a cached packument was out of date and missing a version from the
lockfile, we would fail. Instead we should try again with a forced
re-fetch
- We weren't threading through the workspace patch packages correctly
Fixes#27264. Fixes https://github.com/denoland/deno/issues/28161.
Currently the new lockfile version is gated behind an unstable flag
(`--unstable-lockfile-v5`) until the next minor release, where it will
become the default.
The main motivation here is that it improves startup performance when
using the global cache or `--node-modules-dir=auto`.
In a create-next-app project, running an empty file:
```
❯ hyperfine --warmup 25 -N --setup "rm -f deno.lock" "deno run --node-modules-dir=auto -A empty.js" "deno-this-pr run --node-modules-dir=auto -A empty.js" "deno-this-pr run --node-modules-dir=auto --unstable-lockfile-v5 empty.js" "deno run --node-modules-dir=manual -A empty.js" "deno-this-pr run --node-modules-dir=manual -A empty.js"
Benchmark 1: deno run --node-modules-dir=auto -A empty.js
Time (mean ± σ): 247.6 ms ± 1.7 ms [User: 228.7 ms, System: 19.0 ms]
Range (min … max): 245.5 ms … 251.5 ms 12 runs
Benchmark 2: deno-this-pr run --node-modules-dir=auto -A empty.js
Time (mean ± σ): 169.8 ms ± 1.0 ms [User: 152.9 ms, System: 17.9 ms]
Range (min … max): 168.9 ms … 172.5 ms 17 runs
Benchmark 3: deno-this-pr run --node-modules-dir=auto --unstable-lockfile-v5 empty.js
Time (mean ± σ): 16.2 ms ± 0.7 ms [User: 12.3 ms, System: 5.7 ms]
Range (min … max): 15.2 ms … 19.2 ms 185 runs
Benchmark 4: deno run --node-modules-dir=manual -A empty.js
Time (mean ± σ): 16.2 ms ± 0.8 ms [User: 11.6 ms, System: 5.5 ms]
Range (min … max): 14.9 ms … 19.7 ms 187 runs
Benchmark 5: deno-this-pr run --node-modules-dir=manual -A empty.js
Time (mean ± σ): 16.0 ms ± 0.9 ms [User: 12.0 ms, System: 5.5 ms]
Range (min … max): 14.8 ms … 22.3 ms 190 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Summary
deno-this-pr run --node-modules-dir=manual -A empty.js ran
1.01 ± 0.08 times faster than deno run --node-modules-dir=manual -A empty.js
1.01 ± 0.07 times faster than deno-this-pr run --node-modules-dir=auto --unstable-lockfile-v5 empty.js
10.64 ± 0.60 times faster than deno-this-pr run --node-modules-dir=auto -A empty.js
15.51 ± 0.88 times faster than deno run --node-modules-dir=auto -A empty.js
```
When using the new lockfile version, this leads to a 15.5x faster
startup time compared to the current deno version.
Install times benefit as well, though to a lesser degree.
`deno install` on a create-next-app project, with everything cached
(just setting up node_modules from scratch):
```
❯ hyperfine --warmup 5 -N --prepare "rm -rf node_modules" --setup "rm -rf deno.lock" "deno i" "deno-this-pr i" "deno-this-pr i --unstable-lockfile-v5"
Benchmark 1: deno i
Time (mean ± σ): 464.4 ms ± 8.8 ms [User: 227.7 ms, System: 217.3 ms]
Range (min … max): 452.6 ms … 478.3 ms 10 runs
Benchmark 2: deno-this-pr i
Time (mean ± σ): 368.8 ms ± 22.0 ms [User: 150.8 ms, System: 198.1 ms]
Range (min … max): 344.8 ms … 397.6 ms 10 runs
Benchmark 3: deno-this-pr i --unstable-lockfile-v5
Time (mean ± σ): 211.9 ms ± 17.1 ms [User: 7.1 ms, System: 177.2 ms]
Range (min … max): 191.3 ms … 233.4 ms 10 runs
Summary
deno-this-pr i --unstable-lockfile-v5 ran
1.74 ± 0.17 times faster than deno-this-pr i
2.19 ± 0.18 times faster than deno i
```
With lockfile v5, a 2.19x faster install time compared to the current
deno.
This adds support for using a local copy of an npm package.
```js
// deno.json
{
"patch": [
"../path/to/local_npm_package"
],
// required until Deno 2.3, but it will still be considered unstable
"unstable": ["npm-patch"]
}
```
1. Requires using a node_modules folder.
2. When using `"nodeModulesDir": "auto"`, it recreates the folder in the
node_modules directory on each run which will slightly increase startup
time.
3. When using the default with a package.json (`"nodeModulesDir":
"manual"`), updating the package requires running `deno install`. This
is to get the package into the node_modules directory of the current
workspace. This is necessary instead of linking because packages can
have multiple "copy packages" due to peer dep resolution.
Caveat: Specifying a local copy of an npm package or making changes to
its dependencies will purge npm packages from the lockfile. This might
cause npm resolution to resolve differently and it may end up not using
the local copy of the npm package. It's very difficult to only
invalidate resolution midway through the graph and then only rebuild
that part of the resolution, so this is just a first pass that can be
improved in the future. In practice, this probably won't be an issue for
most people.
Another limitation is this also requires the npm package name to exist
in the registry at the moment.
Updates to use rust 1.85. Doesn't move to the 2024 edition, as that's a
fair bit more involved.
A nice side benefit is that the new rustc version seems to lead to a
slight reduction in binary size (at least on mac):
```
FILE SIZE
--------------
+4.3% +102Ki __DATA_CONST,__const
[NEW] +69.3Ki __TEXT,__literals
[NEW] +68.5Ki Rebase Info
+5.0% +39.9Ki __TEXT,__unwind_info
+57% +8.85Ki [__TEXT]
[NEW] +8.59Ki Lazy Binding Info
[NEW] +5.16Ki __TEXT,__stub_helper
[NEW] +3.58Ki Export Info
[NEW] +3.42Ki __DATA,__la_symbol_ptr
-0.1% -726 [12 Others]
-21.4% -3.10Ki [__DATA_CONST]
-95.8% -3.39Ki __DATA_CONST,__got
-20.9% -3.43Ki [__DATA]
-0.5% -4.52Ki Code Signature
-100.0% -11.6Ki [__LINKEDIT]
-1.0% -43.5Ki Symbol Table
-1.6% -44.0Ki __TEXT,__gcc_except_tab
-0.2% -48.1Ki __TEXT,__const
-3.3% -78.6Ki __TEXT,__eh_frame
-0.7% -320Ki __TEXT,__text
-1.5% -334Ki String Table
-0.5% -586Ki TOTAL
```
Fixes https://github.com/denoland/deno/issues/27569.
Fixes https://github.com/denoland/deno/issues/27215.
This PR makes it so type resolution falls back to looking for definitely
typed packages (`@types/foo`) if a given NPM package does not contain
type declarations.
One complication is choosing _which_ version of the `@types/*` package
to use, if the project depends on multiple versions. The heuristic here
is to try to match the major and minor versions, falling back to the
latest version. So if you have
```
@types/foo: 0.1.0, 0.2.0, 3.1.0, 3.1.2, 4.0.0
foo: 3.1.0
```
we would choose `@types/foo@3.1.2` when resolving types for `foo`.
---
Note that this only uses `@types/` packages if you _already_ depend on
them. So a follow up to this PR could be to add a diagnostic and
quickfix to install `@types/foo` if we don't find types for `foo`.
Extracted out of https://github.com/denoland/deno/pull/27838/files
Reduces some allocations by accepting either a pathbuf or url for the
referrer for resolution and returning either a pathbuf or url at the
end, which the caller can then convert into to their preferred state.
This is about 4% faster when still converting the final result to a url
and 6% faster when keeping the result as a path in a benchmark I ran.
Allows easily constructing a `DenoResolver` using the exact same logic
that we use in the CLI (useful for dnt and for external bundlers). This
code is then used in the CLI to ensure the logic is always up-to-date.
```rs
use std::rc::Rc;
use deno_resolver:🏭:ResolverFactory;
use deno_resolver:🏭:WorkspaceFactory;
use sys_traits::impls::RealSys;
let sys = RealSys;
let cwd = sys.env_current_dir()?;
let workspace_factory = Rc::new(WorkspaceFactory::new(sys, cwd, Default::default()));
let resolver_factory = ResolverFactory::new(workspace_factory.clone(), Default::default());
let deno_resolver = resolver_factory.deno_resolver().await?;
```
Currently deno eagerly caches all npm packages in the workspace's npm
resolution. So, for instance, running a file `foo.ts` that imports
`npm:chalk` will also install all dependencies listed in `package.json`
and all `npm` dependencies listed in the lockfile.
This PR refactors things to give more control over when and what npm
packages are automatically cached while building the module graph.
After this PR, by default the current behavior is unchanged _except_ for
`deno install --entrypoint`, which will only cache npm packages used by
the given entrypoint. For the other subcommands, this behavior can be
enabled with `--unstable-npm-lazy-caching`
Fixes#25782.
---------
Signed-off-by: Nathan Whitaker <17734409+nathanwhit@users.noreply.github.com>
Co-authored-by: Luca Casonato <hello@lcas.dev>
This was doing an allocation for reparsing the specifier. Might as well
do `.join` here and it means I can extract out this file fetcher code to
deno_cache_dir more easily.
This resurrects the `--unstable-detect-cjs` flag (which became stable),
and repurposes it to attempt loading .js/.jsx/.ts/.tsx files as CJS in
the following additional scenarios:
1. There is no package.json
1. There is a package.json without a "type" field
Also cleans up the implementation of this in the LSP a lot by hanging
`resolution_mode()` off `Document` (didn't think about doing that until
now).
Ensures a dynamic import in a CJS file will consider the referrer as an import for node resolution.
Also adds fixes (adds) support for `"resolution-mode"` in TypeScript.
This will respect `"type": "commonjs"` in a package.json to determine if
`.js`/`.jsx`/`.ts`/.tsx` files are CJS or ESM. If the file is found to
be ESM it will be loaded as ESM though.
* cts support
* better cjs/cts type checking
* deno compile cjs/cts support
* More efficient detect cjs (going towards stabilization)
* Determination of whether .js, .ts, .jsx, or .tsx is cjs or esm is only
done after loading
* Support `import x = require(...);`
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
