Commit graph

86 commits

Author SHA1 Message Date
David Sherret
efac7cd67a
refactor: extract CJS analysis from CLI crate (#29975)
Moves the `CliCjsCodeAnalyzer` out of the CLI crate and into
deno_resolver.
2025-07-02 15:31:25 -04:00
David Sherret
7a1e949c47
refactor: extract DenoDir, Emitter, EmitCache and ParsedSourceCache from cli crate (#29961) 2025-07-02 08:56:02 -04:00
David Sherret
75762194ff
refactor: move CliFileFetcher and DenoGraphLoader into deno_resolver::file_fetcher (#29473) 2025-05-30 10:10:33 -04:00
David Sherret
cb23193f74
refactor: allow deno_permissions to compile to wasm32-unknown-unknown (#29487)
Some checks are pending
ci / pre-build (push) Waiting to run
ci / test debug linux-aarch64 (push) Blocked by required conditions
ci / test release linux-aarch64 (push) Blocked by required conditions
ci / test debug macos-aarch64 (push) Blocked by required conditions
ci / test release macos-aarch64 (push) Blocked by required conditions
ci / bench release linux-x86_64 (push) Blocked by required conditions
ci / lint debug linux-x86_64 (push) Blocked by required conditions
ci / lint debug macos-x86_64 (push) Blocked by required conditions
ci / lint debug windows-x86_64 (push) Blocked by required conditions
ci / test debug linux-x86_64 (push) Blocked by required conditions
ci / test release linux-x86_64 (push) Blocked by required conditions
ci / test debug macos-x86_64 (push) Blocked by required conditions
ci / test release macos-x86_64 (push) Blocked by required conditions
ci / test debug windows-x86_64 (push) Blocked by required conditions
ci / test release windows-x86_64 (push) Blocked by required conditions
ci / build wasm32 (push) Blocked by required conditions
ci / publish canary (push) Blocked by required conditions
2025-05-28 18:27:42 +00:00
David Sherret
e7bb7cca24
fix: load changed non-remote dynamic imports at runtime (#29413) 2025-05-26 21:38:51 +00:00
Paolo Barbolini
1516e3ef58
refactor: reduce dependency on futures crate (#28515)
Over time many `futures` APIs have been stabilized in `std`. This
migrates most uses to the std variants.
2025-03-17 14:14:19 +00:00
David Sherret
273ec9fbf2
refactor: add WorkspaceFactory and ResolverFactory (#27766)
Allows easily constructing a `DenoResolver` using the exact same logic
that we use in the CLI (useful for dnt and for external bundlers). This
code is then used in the CLI to ensure the logic is always up-to-date.

```rs
use std::rc::Rc;

use deno_resolver:🏭:ResolverFactory;
use deno_resolver:🏭:WorkspaceFactory;
use sys_traits::impls::RealSys;

let sys = RealSys;
let cwd = sys.env_current_dir()?;
let workspace_factory = Rc::new(WorkspaceFactory::new(sys, cwd, Default::default()));
let resolver_factory = ResolverFactory::new(workspace_factory.clone(), Default::default());
let deno_resolver = resolver_factory.deno_resolver().await?;
```
2025-01-23 18:52:55 -05:00
David Sherret
57dd66ec3d
refactor: move denort to separate crate (#27688)
This slightly degrades the performance of CJS export analysis on
subsequent runs because I changed it to no longer cache in the DENO_DIR
with this PR (denort now properly has no idea about the DENO_DIR). We'll
have to change it to embed this data in the binary and that will also
allow us to get rid of swc in denort (will do that in a follow-up PR).
2025-01-17 20:39:29 +00:00
David Sherret
05dc69932d
refactor: create deno_lib crate (#27673)
Shifts just some code down for now. I'll do the rest of the refactor in
the next pr, but didn't want to drop a huge refactor.
2025-01-15 09:35:46 -05:00
David Sherret
0b033140c0
refactor: move CliNpmResolver to deno_resolver::npm::NpmResolver (#27659)
As title. After this PR all npm resolution will be out of the CLI crate.
2025-01-14 10:01:05 -05:00
Leo Kettmeir
ea30e188a8
refactor: update deno_core for error refactor (#26867)
Closes #26171

---------

Co-authored-by: David Sherret <dsherret@gmail.com>
2025-01-08 14:52:32 -08:00
Bartek Iwańczuk
b6f2646c1c
refactor: make IncrementalCache accept a CacheDBHash (#27570) 2025-01-06 23:56:36 +00:00
Kenta Moriuchi
8fb073d7b4
chore: Happy New Year 2025 (#27509) 2024-12-31 19:12:39 +00:00
David Sherret
ac7b33a340
chore: cargo fmt - turn on group_imports=StdExternalCrate (#26646) 2024-12-31 12:13:39 -05:00
David Sherret
4638caa740
refactor: do not use deno_fs::FileSystem everywhere (#27508)
This changes the cli to mostly use `std::fs` via `sys_traits` instead of
the implemention of `deno_fs::FileSystem`.
2024-12-31 11:29:07 -05:00
David Sherret
c391ad315e
refactor: use sys_traits (#27480) 2024-12-30 12:38:20 -05:00
David Sherret
95928c46eb
refactor: extract out FileFetcher to deno_cache_dir (#27263) 2024-12-16 23:39:40 +00:00
Bartek Iwańczuk
1d49b3cb0f
fix: support workspace:^ and workspace:~ version constraints (#27096)
This commit adds support for understanding "workpace:^"
and "workspace:~" version constraints in npm/pnpm workspaces.

This is done by upgrading various crates to their latest versions.

Closes https://github.com/denoland/deno/issues/26726

---------

Co-authored-by: David Sherret <dsherret@gmail.com>
2024-11-30 00:54:26 +01:00
David Sherret
f161adf19e
perf(compile): read embedded files as static references when UTF-8 and reading as strings (#27033) 2024-11-27 21:28:41 -05:00
David Sherret
826e42a5b5
fix: improved support for cjs and cts modules (#26558)
* cts support
* better cjs/cts type checking
* deno compile cjs/cts support
* More efficient detect cjs (going towards stabilization)
* Determination of whether .js, .ts, .jsx, or .tsx is cjs or esm is only
done after loading
* Support `import x = require(...);`

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2024-11-01 12:27:00 -04:00
Marvin Hagemeister
5f0bb3c6f4
fix: .npmrc settings not being passed to install/add command (#26473)
We weren't passing the resolved npmrc settings to the install commands.
This lead us to always fall back to the default registry url instead of
using the one from npmrc.

Fixes https://github.com/denoland/deno/issues/26139
Fixes https://github.com/denoland/deno/issues/26033
Fixes https://github.com/denoland/deno/issues/25924
Fixes https://github.com/denoland/deno/issues/25822
Fixes https://github.com/denoland/deno/issues/26152

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2024-10-24 20:03:56 +02:00
David Sherret
1a0cb5b531
feat(unstable): --unstable-detect-cjs for respecting explicit "type": "commonjs" (#26149)
When using the `--unstable-detect-cjs` flag or adding `"unstable":
["detect-cjs"]` to a deno.json, it will make a JS file CJS if the
closest package.json contains `"type": "commonjs"` and the file is not
an ESM module (no TLA, no `import.meta`, no `import`/`export`).
2024-10-14 20:48:39 -04:00
David Sherret
4c8d57db03
BREAKING: rename "deps" remote cache folder to "remote" (#25969)
Closes https://github.com/denoland/deno/issues/25967
Closes #25968
2024-10-01 14:05:40 -04:00
David Sherret
1bb47805d6
refactor: move NpmCacheDir to deno_cache_dir (#25916)
Part of the ongoing work to move more of Deno's resolution out of the
CLI crate (for use in Wasm and other things)

Includes:

* https://github.com/denoland/deno_cache_dir/pull/60
2024-09-28 08:50:16 -04:00
Bartek Iwańczuk
5504acea67
feat: add --allow-import flag (#25469)
This replaces `--allow-net` for import permissions and makes the
security sandbox stricter by also checking permissions for statically
analyzable imports.

By default, this has a value of
`--allow-import=deno.land:443,jsr.io:443,esm.sh:443,raw.githubusercontent.com:443,gist.githubusercontent.com:443`,
but that can be overridden by providing a different set of hosts.

Additionally, when no value is provided, import permissions are inferred
from the CLI arguments so the following works because
`fresh.deno.dev:443` will be added to the list of allowed imports:

```ts
deno run -A -r https://fresh.deno.dev
```

---------

Co-authored-by: David Sherret <dsherret@gmail.com>
2024-09-26 01:50:54 +00:00
Leo Kettmeir
a1d0a427e8
feat: default to TS for file extension and support ext flag in more scenarios (#25472)
Closes #11220
Currently does lint, fmt, and repl
2024-09-18 21:15:13 +02:00
David Sherret
62e952559f
refactor(permissions): split up Descriptor into Allow, Deny, and Query (#25508)
This makes the permission system more versatile.
2024-09-16 21:39:37 +01:00
David Sherret
c678550a17
BREAKING: remove "emit" and "map" from deno info output (#25468)
The map field has been empty for years now and we don't want the emit
file to be exposed so it allows us to iterate on making the cache
faster. Additionally, it's racy/unreliable to rely on this information.
Instead, people should emit the TS files themselves using tools like
deno_emit, typescript, esbuild, etc.

Closes https://github.com/denoland/deno/issues/17703
2024-09-05 14:22:13 +00:00
David Sherret
3afa3db4d3
fix: delete old Deno 1.x headers file when loading cache (#25283) 2024-08-28 23:47:15 +00:00
David Sherret
c89a20b428
perf(cache): single cache file for remote modules (#24983)
This changes the global cache to store the cache file for remote modules
in one file instead of two.
2024-08-26 23:59:17 +00:00
David Sherret
6fce23c54e
perf: skip saving to emit cache after first failure (#24896) 2024-08-08 09:41:30 +00:00
Bartek Iwańczuk
2a2ff96be1
fix(ext/node): discover .npmrc in user's homedir (#24021)
This commit adds discovery of `.npmrc` files in user's homedir.

This is not a perfect fix as it doesn't merge multiple `.npmrc` files
together as per https://github.com/denoland/deno/issues/23954
but allows to fallback if no `.npmrc` file is discovered in the project
root.
2024-06-27 00:17:00 +02:00
David Sherret
ed20102713
fix: make writing to the deps cache more reliable (#24135)
I was able to reproduce this locally.

```
[error] Failed to execute snippet: 
import { validate } from "@std/uuid";
import { assert, assertFalse } from "@std/assert";

assert(validate("6ec0bd7f-11c0-43da-975e-2a8ad9ebae0b"));
assertFalse(validate("not a UUID"));
Download https://jsr.io/@std/uuid/meta.json
Download https://jsr.io/@std/uuid/1.0.0-rc.1_meta.json
Download https://jsr.io/@std/uuid/1.0.0-rc.1/mod.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/common.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/constants.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v1.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v3.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v4.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v5.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/_common.ts
error: Access is denied. (os error 5) (for 'V:\.cache\deno\deps\https\jsr.io\2ae5bb614c7526d0876be0b76da1372fd51304ae27d6202ee94df720b3523d08')
 at file:///V:/deno_std/uuid/common.ts:43
[error] Failed to execute snippet:
import { v5, NAMESPACE_DNS, NIL_UUID } from "@std/uuid";
import { assert, assertFalse } from "@std/assert";

const data = new TextEncoder().encode("deno.land");
const uuid = await v5.generate(NAMESPACE_DNS, data);

assert(v5.validate(uuid));
assertFalse(v5.validate(NIL_UUID));
Download https://jsr.io/@std/uuid/meta.json
Download https://jsr.io/@std/uuid/1.0.0-rc.1_meta.json
Download https://jsr.io/@std/uuid/1.0.0-rc.1/mod.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/common.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/constants.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v1.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v3.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v4.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v5.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/_common.ts
error: Access is denied. (os error 5) (for 'V:\.cache\deno\deps\https\jsr.io\63dd818c5fc1ac39c04df9b42bd9dd4bbc07f7d1b174e405d003731125778da1')
    at https://jsr.io/@std/uuid/1.0.0-rc.1/mod.ts:30:15
 at file:///V:/deno_std/uuid/mod.ts:4
[error] Failed to execute snippet:
import { isNil } from "@std/uuid";
import { assert, assertFalse } from "@std/assert";

assert(isNil("00000000-0000-0000-0000-000000000000"));
assertFalse(isNil(crypto.randomUUID()));
Download https://jsr.io/@std/uuid/meta.json
Download https://jsr.io/@std/uuid/1.0.0-rc.1_meta.json
Download https://jsr.io/@std/uuid/1.0.0-rc.1/mod.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/common.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/constants.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v1.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v3.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v4.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v5.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/_common.ts
error: Access is denied. (os error 5) (for 'V:\.cache\deno\deps\https\jsr.io\fd3a12fc091d16ee29f10fa7a05eeeb8bd6c3cc014642e72478c757f00e7261e')
    at https://jsr.io/@std/uuid/1.0.0-rc.1/mod.ts:34:40
 at file:///V:/deno_std/uuid/common.ts:23
[error] Failed to execute snippet:
import { version } from "@std/uuid";
import { assertEquals } from "@std/assert/assert-equals";

assertEquals(version("d9428888-122b-11e1-b85c-61cd3cbb3210"), 1);
assertEquals(version("6ec0bd7f-11c0-43da-975e-2a8ad9ebae0b"), 4);
Download https://jsr.io/@std/uuid/meta.json
Download https://jsr.io/@std/uuid/1.0.0-rc.1_meta.json
Download https://jsr.io/@std/uuid/1.0.0-rc.1/mod.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/common.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/constants.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v1.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v3.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v4.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/v5.ts
Download https://jsr.io/@std/uuid/1.0.0-rc.1/_common.ts
error: Access is denied. (os error 5) (for 'V:\.cache\deno\deps\https\jsr.io\2ae5bb614c7526d0876be0b76da1372fd51304ae27d6202ee94df720b3523d08')
 at file:///V:/deno_std/uuid/common.ts:66
4 errors found
```

It occurs when many Deno processes are writing to the deps cache at the
same time. Fix is to use `atomic_write_with_retries` which is much more
reliable (and the function that helped make the ecosystem tests more
reliable too). After this change I no longer have this issue.

Closes https://github.com/denoland/deno/issues/24073
2024-06-07 13:06:17 -04:00
David Sherret
386d5c8310
refactor: remove PermissionsContainer in deno_runtime (#24119)
Also removes permissions being passed in for node resolution. It was
completely useless because we only checked it for reading package.json
files, but Deno reading package.json files for resolution is perfectly
fine.

My guess is this is also a perf improvement because Deno is doing less
work.
2024-06-06 23:37:53 -04:00
David Sherret
3341c50b6a
refactor: don't share reqwest::HttpClient across tokio runtimes (#24092)
This also fixes several issues where we weren't properly creating http
clients with the user's settings.
2024-06-03 21:17:08 +00:00
David Sherret
94f040ac28
fix: bump cache sqlite dbs to v2 for WAL journal mode change (#24030)
In https://github.com/denoland/deno/pull/23955 we changed the sqlite db
journal mode to WAL. This causes issues when someone is running an old
version of Deno using TRUNCATE and a new version because the two fight
against each other.
2024-05-29 18:38:18 +00:00
David Sherret
448fe67b7a
feat(vendor): support modifying remote files in vendor folder without checksum errors (#23979)
Includes:

* https://github.com/denoland/deno_graph/pull/486
* https://github.com/denoland/deno_graph/pull/488
* https://github.com/denoland/deno_lockfile/pull/25
* https://github.com/denoland/deno_lockfile/pull/22
* https://github.com/denoland/deno_graph/pull/483
* https://github.com/denoland/deno_graph/pull/470
2024-05-28 14:58:43 -04:00
David Sherret
48daf2dc88
chore: add tests for moduleGraph2 (#23603)
Also did some renames from underscores to hyphens
2024-04-29 14:09:58 -04:00
David Sherret
c497e766f1
refactor: move redirect handling into deno_graph (#23444) 2024-04-19 01:43:28 +00:00
Nayeem Rahman
8e77f091ad
perf(lsp): cleanup document dependencies (#23426) 2024-04-19 00:51:16 +00:00
Igor Zinkovsky
b3d7df5535
perf: v8 code cache (#23081)
This PR enables V8 code cache for ES modules and for `require` scripts
through `op_eval_context`. Code cache artifacts are transparently stored
and fetched using sqlite db and are passed to V8. `--no-code-cache` can
be used to disable.

---------

Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
2024-04-17 07:19:55 -07:00
David Sherret
3eaf174bfc
fix(node): improve cjs tracking (#22673)
We were missing saying that a file is CJS when some Deno code imported
from the node_modules directory at runtime.
2024-03-06 00:23:51 +00:00
David Sherret
f90889e5ee
perf(jsr): fast check cache and lazy fast check graph (#22485) 2024-02-20 21:29:57 +00:00
David Sherret
4f80d83774
feat(unstable): single checksum per JSR package in the lockfile (#22421)
This changes the lockfile to not store JSR specifiers in the "remote"
section. Instead a single JSR integrity is stored per package in the
lockfile, which is a hash of the version's `x.x.x_meta.json` file, which
contains hashes for every file in the package. The hashes in this file
are then compared against when loading.

Additionally, when using `{ "vendor": true }` in a deno.json, the files
can be modified without causing lockfile errors—the checksum is only
checked when copying into the vendor folder and not afterwards
(eventually we should add this behaviour for non-jsr specifiers as
well). As part of this change, the `vendor` folder creation is not
always automatic in the LSP and running an explicit cache command is
necessary. The code required to track checksums in the LSP would have
been too complex for this PR, so that all goes through deno_graph now.
The vendoring is still automatic when running from the CLI.
2024-02-15 14:49:35 -05:00
Nayeem Rahman
012a9d8aeb
chore: rename DENO_REGISTRY_URL to JSR_URL (#22414) 2024-02-14 18:30:44 +00:00
David Sherret
c6def993e0
fix(publish): lazily parse sources (#22301)
Closes #22290
2024-02-06 15:57:10 -05:00
David Sherret
4b7c6049ef
refactor: load bytes in deno_graph (#22212)
Upgrades deno_graph to 0.64 where deno_graph is now responsible for
turning bytes into a string. This is in preparation for Wasm modules.
2024-02-01 03:15:22 +00:00
David Sherret
3298ca06f5
fix(jsr): ensure proper storage of ahead of time module infos (#21918)
This was incorrectly being stored so the AOT cache for JSR specifiers
wasn't being used. I added a wrapper type to help prevent the API being
used incorrectly.
2024-01-13 20:56:56 -05:00
林炳权
96b581bdd2
chore: update to Rust 1.75 (#21731) 2024-01-01 23:22:48 +01:00
David Sherret
7e72f3af61
chore: update copyright to 2024 (#21753) 2024-01-01 19:58:21 +00:00