Since `rust 1.87.0` reported `undefined symbol:
ring::pbkdf2::PBKDF2_HMAC_SHA1::*` in CI and it was difficult to debug
locally, use `rust 1.86.0` in CI tests for troubleshoot the errors
This commit makes `deno check` ignore `Cannot find module` diagnostic if the
missing module specifier matches one of ambient module pattern. (For
example if there's `declare module "*.svg" { ... }` declaration in one
of d.ts files, importing of `<any>.svg` doesn't cause type error with
this change. This is necessary for passing the type checking of default
vite template.)
---------
Co-authored-by: David Sherret <dsherret@gmail.com>
This change configures V8 isolates to respect memory limits imposed by
cgroups on Linux.
It adds support for detecting both cgroups v1 and v2 memory limits,
enabling Deno to properly adapt to containerized environments with
memory constraints. When cgroups information is unavailable or not
applicable, it falls back to using the system's total memory as before.
Closes#29077
## Test
For testing, I created a ubuntu VM with 1Gi memory. Within this VM, set
up a cgroup with 512Mi memory limit, then ran the following script to
see how much heap size limit the V8 isolate had.
```js
import * as v8 from "node:v8";
console.log(v8.getHeapStatistics());
```
### Ubuntu 20.04
In this version of ubuntu, hybrid mode is enabled by default.
```
$ cat /proc/self/cgroup
12:rdma:/
11:blkio:/user.slice
10:devices:/user.slice
9:cpu,cpuacct:/user.slice
8:pids:/user.slice/user-1000.slice/session-3.scope
7:memory:/user.slice/user-1000.slice/session-3.scope
6:perf_event:/
5:freezer:/
4:net_cls,net_prio:/
3:hugetlb:/
2:cpuset:/
1:name=systemd:/user.slice/user-1000.slice/session-3.scope
0::/user.slice/user-1000.slice/session-3.scope
```
Create a new cgroup with 512Mi memory limit and run the above script in
this cgroup:
```
$ sudo cgcreate -g memory:/mygroup
$ sudo cgset -r memory.limit_in_bytes=$((512 * 1024 * 1024)) mygroup
$ sudo cgexec -g memory:mygroup ./deno run main.mjs
{
total_heap_size: 7745536,
total_heap_size_executable: 0,
total_physical_size: 7090176,
total_available_size: 266348216,
used_heap_size: 6276752,
heap_size_limit: 271581184,
malloced_memory: 303200,
peak_malloced_memory: 140456,
does_zap_garbage: 0,
number_of_native_contexts: 1,
number_of_detached_contexts: 0,
total_global_handles_size: 24576,
used_global_handles_size: 22432,
external_memory: 3232012
}
```
This indicates that the isolate was informed of cgroup-constrained
memory limit (512Mi) and hence got ~270M heap limit.
### Ubuntu 22.04
In this version of ubuntu, cgroup v2 is used.
```
$ cat /proc/self/cgroup
0::/user.slice/user-1000.slice/session-3.scope
```
Run the above script using `systemd-run`:
```
$ sudo systemd-run --property=MemoryMax=512M --pty bash -c '/home/ubuntu/deno run /home/ubuntu/main.mjs'
{
total_heap_size: 7745536,
total_heap_size_executable: 0,
total_physical_size: 7090176,
total_available_size: 266348184,
used_heap_size: 6276784,
heap_size_limit: 271581184,
malloced_memory: 303200,
peak_malloced_memory: 140456,
does_zap_garbage: 0,
number_of_native_contexts: 1,
number_of_detached_contexts: 0,
total_global_handles_size: 24576,
used_global_handles_size: 22432,
external_memory: 3232012
}
```
Again the isolate got ~270M heap limit properly.
Note that it should have had bigger heap limit if the entire system
memory, i.e. 1Gi, had been passed to V8. In fact, if we run the same
script outside the cgroup, it does display larger `heap_size_limit` like
below:
```
$ ./deno run main.mjs
{
total_heap_size: 7745536,
total_heap_size_executable: 0,
total_physical_size: 7090176,
total_available_size: 546580152,
used_heap_size: 6276752,
heap_size_limit: 551813120,
malloced_memory: 303200,
peak_malloced_memory: 140456,
does_zap_garbage: 0,
number_of_native_contexts: 1,
number_of_detached_contexts: 0,
total_global_handles_size: 24576,
used_global_handles_size: 22432,
external_memory: 3232012
}
```
---------
Signed-off-by: Yusuke Tanaka <wing0920@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Closes https://github.com/denoland/deno/issues/27229.
TODO:
- [x] Tests
- [x] Make some changes to `deno_cache_dir` so we can get the paths for
the local http cache
- [x] Right now this leaves the node modules setup cache in an incorrect
state (removes the symlinks, but doesn't update the setup cache)
- [ ] ~~Handle code cache and other sqlite caches?~~
This PR adds detection of `tsconfig.json` at the root of a workspace
when there exists either a deno.json or package.json. If a project
already has `deno.json` with a `compilerOptions` value the
`tsconfig.json` is ignored.
This commit adds the formatting of CSS, HTML, and SQL which are embedded in
Tagged Template Literal strings. The embedded languages are detected by
the tag name of the literal.
```ts
// triggers CSS formatting
const Div = styled.div`
margin: 5px;
padding: ${padding}px;
`;
// triggers HTML formatting
document.body.innerHTML = html`
<main>
<h1>Hello ${name}!<h1>
</main>
`;
// triggers SQL formatting when `--unstable-sql` passed
const query = sql`
SELECT
${table}.${field}
FROM
${table};
`
```
See https://github.com/dprint/dprint-plugin-typescript/pull/701 for more
details
Fixes two issues:
- If a cached packument was out of date and missing a version from the
lockfile, we would fail. Instead we should try again with a forced
re-fetch
- We weren't threading through the workspace patch packages correctly
Fixes#27264. Fixes https://github.com/denoland/deno/issues/28161.
Currently the new lockfile version is gated behind an unstable flag
(`--unstable-lockfile-v5`) until the next minor release, where it will
become the default.
The main motivation here is that it improves startup performance when
using the global cache or `--node-modules-dir=auto`.
In a create-next-app project, running an empty file:
```
❯ hyperfine --warmup 25 -N --setup "rm -f deno.lock" "deno run --node-modules-dir=auto -A empty.js" "deno-this-pr run --node-modules-dir=auto -A empty.js" "deno-this-pr run --node-modules-dir=auto --unstable-lockfile-v5 empty.js" "deno run --node-modules-dir=manual -A empty.js" "deno-this-pr run --node-modules-dir=manual -A empty.js"
Benchmark 1: deno run --node-modules-dir=auto -A empty.js
Time (mean ± σ): 247.6 ms ± 1.7 ms [User: 228.7 ms, System: 19.0 ms]
Range (min … max): 245.5 ms … 251.5 ms 12 runs
Benchmark 2: deno-this-pr run --node-modules-dir=auto -A empty.js
Time (mean ± σ): 169.8 ms ± 1.0 ms [User: 152.9 ms, System: 17.9 ms]
Range (min … max): 168.9 ms … 172.5 ms 17 runs
Benchmark 3: deno-this-pr run --node-modules-dir=auto --unstable-lockfile-v5 empty.js
Time (mean ± σ): 16.2 ms ± 0.7 ms [User: 12.3 ms, System: 5.7 ms]
Range (min … max): 15.2 ms … 19.2 ms 185 runs
Benchmark 4: deno run --node-modules-dir=manual -A empty.js
Time (mean ± σ): 16.2 ms ± 0.8 ms [User: 11.6 ms, System: 5.5 ms]
Range (min … max): 14.9 ms … 19.7 ms 187 runs
Benchmark 5: deno-this-pr run --node-modules-dir=manual -A empty.js
Time (mean ± σ): 16.0 ms ± 0.9 ms [User: 12.0 ms, System: 5.5 ms]
Range (min … max): 14.8 ms … 22.3 ms 190 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet system without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Summary
deno-this-pr run --node-modules-dir=manual -A empty.js ran
1.01 ± 0.08 times faster than deno run --node-modules-dir=manual -A empty.js
1.01 ± 0.07 times faster than deno-this-pr run --node-modules-dir=auto --unstable-lockfile-v5 empty.js
10.64 ± 0.60 times faster than deno-this-pr run --node-modules-dir=auto -A empty.js
15.51 ± 0.88 times faster than deno run --node-modules-dir=auto -A empty.js
```
When using the new lockfile version, this leads to a 15.5x faster
startup time compared to the current deno version.
Install times benefit as well, though to a lesser degree.
`deno install` on a create-next-app project, with everything cached
(just setting up node_modules from scratch):
```
❯ hyperfine --warmup 5 -N --prepare "rm -rf node_modules" --setup "rm -rf deno.lock" "deno i" "deno-this-pr i" "deno-this-pr i --unstable-lockfile-v5"
Benchmark 1: deno i
Time (mean ± σ): 464.4 ms ± 8.8 ms [User: 227.7 ms, System: 217.3 ms]
Range (min … max): 452.6 ms … 478.3 ms 10 runs
Benchmark 2: deno-this-pr i
Time (mean ± σ): 368.8 ms ± 22.0 ms [User: 150.8 ms, System: 198.1 ms]
Range (min … max): 344.8 ms … 397.6 ms 10 runs
Benchmark 3: deno-this-pr i --unstable-lockfile-v5
Time (mean ± σ): 211.9 ms ± 17.1 ms [User: 7.1 ms, System: 177.2 ms]
Range (min … max): 191.3 ms … 233.4 ms 10 runs
Summary
deno-this-pr i --unstable-lockfile-v5 ran
1.74 ± 0.17 times faster than deno-this-pr i
2.19 ± 0.18 times faster than deno i
```
With lockfile v5, a 2.19x faster install time compared to the current
deno.
Updates to use rust 1.85. Doesn't move to the 2024 edition, as that's a
fair bit more involved.
A nice side benefit is that the new rustc version seems to lead to a
slight reduction in binary size (at least on mac):
```
FILE SIZE
--------------
+4.3% +102Ki __DATA_CONST,__const
[NEW] +69.3Ki __TEXT,__literals
[NEW] +68.5Ki Rebase Info
+5.0% +39.9Ki __TEXT,__unwind_info
+57% +8.85Ki [__TEXT]
[NEW] +8.59Ki Lazy Binding Info
[NEW] +5.16Ki __TEXT,__stub_helper
[NEW] +3.58Ki Export Info
[NEW] +3.42Ki __DATA,__la_symbol_ptr
-0.1% -726 [12 Others]
-21.4% -3.10Ki [__DATA_CONST]
-95.8% -3.39Ki __DATA_CONST,__got
-20.9% -3.43Ki [__DATA]
-0.5% -4.52Ki Code Signature
-100.0% -11.6Ki [__LINKEDIT]
-1.0% -43.5Ki Symbol Table
-1.6% -44.0Ki __TEXT,__gcc_except_tab
-0.2% -48.1Ki __TEXT,__const
-3.3% -78.6Ki __TEXT,__eh_frame
-0.7% -320Ki __TEXT,__text
-1.5% -334Ki String Table
-0.5% -586Ki TOTAL
```
Fixes https://github.com/denoland/deno/issues/28223
This is kind of an ugly fix, but it works, and I think is the easiest
way to handle the fact that when caching the module graph we might
encounter imports that won't actually error at runtime (for instance in
files that will be bundled).
Fixes https://github.com/denoland/deno/issues/27569.
Fixes https://github.com/denoland/deno/issues/27215.
This PR makes it so type resolution falls back to looking for definitely
typed packages (`@types/foo`) if a given NPM package does not contain
type declarations.
One complication is choosing _which_ version of the `@types/*` package
to use, if the project depends on multiple versions. The heuristic here
is to try to match the major and minor versions, falling back to the
latest version. So if you have
```
@types/foo: 0.1.0, 0.2.0, 3.1.0, 3.1.2, 4.0.0
foo: 3.1.0
```
we would choose `@types/foo@3.1.2` when resolving types for `foo`.
---
Note that this only uses `@types/` packages if you _already_ depend on
them. So a follow up to this PR could be to add a diagnostic and
quickfix to install `@types/foo` if we don't find types for `foo`.
adds tracing and opentelemetry exporting to the LSP.
enable it in `.vscode/settings.json` (or wherever you configure the
LSP), like
```
{
"deno.tracing": true
}
```
which will by default export opentelemetry traces to `localhost:4317`
or
```
{
"deno.tracing": {
// all fields optional
"collector": "openTelemetry" (default) | "logging" (output in lsp log window)
"collectorEndpoint": "http://localhost:4318" (for opentelemetry)
"enable": true | false,
"filter": "info" // defaults to "info", but can be any span filter
}
}
```
---
a full working setup would be
1: Run jaeger (an opentelemetry collector with a nice UI):
```
docker run --rm -p 16686:16686 -p 4317:4317 jaegertracing/jaeger
```
2. Enable in .vscode/settings.json
```
{
"deno.tracing": true
}
```
3. Restart LSP (right now it only will start the opentelemetry exporter
on LSP startup)
3. open `http://localhost:16686` in your browser
---------
Co-authored-by: Nathan Whitaker <nathan@deno.com>
Removes the TSC snapshot to unblock the V8 upgrade (which enables shared
RO heap, and is incompatible with multiple snapshots).
this compresses the sources and declaration files as well, which leads
to a roughly 4.2MB reduction in binary size.
this currently adds about 80ms to deno check times, but code cache isn't
wired up for the extension code (namely `00_typescript.js`) yet
```
❯ hyperfine --warmup 5 -p "rm -rf ~/Library/Caches/deno/check_cache_v2" "../../deno/target/release-lite/deno check main.ts" "deno check main.ts"
Benchmark 1: ../../deno/target/release-lite/deno check main.ts
Time (mean ± σ): 184.2 ms ± 2.2 ms [User: 378.3 ms, System: 48.2 ms]
Range (min … max): 181.5 ms … 189.9 ms 15 runs
Benchmark 2: deno check main.ts
Time (mean ± σ): 107.4 ms ± 1.2 ms [User: 155.3 ms, System: 23.9 ms]
Range (min … max): 105.3 ms … 109.6 ms 26 runs
Summary
deno check main.ts ran
1.72 ± 0.03 times faster than ../../deno/target/release-lite/deno check main.ts
```
---------
Co-authored-by: Bartek Iwańczuk <biwanczuk@gmail.com>
Allows easily constructing a `DenoResolver` using the exact same logic
that we use in the CLI (useful for dnt and for external bundlers). This
code is then used in the CLI to ensure the logic is always up-to-date.
```rs
use std::rc::Rc;
use deno_resolver:🏭:ResolverFactory;
use deno_resolver:🏭:WorkspaceFactory;
use sys_traits::impls::RealSys;
let sys = RealSys;
let cwd = sys.env_current_dir()?;
let workspace_factory = Rc::new(WorkspaceFactory::new(sys, cwd, Default::default()));
let resolver_factory = ResolverFactory::new(workspace_factory.clone(), Default::default());
let deno_resolver = resolver_factory.deno_resolver().await?;
```
This slightly degrades the performance of CJS export analysis on
subsequent runs because I changed it to no longer cache in the DENO_DIR
with this PR (denort now properly has no idea about the DENO_DIR). We'll
have to change it to embed this data in the binary and that will also
allow us to get rid of swc in denort (will do that in a follow-up PR).
