language-servers/deno
1
0
Fork 0
mirror of https://github.com/denoland/deno.git synced 2025-08-04 19:08:15 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 3

Async compiler processing (#3043)

Browse source 
Basically this does pre-processing of TypeScript files and gathers all the
dependencies asynchronously. Only then after all the dependencies are gathered,
does it do a compile, which at that point all the dependencies are cached in
memory in the compiler, so with the exception of the hard coded assets, there
are no ops during the compilation.

Because op_fetch_source_files is now handled asynchronously in the runtime, we
can eliminate the tokio_util::block_on() which was causing the increase in
threads. Benchmarking on my machine has shown about a 5% improvement in speed
when dealing with compiling TypeScript. Still a long way to go, but an
improvement.

In theory the module name resolution and the fetching of the source files could
be broken out as two different ops. This would prevent situations of sending the
full source file all the time when actually the module is the same module
referenced by multiple modules, but that could be done subsequently to this.
This commit is contained in:
Kitson Kelly 2019-10-03 21:23:29 +10:00 • committed by Ryan Dahl
parent c878a14365
commit d9ff4eccb5
11 changed files with 290 additions and 240 deletions
Download patch file Download diff file Expand all files Collapse all files

39
cli/ops/compiler.rs
View file

@ -1,7 +1,8 @@
// Copyright 2018-2019 the Deno authors. All rights reserved. MIT license.
use super::dispatch_json::{Deserialize, JsonOp, Value};
use crate::futures::future::join_all;
use crate::futures::Future;
use crate::state::ThreadSafeState;
use crate::tokio_util;
use deno::*;
#[derive(Deserialize)]
@ -40,7 +41,7 @@ struct FetchSourceFilesArgs {
pub fn op_fetch_source_files(
state: &ThreadSafeState,
args: Value,
_zero_copy: Option<PinnedBuf>,
_data: Option<PinnedBuf>,
) -> Result<JsonOp, ErrBox> {
let args: FetchSourceFilesArgs = serde_json::from_value(args)?;
@ -58,23 +59,25 @@ pub fn op_fetch_source_files(
futures.push(fut);
}
// WARNING: Here we use tokio_util::block_on() which starts a new Tokio
// runtime for executing the future. This is so we don't inadvertently run
// out of threads in the main runtime.
let files = tokio_util::block_on(futures::future::join_all(futures))?;
let res: Vec<serde_json::value::Value> = files
.into_iter()
.map(|file| {
json!({
"moduleName": file.url.to_string(),
"filename": file.filename.to_str().unwrap(),
"mediaType": file.media_type as i32,
"sourceCode": String::from_utf8(file.source_code).unwrap(),
})
})
.collect();
let future = join_all(futures)
.map_err(ErrBox::from)
.and_then(move |files| {
let res = files
.into_iter()
.map(|file| {
json!({
"url": file.url.to_string(),
"filename": file.filename.to_str().unwrap(),
"mediaType": file.media_type as i32,
"sourceCode": String::from_utf8(file.source_code).unwrap(),
})
})
.collect();
Ok(JsonOp::Sync(json!(res)))
futures::future::ok(res)
});
Ok(JsonOp::Async(Box::new(future)))
}
#[derive(Deserialize)]