Embedding the latest version of deno with deno_runtime

[ganesh-pm]

Hello All,

I'm trying to embed deno in my Rust program - I'm using deno_runtime-0.245.0 and deno_core-0.390.0. Here is the code I have

use anyhow::Result;
use std::rc::Rc;
use std::sync::Arc;

use deno_core::{extension, op2};
use deno_core::url::Url;

use deno_runtime::worker::{
    MainWorker,
    WorkerOptions,
    WorkerServiceOptions,
};

mod loader;
use loader::FsModuleLoader;

#[op2(fast)]
fn op_custom_add(a: i32, b: i32) -> i32 {
    a + b
}

deno_core::extension!(
    my_extension,
    ops = [op_custom_add],
    esm_entry_point = "ext:my_extension/01_my_extension.js",
    esm = [dir "src", "01_my_extension.js"],
);

#[tokio::main]
async fn main() -> Result<()> {

    let main_module_path = std::env::current_dir()?.join("src/main.js");
    let main_module = Url::from_file_path(&main_module_path)
        .map_err(|_| anyhow::anyhow!("Failed to create file URL from path: {}", main_module_path.display()))?;

    let module_loader = Rc::new(FsModuleLoader);

    let fs = Arc::new(deno_fs::RealFs);
    let permission_desc_parser = Arc::new(
        deno_permissions::RuntimePermissionDescriptorParser::new(sys_traits::impls::RealSys)
    );

    let feature_checker = Arc::new(deno_runtime::FeatureChecker::default());

    let services = WorkerServiceOptions::<
        deno_resolver::npm::ByonmInNpmPackageChecker,
        deno_resolver::npm::ByonmNpmResolver<sys_traits::impls::RealSys>,
        sys_traits::impls::RealSys,
    > {
        module_loader,
        blob_store: Default::default(),
        broadcast_channel: Default::default(),
        deno_rt_native_addon_loader: None,
        feature_checker,
        fs,
        node_services: None,
        npm_process_state_provider: None,
        permissions: deno_permissions::PermissionsContainer::allow_all(permission_desc_parser),
        root_cert_store_provider: None,
        fetch_dns_resolver: Default::default(),
        shared_array_buffer_store: None,
        compiled_wasm_module_store: None,
        v8_code_cache: None,
        bundle_provider: None,
    };

    let mut options = WorkerOptions {
        //startup_snapshot: deno_snapshots::CLI_SNAPSHOT,
        ..Default::default()
    };
    options.extensions.push(my_extension::init());

    let mut worker =
        MainWorker::bootstrap_from_options(
            &main_module,
            services,
            options,
        );

    worker.execute_main_module(&main_module).await?;

    worker.run_event_loop(false).await?;

    Ok(())
}

When I run this code, the code fails to load telemetry.ts from deno_telemetry - Here is the error I get

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 3.11s
     Running `target/debug/deno_embed`

thread 'main' (989178) panicked at /home/ganesh/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/deno_core-0.390.0/runtime/jsruntime.rs:630:9:
Failed to initialize a JsRuntime: Uncaught SyntaxError: Unexpected token ':'
    at ext:deno_telemetry/telemetry.ts:80:23
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

When I load the CLI Snapshot ( uncomment the CLI_SNAPSHOT ), I the above problem goes away but my extension is not available in ext:core/ops - I'm not sure whether it is possible to load extensions over a snapshot.

I did some searching and at a couple of places, I have seen the mention of an example for using deno_runtime under runtime/examples. However, I'm unable to find it. I can see examples for deno_core but not for deno_runtime.

Can someone help me with what is wrong in the above code or point me to a sample code that uses deno_runtime.

Thanks,
Ganesh

[bartlomieju]

You need to use RuntimeOptions.extension_transpiler option because deno_telemetry ships with TS source. You can use one from https://github.com/denoland/deno/blob/main/runtime/transpile.rs.

[ganesh-pm]

Many thanks - I enabled the "transpile" feature of the deno_runtime crate and I'm able to proceed ahead. I got another panic after this

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.27s
     Running `target/debug/deno_embed`

thread 'main' (1065937) panicked at /home/ganesh/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/deno_core-0.390.0/gotham_state.rs:74:3:
required type deno_runtime::ops::bootstrap::SnapshotOptions is not present in GothamState container
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

thread 'main' (1065937) panicked at library/core/src/panicking.rs:225:5:
panic in a function that cannot unwind
stack backtrace:
   0:     0x5f4986d577b2 - std::backtrace_rs::backtrace::libunwind::trace::h73aabaf37ceb5073
                               at /rustc/01f6ddf7588f42ae2d7eb0a2f21d44e8e96674cf/library/std/src/../../backtrace/src/backtrace/libunwind.rs:117:9
   1:     0x5f4986d577b2 - std::backtrace_rs::backtrace::trace_unsynchronized::h30862f246760437f
                               at /rustc/01f6ddf7588f42ae2d7eb0a2f21d44e8e96674cf/library/std/src/../../backtrace/src/backtrace/mod.rs:66:14
   2:     0x5f4986d577b2 - std::sys::backtrace::_print_fmt::h2d1afd8848eb5d7a
                               at /rustc/01f6ddf7588f42ae2d7eb0a2f21d44e8e96674cf/library/std/src/sys/backtrace.rs:68:9
   3:     0x5f4986d577b2 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h1851ca2a850bd9a9
                               at /rustc/01f6ddf7588f42ae2d7eb0a2f21d44e8e96674cf/library/std/src/sys/backtrace.rs:38:26
   4:     0x5f4986d6c757 - core::fmt::rt::Argument::fmt::he8640bda190d4d38
                               at /rustc/01f6ddf7588f42ae2d7eb0a2f21d44e8e96674cf/library/core/src/fmt/rt.rs:152:76
   5:     0x5f4986d6c757 - core::fmt::write::h22467d3ad5dd5554
   6:     0x5f4986d1d856 - std::io::default_write_fmt::h351a88ae8ee5bcc5
                               at /rustc/01f6ddf7588f42ae2d7eb0a2f21d44e8e96674cf/library/std/src/io/mod.rs:639:11
   7:     0x5f4986d1d856 - std::io::Write::write_fmt::h5e3b6a876f7a20bf
                               at /rustc/01f6ddf7588f42ae2d7eb0a2f21d44e8e96674cf/library/std/src/io/mod.rs:1994:13
   8:     0x5f4986d33fc9 - std::sys::backtrace::BacktraceLock::print::hc25d10722ea4032d
                               at /rustc/01f6ddf7588f42ae2d7eb0a2f21d44e8e96674cf/library/std/src/sys/backtrace.rs:41:9
   9:     0x5f4986d33fc9 - std::panicking::default_hook::{{closure}}::he43c3ac33dfa4b50
                               at /rustc/01f6ddf7588f42ae2d7eb0a2f21d44e8e96674cf/library/std/src/panicking.rs:292:27
  10:     0x5f4986d33e29 - std::panicking::default_hook::hd124da54acf1152f
                               at /rustc/01f6ddf7588f42ae2d7eb0a2f21d44e8e96674cf/library/std/src/panicking.rs:319:9
  11:     0x5f4986d342bb - std::panicking::panic_with_hook::h9b5f1f19954f65a8
                               at /rustc/01f6ddf7588f42ae2d7eb0a2f21d44e8e96674cf/library/std/src/panicking.rs:825:13
  12:     0x5f4986d340ba - std::panicking::panic_handler::{{closure}}::hf431df8c849ee0d6
                               at /rustc/01f6ddf7588f42ae2d7eb0a2f21d44e8e96674cf/library/std/src/panicking.rs:691:13
  13:     0x5f4986d2d629 - std::sys::backtrace::__rust_end_short_backtrace::hf97362b31a346cc0
                               at /rustc/01f6ddf7588f42ae2d7eb0a2f21d44e8e96674cf/library/std/src/sys/backtrace.rs:176:18
  14:     0x5f4986d1089d - __rustc[9e6a08e89e4b9111]::rust_begin_unwind
                               at /rustc/01f6ddf7588f42ae2d7eb0a2f21d44e8e96674cf/library/std/src/panicking.rs:689:5
  15:     0x5f4986d7844d - core::panicking::panic_nounwind_fmt::runtime::hccb3971791e1afcd
                               at /rustc/01f6ddf7588f42ae2d7eb0a2f21d44e8e96674cf/library/core/src/panicking.rs:122:22
  16:     0x5f4986d7844d - core::panicking::panic_nounwind_fmt::hf349702e3facb1fd
                               at /rustc/01f6ddf7588f42ae2d7eb0a2f21d44e8e96674cf/library/core/src/intrinsics/mod.rs:2449:9
  17:     0x5f4986d783cb - core::panicking::panic_nounwind::h9a55331e46709e5f
                               at /rustc/01f6ddf7588f42ae2d7eb0a2f21d44e8e96674cf/library/core/src/panicking.rs:225:5
  18:     0x5f4986d78557 - core::panicking::panic_cannot_unwind::h9d41c6c1d0e0d4e5
                               at /rustc/01f6ddf7588f42ae2d7eb0a2f21d44e8e96674cf/library/core/src/panicking.rs:337:5
  19:     0x5f497ea3a355 - deno_runtime::ops::bootstrap::op_snapshot_options::op_snapshot_options::v8_fn_ptr::h978f798f2294e273
                               at /home/ganesh/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/deno_runtime-0.245.0/ops/bootstrap.rs:68:1
  20:     0x5f49867d0be9 - Builtins_CallApiCallbackGeneric
thread caused non-unwinding panic. aborting.
Aborted (core dumped)

I enabled the "hmr" feature ( Hoping that enabling this feature would bypass the issue ) and it worked.
Is it okay to enable this feature...

Thanks,
Ganesh

[aniruddhaadak80]

To embed deno_runtime easily, make sure you align the Tokio versions correctly as Deno heavily relies on specific Tokio features. You can use the JsRuntime struct in deno_core alongside extension traits instead of building the whole CLI binary. Check the deno_core examples folder for minimal embedding bootstraps!

[aniruddhaadak80]

When embedding Deno, make sure that all extension modules are successfully loaded by explicitly defining them in the options.extensions vector and passing them to WorkerOptions. If relying on snapshots, ensure the snapshot contains your extension!

[aniruddhaadak80]

When embedding Deno, make sure that all extension modules are successfully loaded by explicitly defining them in the options.extensions vector and passing them to WorkerOptions. If relying on snapshots, ensure the snapshot contains your extension!