Host Integration

In Node, the standard library relies on the operating system for things like files, sockets and processes. Outside Node, nstdlib has none of those by default. This page shows how to plug them in, either with the ready-made helpers in nstdlib/globals or with your own implementation.

Features that can't be plugged in at all are listed in Limitations.

#process and Buffer

Node creates process and Buffer in C++ before any of your code runs. A browser or worker has neither. nstdlib's modules don't need you to add them: if there's no global process when the first module loads, nstdlib creates a default one and installs it on globalThis. That default says the platform is "browser", the working directory is /, the environment is empty, and anything written to stdout or stderr is dropped.

To give process your own values, call installGlobals() before importing any nstdlib/node/* module. It also adds a global Buffer, which code you run on top of nstdlib may expect:

import { installGlobals } from "nstdlib/globals";

// adds process and Buffer, non-enumerable like Node's
await installGlobals(globalThis, { cwd: "/app", env: { HOME: "/app" } });

const path = await import("nstdlib/node/path");

It only adds what's missing, so it never replaces an existing process. That includes the default one: once any nstdlib/node/* module has loaded, a later installGlobals() call keeps the default and ignores the values you pass. If you want the process object without installing it as a global, use createProcess() instead.

#A filesystem

Most of fs needs a disk, which nstdlib doesn't have. installFileSystem() gives it one. Pass it a provider from node:vfs, such as MemoryProvider, which keeps files in memory. After that, the whole fs API works on it: sync, callback and promise functions, streams, and require() and import().

import { installFileSystem } from "nstdlib/globals";
import { MemoryProvider } from "nstdlib/node/vfs";
import fs from "nstdlib/node/fs";
import fsp from "nstdlib/node/fs/promises";

const mem = await installFileSystem(globalThis, new MemoryProvider(), {
  mount: "/mem", // defaults to "/"
});

fs.writeFileSync("/mem/hello.txt", "hi");
fs.readFileSync("/mem/hello.txt", "utf8"); // 'hi'
await fsp.readdir("/mem"); // [ 'hello.txt' ]

mem.dispose(); // unmount; the files are gone

A few things worth knowing:

• Order doesn't matter here. You can install a filesystem before or after importing fs.
• You can mount several. Each handles paths under its own mount point. A path no mount handles still throws an error rather than silently doing nothing.
• Errors look like Node's, with real codes and messages such as ENOENT: no such file or directory, open '/mem/nope'.
• File descriptors are plain numbers starting at 3.

If you run several programs one after another in the same page, mem.scope() helps you clean up fs.watch() watchers between them. It tracks the watchers started after you call it. Its holding property counts the ones that would keep a Node process alive (so not those created with persistent: false or unref()), and close() stops them the way a process exit would.

#Write your own provider

To keep files somewhere other than memory, such as IndexedDB, a real disk or a network share, extend VirtualProvider from nstdlib/node/vfs and install it the same way. Your provider must return the same classes fs expects, so import them from these modules:

import { VirtualProvider } from "nstdlib/node/vfs"; // the class to extend
import { VirtualFileHandle } from "nstdlib/node/internal/vfs/file_handle"; // what open() returns
import { createENOENT } from "nstdlib/node/internal/vfs/errors"; // errors to throw
import "nstdlib/node/internal/vfs/stats"; // createFileStats() and friends, for stat()

Some tips for writing one:

• Paths are relative to the mount point, so the same provider works at / or at /mem.
• Sync methods are enough. If you only implement the *Sync methods, the callback and promise forms of fs fall back to them. You can also implement the async methods separately; fs.readFile() and fs.promises.readFile() call your async readFile() when you have one.
• Some methods are optional. Without chmodSync, chownSync, utimesSync (or their l versions), those calls fail with ENOSYS.
• Watching is optional too. If your provider has a watch() method, fs.watch() uses it. Otherwise nstdlib polls your provider with statSync() and readdirSync().

If you're extending node:vfs itself, two lower-level modules are also published: nstdlib/node/internal/fs/utils exports vfsState.handlers, and nstdlib/node/internal/fs/watchers exports the FSWatcher and StatWatcher classes.

#A network

net and http are fully implemented in nstdlib: connections, servers, backlogs, half-closed sockets and so on. What's missing is the socket itself. Without one, listen() and connect() fail with ENOSYS.

installNetwork() provides an in-memory network that lives inside your JavaScript environment. Call it before the first import of nstdlib/node/net (or anything that uses it, such as http):

import { installGlobals, installNetwork } from "nstdlib/globals";

await installGlobals();
await installNetwork();

const http = await import("nstdlib/node/http");
http.createServer((req, res) => res.end("hi")).listen(8080);

Now http.request(), net.connect() and net.createServer() work between servers and clients in the same page. That's enough to run code written for node:http with nothing underneath it.

If your runtime already has a real network (for example, in nstd), installNetwork() does nothing and returns null.

#Reach the outside world

To connect to real servers, pass a provider. Its connect() method is called whenever code connects to an address nothing in the page is listening on. Return null to refuse the connection, or return a channel with write(), and optionally end() and close(). For example, through a WebSocket relay:

const network = await installNetwork(globalThis, {
  provider: {
    connect(address, port, sink) {
      const socket = new WebSocket(`wss://relay.example/${address}/${port}`);
      socket.binaryType = "arraybuffer";
      socket.onopen = () => sink.open();
      socket.onmessage = (e) => sink.data(new Uint8Array(e.data));
      socket.onclose = () => sink.end();
      socket.onerror = () => sink.error("ECONNRESET");
      return { write: (bytes) => (socket.send(bytes), true) };
    },
  },
});

A WebTransport relay, a WebRTC data channel or your own network stack plugs in the same way.

The returned network also works in the other direction. network.dial() lets code outside the page connect to a server inside it. For example, a service worker can send the page's fetch requests to an in-page http.createServer():

const channel = network.dial("127.0.0.1", 8080, sink);
channel.write(requestBytes);

network.endpoints lists what's currently listening, and network.descriptors shows how many sockets are open.

#DNS

node:dns works without any setup by sending DNS-over-HTTPS requests with fetch. By default, lookups go to Cloudflare's public resolver. That means a third party sees every hostname you look up, so you may want to change it:

import { configureDoH } from "nstdlib/node/stub/binding/cares_wrap";

configureDoH({ url: "https://dns.google/dns-query" }); // any RFC 8484 resolver
configureDoH({ fetch: myFetch }); // use your own fetch
configureDoH({ timeout: 2000 }); // in milliseconds

You can also set the resolver with the NSTDLIB_DOH_URL environment variable (process.env.NSTDLIB_DOH_URL), and dns.getServers() shows which resolver is in use.

If your runtime can resolve hostnames itself, define globalThis.node_binding_cares_wrap_getaddrinfo before importing nstdlib, and dns.lookup() will use it instead of DNS-over-HTTPS. The dns.resolve*() functions still use DNS-over-HTTPS, because they ask for specific DNS records.

#Child processes

installProcesses() makes spawn(), exec(), execFile() and fork() work on top of whatever your application considers a "command". You provide a CommandHost object whose spawn() method starts a command and returns its stdio and exit status. nstdlib doesn't include one; what a command means is up to you. The expected shape is defined in src/embed/types/commands.ts.

import {
  installGlobals,
  installNetwork,
  installProcesses,
} from "nstdlib/globals";

await installGlobals();
const network = await installNetwork();
await installProcesses(globalThis, myCommandHost, { network });

const { spawn } = await import("nstdlib/node/child_process");

A few rules:

• It needs installNetwork(). A child's stdin, stdout and stderr travel over the in-memory network, so pass its return value as network.
• Install it before the first import of nstdlib/node/child_process.
• fork() messages can go both ways if your host's child object has a write(fd, bytes) method. Without it, messages only travel from child to parent, and send() from the parent fails with an error.
• spawnSync() and the other sync functions aren't supported. They still throw.

#ES modules

nstdlib can't compile .mjs source on its own: JavaScript has no standard way to compile a module while letting you control how its imports resolve. nstdlib/esm fills that gap with a compiler that transforms each module into a function. Install it like this:

import { compileSourceTextModule } from "nstdlib/esm";

globalThis.node_binding_module_wrap_compileSourceTextModule =
  compileSourceTextModule;

After that, Node's own ES module loader can load .mjs files from an installed filesystem, with the real resolver. You can call the loader directly:

import { getOrInitializeCascadedLoader } from "nstdlib/node/internal/modules/esm/loader";

const loader = getOrInitializeCascadedLoader();
const mod = await loader.import("node:path", "file:///app/index.mjs", {
  __proto__: null,
});
mod.join("a", "b"); // 'a/b'

Like require(), this compiles code from strings, so it needs a runtime that allows new Function(). Builtins still have to be registered first, as described below.

nstdlib/esm also exports createLoader(), a smaller standalone loader that takes your own readSource(), resolve() and builtin() functions.

#The builtin registry

require() and import() of a builtin, such as require("path"), only work for builtins you register. nstdlib doesn't register any by default, so a program doesn't pull in the whole library just because it asked for one module:

import { BuiltinModule } from "nstdlib/node/internal/bootstrap/realm";

BuiltinModule.register("path", await import("nstdlib/node/path"));
BuiltinModule.registerAll({
  os: await import("nstdlib/node/os"),
  events: await import("nstdlib/node/events"),
});

#More advanced hooks

nstdlib/globals has a few more helpers for embedders:

• installNodeProcess() adds the rest of Node's process to the default one: cpuUsage(), kill(), execve(), a validating emitWarning() that prints, process.binding(), features, allowedNodeEnvironmentFlags, setSourceMapsEnabled() and getBuiltinModule(), each Node's own function over nstdlib's bindings. Call it after installGlobals() when you run whole programs rather than import a module or two. A method with nothing behind it throws and names the globalThis.node_binding_process_methods_* hook that would back it; umask(), chdir() and cwd() are replaced only if you installed those hooks first. It does nothing to a process nstdlib didn't create.
• installParser() loads the JavaScript parser, which TypeScript type-stripping (.ts files) needs. It's optional because the parser is the largest part of the package.
• installWorkers(globalThis, host) makes worker_threads work, given a host that can run a second JavaScript environment on another thread. See src/embed/types/workers.ts for the shape.
• installRealms(globalThis, host) makes vm.createContext() work, given a host that can create a separate global scope, such as an <iframe>. See src/embed/types/vm.ts for the shape.

Like the others, each one leaves an existing implementation alone, so they're safe to call in nstd or under Node.