Building a crude Node.js from scratch

Node is powered by the JavaScript engine used in Google Chrome, called V8¹. In this post I'm going to guide you through two steps:

• making a "Hello World" example in V8
• making a crude Node runtime with support for 3 statements: console.log, console.error and quit for quitting the process

In our new crude runtime we'll execute the following script:

console.log("🎉");  
b=4+4;  
console.error(b);  
quit();

console.log("never reach this");  

Setting up the hello world example

First things first. Let's execute a JavaScript string concatenation in V8! We'll write an example that takes a JavaScript statement as a string argument, executes it as JavaScript code, and prints the result to standard out. The string will be

'Hello' + ', World!'  

The setup will loosely follow this gist in combination with the V8 getting started with embedding wiki.

We're going to use version 5.8 of V8. I'm going to assume you're using MacOS with git, Python 2.7 and Xcode installed and that you're using Bash as your shell of choice.

Clone the v8 source

git clone https://github.com/v8/v8.git  

Install depot tools and add it to our path

git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git  
cd depot_tools  
echo "export PATH=`pwd`:\"$PATH\"" >> ~/.bashrc # replace with ~/.zshrc if using ZSH  
source ~/.bashrc # or ~/.zshrc  
cd -  

Make sure you have clang and clang++ installed

which clang && which clang++ # should output paths to clangs  

If you don't have clang and clang++ make sure you have installed Xcode.

Add environment libraries for clang and clang++

cat <<EOT >> ~/.bashrc # replace with ~/.zshrc if using ZSH  
export CXX="`which clang++`"  
export CC="`which clang`"  
export CPP="`which clang` -E"  
export LINK="`which clang++`"  
export CXX_host="`which clang++`"  
export CC_host="`which clang`"  
export CPP_host="`which clang` -E"  
export LINK_host="`which clang++`"  
export GYP_DEFINES="clang=1"  
EOT

source ~/.bashrc # or ~/.zshrc  

Update .git/config of V8 to fetch remotes and tags

cd v8  
vim .git/config  

Update config for remote origin to this

[remote "origin"]
  url = https://chromium.googlesource.com/v8/v8.git
  fetch = +refs/branch-heads/*:refs/remotes/branch-heads/*
  fetch = +refs/tags/*:refs/tags/*

Exit vim² and fetch origin.

git fetch  

Checkout to a new branch for version 5.8 of V8

git checkout -b 5.8 -t branch-heads/5.8  

Sync this git repo

gclient sync  

Create build configuration

tools/dev/v8gen.py x64.release  

Edit the default build configuration

gn args out.gn/x64.release  

And add these two lines to that configuration:

is_component_build = false  
v8_static_library = true  

Build v8

You'll have to detect a number of cores your CPU has. Go to Activity Monitor and click on CPU LOAD section. Window with graphs should pop up - the number of cores is the number of panels on that window.

For my 2015 i7 CPU it's 4, so I'm running the following command:

make native -j 4  

If you're not sure just run it without -j flag

make native  

Alternatively you can just use Ninja like this:

ninja -C out.gn/x64.release  

Add hello world

vim hello_world.cpp  

Save this in hello_world.cpp³:

// Copyright 2015 the V8 project authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file.
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "include/libplatform/libplatform.h"
#include "include/v8.h"
using namespace v8;  
int main(int argc, char* argv[]) {  
  // Initialize V8.
  V8::InitializeICUDefaultLocation(argv[0]);
  V8::InitializeExternalStartupData(argv[0]);
  Platform* platform = platform::CreateDefaultPlatform();
  V8::InitializePlatform(platform);
  V8::Initialize();
  // Create a new Isolate and make it the current one.
  Isolate::CreateParams create_params;
  create_params.array_buffer_allocator =
      v8::ArrayBuffer::Allocator::NewDefaultAllocator();
  Isolate* isolate = Isolate::New(create_params);
  {
    Isolate::Scope isolate_scope(isolate);
    // Create a stack-allocated handle scope.
    HandleScope handle_scope(isolate);
    // Create a new context.
    Local<Context> context = Context::New(isolate);
    // Enter the context for compiling and running the hello world script.
    Context::Scope context_scope(context);
    // Create a string containing the JavaScript source code.
    Local<String> source =
        String::NewFromUtf8(isolate, "'Hello' + ', World!'",
                            NewStringType::kNormal).ToLocalChecked();
    // Compile the source code.
    Local<Script> script = Script::Compile(context, source).ToLocalChecked();
    // Run the script to get the result.
    Local<Value> result = script->Run(context).ToLocalChecked();
    // Convert the result to an UTF8 string and print it.
    String::Utf8Value utf8(result);
    printf("%s\n", *utf8);
  }
  // Dispose the isolate and tear down V8.
  isolate->Dispose();
  V8::Dispose();
  V8::ShutdownPlatform();
  delete platform;
  delete create_params.array_buffer_allocator;
  return 0;
}

Copy snapshots

cp out.gn/x64.release/*.bin .  

Compile the hello world example

clang++ -Iinclude out/native/*.a hello_world.cpp -o hello_world  

Run the hello world example

./hello_world

You should see

Hello, World!  

in your shell. Now that we have a hello world example up and running, we can start adding support for console.log, console.error and quit.

Add support for console and quit statements

Add the following to run_script.cpp⁴:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <fstream>
#include <iostream>
#include <sstream>
#include "include/libplatform/libplatform.h"
#include "include/v8.h"

using namespace v8;

// Define a quit function that exits.
void quit(const v8::FunctionCallbackInfo<v8::Value>& args) {  
  std::exit(0);
}

// Store isolate in a global variable.
Isolate* isolate_;  
Isolate* GetIsolate() { return isolate_; }

// Define a Console class.
class Console { };

// Extracts a C string from a V8 Utf8Value.
const char* ToCString(const v8::String::Utf8Value& value) {  
  return *value ? *value : "<string conversion failed>";
}

// Define a log function that prints to stdout.
void log(const FunctionCallbackInfo<Value>& args){  
  v8::String::Utf8Value str(args[0]);
  const char* cstr = ToCString(str);
  printf("%s\n", cstr);
}

// Define an error function that prints to stderr.
void error(const FunctionCallbackInfo<Value>& args){  
  v8::String::Utf8Value str(args[0]);
  const char* cstr = ToCString(str);
  fprintf(stderr,"%s\n", cstr);
}

Local<Object> WrapConsoleObject(Console *c) {  
  EscapableHandleScope handle_scope(GetIsolate());

  Local<ObjectTemplate> class_t;
  Local<ObjectTemplate> raw_t = ObjectTemplate::New(GetIsolate());
  raw_t->SetInternalFieldCount(1);

  // Set log method.
  raw_t->Set(
      v8::String::NewFromUtf8(GetIsolate(), "log", v8::NewStringType::kNormal).ToLocalChecked(),
      v8::FunctionTemplate::New(GetIsolate(), log));

  // Set error method.
  raw_t->Set(
      v8::String::NewFromUtf8(GetIsolate(), "error", v8::NewStringType::kNormal).ToLocalChecked(),
      v8::FunctionTemplate::New(GetIsolate(), error));
  class_t = Local<ObjectTemplate>::New(GetIsolate(),raw_t);

  // Create instance.
  Local<Object> result = class_t->NewInstance(GetIsolate()->GetCurrentContext()).ToLocalChecked();

  // Create wrapper.
  Local<External> ptr = External::New(GetIsolate(),c);
  result->SetInternalField(0,ptr);
  return handle_scope.Escape(result);
}


int main(int argc, char* argv[]) {  
  // Initialize V8.
  V8::InitializeICUDefaultLocation(argv[0]);
  V8::InitializeExternalStartupData(argv[0]);
  Platform* platform = platform::CreateDefaultPlatform();
  V8::InitializePlatform(platform);
  V8::Initialize();

  // Get JavaScript script file from the first argument.
  FILE* file = fopen(argv[1],"r");
  fseek(file, 0, SEEK_END);
  size_t size = ftell(file);
  rewind(file);
  char* fileScript = new char[size + 1];
  fileScript[size] = '\0';
  for (size_t i = 0; i < size;) {
    i += fread(&fileScript[i], 1, size - i, file);
  }
  fclose(file);

  // Create a new Isolate and make it the current one.
  Isolate::CreateParams create_params;
  create_params.array_buffer_allocator =
    v8::ArrayBuffer::Allocator::NewDefaultAllocator();
  Isolate* isolate = Isolate::New(create_params);
  {
    Isolate::Scope isolate_scope(isolate);
    isolate_ = isolate;

    // Create a stack-allocated handle scope.
    HandleScope handle_scope(isolate);

    // Create a template.
    v8::Local<v8::ObjectTemplate> global = v8::ObjectTemplate::New(isolate);

    // Set a quit statement to global context.
    global->Set(
        v8::String::NewFromUtf8(isolate, "quit", v8::NewStringType::kNormal)
        .ToLocalChecked(),
        v8::FunctionTemplate::New(isolate, quit));

    // Create a new context.
    Local<Context> context = Context::New(isolate, NULL, global);

    // Enter the context for compiling and running the hello world script.
    Context::Scope context_scope(context);

    // Create a JavaScript console object.
    Console* c = new Console();
    Local<Object> con = WrapConsoleObject(c);
    // Set a console statement to global context.
    context->Global()->Set(String::NewFromUtf8(isolate, "console", NewStringType::kNormal).ToLocalChecked(),
        con);

    // Create a string containing the JavaScript source code.
    Local<String> source =
      String::NewFromUtf8(isolate, fileScript,
          NewStringType::kNormal).ToLocalChecked();

    // Compile the source code.
    Local<Script> script = Script::Compile(context, source).ToLocalChecked();

    // Run the script to get the result.
    Local<Value> result = script->Run(context).ToLocalChecked();
  }
  // Dispose the isolate and tear down V8.
  isolate->Dispose();
  V8::Dispose();
  V8::ShutdownPlatform();
  delete platform;
  delete create_params.array_buffer_allocator;
  return 0;
}

After that, set up a JavaScript script at test.js:

console.log("🎉");  
b=4+4;  
console.error(b);  
quit();

console.log("never reach this");  

Compile our new program with

clang++ -Iinclude out/native/*.a run_script.cpp -o run_script  

And run it with

./run_script test.js

You should see the following output

🎉
8  

Phew! That's it - congratulate yourself for building a crude version of Node that only supports console.log, console.error and quit statements.

Having a global function quit like this is a little weird, so implementing a more normal process.exit is left as an exercise for the reader.

If this article sparked your interest about Node internals, I strongly suggest you continue reading article named How does NodeJS work, which explains the concepts used in this article in greater detail.