Experiments · JavaScript: From Closures to Autonomous Systems · Chapter 5

A function warming up

Tiered JIT compilation in JavaScript

Theme
Try itA function warming up
Press the button and the browser calls the book’s dot a few thousand times from a cold start. The chart shows what a call costs at first and after the engine has compiled it, with V8’s tiers from the book as bands. “Again” takes a new cold copy of the function, and the drop may land on a different call.

Everything is computed in your browser; nothing is sent anywhere.

About this experiment

What is tiered compilation in JavaScript?

A JavaScript engine does not compile a program ahead of time. It starts running code in an interpreter, noting which types pass through each operation, and turns a function into machine code only once it is called often. There are several such tiers. In V8 they are the Ignition interpreter, the simple Sparkplug compiler, Maglev, a fast optimising compiler, and TurboFan, which builds the best code and spends the most time on it. The optimising compilers rely on the recorded types; if the code later meets other types, the optimised version is thrown away and the function drops back down.

In May 2017 V8 settled on a pair, the Ignition interpreter and the TurboFan compiler. Sparkplug was added between them in 2021 and Maglev in 2023, with Chrome 117. Each tier answers the same question in its own way: what compiling costs and what it will bring back. A function that is called often and behaves the same way each time gets moved up, and V8’s log gives the reason in three words, hot and stable.

The button calls dot, the function from the book’s experiment, the dot product of two arrays of 256 numbers, eight thousand times from a cold start in your browser. The chart puts the call number across and the time of one call up, both on log scales. Browsers coarsen their clock, so the calls go through several cold copies of the function and in batches, and hollow points are samples at the limit of the clock’s precision. “Again” compiles a new copy, and everything starts over in the interpreter.

In the book, on Node 24 and an M4 Max, a call took 7.0–7.4 µs in the interpreter, 0.39–0.64 µs on Maglev and 0.19–0.36 µs on TurboFan. Over five runs Maglev took over at calls 36 to 180 and TurboFan at calls 1620 to 3780; in one run it did not arrive within 4060 calls at all. The call on which the switch happens cost 17–300 µs.

The chapter draws a conclusion about predictability from this. The program always gives the same answer, but its speed depends on how it ran before and on how the operating system shared time with the compiler’s background threads. That is why every measurement in the book is taken after a warm-up, in separate processes and as a median rather than an average, and why two presses of the button can draw two different charts.

From the book

JavaScript: From Closures to Autonomous Systems · Chapter 5

From Interpreter to JIT

Next to this experiment in the chapter
[marking ... for optimization to MAGLEV, ConcurrencyMode::kConcurrent, reason: hot and stable] The words «hot and stable» describe the engine’s whole strategy. Compilation costs time and memory, so it makes sense to spend them only on code that is called often and behaves the same way every time. Otherwise the optimized code will have to be thrown away at the first change of types. The engine invests in a function the way a cautious investor invests in a company that has turned a profit several quarters in a row, and the longer the function lives up to expectations, the more is invested in it.