18
A Runtime for a Nondeterministic Executor
In all the previous chapters the runtime placed bets on a program written by a human. V8 guessed what shape of objects would come into a function, Meteor guessed what the server would answer, Temporal required the code to behave on replay the same way as the first time. The program itself was known in advance. You could read it, check its types, run tests, and everything it could do was written in its text.
With an agent, things are different. In 2022 Shunyu Yao and his co-authors described a scheme they called ReAct: a language model alternately reasons and acts. It receives a task, writes what it is going to do, calls a tool, receives the result, reasons again and decides what to do next. In JavaScript such a loop fits into a dozen lines:
while (true) {
const reply = await model(messages, tools);
messages.push(reply);
if (reply.stop_reason !== 'tool_use') break;
const results = await runTools(reply.toolCalls);
messages.push({ role: 'user', content: results });
}
The model answers either with text, and then the work is done, or with a request to call a tool with such-and-such arguments. The code calls the tool, puts the result into the conversation and calls the model again. Which tool it will ask for on the fifth step and with which arguments cannot be learned from this code. The program’s next action is chosen at run time, and it is chosen by an executor that can be wrong.
Usually, what lies between the text of a program and its execution is checked for the programmer by someone else. Syntax is checked by the parser, types by the TypeScript compiler, process permissions by the operating system. In an agent loop all these checks have to be done anew at every step, because the program is written anew at every step. The code around the model that does this has come to be called the harness.
Free, one step away
The rest of the book is free
Leave your email and I will send a link to confirm it. Once you click it, every chapter opens here, on this device, for a year. When a new book comes out I will write to this address; reply “stop” to any email and I will not write again.
Have an INITE account? Sign in and the book opens without an email.
I do not charge for the book. If you would like to support it, you can donate from any chapter.
How your email is stored