Input / output operations, e.g. file reading/writing and network transactions, are much slower than CPU. Introductory programming examples (e.g. in Java and Python) typically use synchronous function call to perform I/O. Each file operation takes relatively long time to complete, and while the function call is waiting the I/O completion, the process / thread is blocked. When the I/O is completed, the program can continue to the next line in the program.
import java.io.File;
import java.util.Scanner;
public class JavaExample
{
public static void main(String[] args) throws Exception
{
// wait some time for the file open
File file = new File("test.txt");
Scanner sc = new Scanner(file);
while (sc.hasNextLine())
// wait some time to read some text from file
System.out.println(sc.nextLine());
}
}
However, there is only 1 thread to run JavaScript code in most platforms (one thread per browser window, one thread per Node.js program). If this thread blocks to wait for I/O completion, the whole Node process is blocked and cannot process other events. Consider the following example.
There is exception. A Node.js that can only use 1 thread results in bad CPU utilization of multi-core CPUs. So modern JavaScript platforms support multiprocessing, e.g. Web workers in web browsers, and worker threads in Node.js.
Therefore, JavaScript libraries employ asynchronous function call to handle I/O completion. Consider the following example, in which the program sends a DNS query to a name server to find the IP addresses of a domain name. The function call dns.resolve() starts the network transaction (i.e. sends a DNS query), and registers a callback function for I/O completion. But it does not block. When the name server returns a reply, the event loop calls the callback function.
/// p111-a.mjs
import dns from 'dns';
const govHost = "www.gov.mo";
dns.resolve(govHost, (err, records) => {
if (err) {
console.error(err); return;
}
console.log(`${govHost} resolves to ${records}`);
});
// this runs before we got an answer from a name server
let x = 1+2;
// event loop
Similarly, file input/output usually use asynchronous callback to handle I/O completion. For more information, read online reference for fs.readFile and fs.writeFile.
// p112.mjs
import fs from 'fs';
/* function getPrime() returns an array of prime numbers */
const textData = genPrime(200).toString();
fs.writeFile('./prime.txt', textData, 'utf8', (err)=>{
if (err) throw err;
console.log('Some prime numbers are written to prime.txt');
});
// at this point, file write is still in progress
// ...
// event loop waits for writeFile to finish
// p113.mjs
import fs from 'fs';
fs.readFile('./prime.txt', 'utf8', (err, data)=>{
if (err) throw err;
let primes = data.split(',');
console.log(`${primes.length} prime numbers found in prime.txt.`);
console.log(`The last 5 are ${primes.slice(-5)}`);
});
// ...
// event loop waits for readFile to finish
The three I/O operation above (dns.resolve, fs.readFile, fs.writeFile) may either fails with an Error or succeeds and return the data requested. Node.js typically uses completion callback with two parameters to handle these. The first is often an Error object err to report I/O error. If no error has occurred and the I/O operation completes successfully, err equals null or undefined, and the result of the I/O operation is given in the second parameter.
In the examples, we
throwthe error object while the event loop calls the callback function. Since the event loop does not catch any exception, throwing the error object will abort the Node.js process.
And remember, I/O completion callbacks usually are called by the event loop, and usually after all statements of the JavaScript program has executed. In this aspect, completion callbacks are similar to the callbacks in one-time timers.
setTimeout(
/* this callback runs 10min later, after all the statements below */
()=>{ },
10*60*1000
);
// ...
// ...
// event loop
The execution time of asynchronous callbacks make the proper sequencing of I/O operations confusing to beginning JavaScript programmers. To properly order two I/O operations, you often need to put the second I/O operation inside the callback of the first I/O operation. Consider the following example of copying the file mpi-info.txt to copy-mpi-info.txt. example p115.mjs
// p115.mjs - file copy
import fs from 'fs';
fs.readFile('mpi-info.txt', 'utf8', (err, data) => {
if (err) throw err;
// at this moment, file read is done
// `data` contains content of the input file
fs.writeFile('copy-mpi-info.txt', data, 'utf8', (err) => {
if (err) throw err;
});
});
If the two calls of readFile and writeFile are not nested, but put into a sequence as in example p116-mistake.mjs, writeFile will not receive any data from readFile.
In general, doing some I/O operations in sequence would involve nesting callback functions. The result is sometimes known as callback hell in the Node community. We’ll explain a modern JavaScript language feature called Promise that mitigates this problem.
// p117.mjs - concatenate two files
import fs from 'fs';
fs.readFile('p115.mjs', 'utf8', (err, file1Data)=>{
if (err) throw err;
fs.readFile('p116-mistake.mjs', 'utf8', (err, file2Data)=>{
if (err) throw err;
fs.writeFile('combined.txt', file1Data+file2Data, 'utf8', (err)=>{
if (err) throw err;
});
});
});
On the other hand, some I/O processes detect some kind of event several times (0, 1, 2 or more), and need to run a callback function asynchronously to handle each event. For example, a web server runs a callback whenever it receives an HTTP request from a browser. The following example uses the built-in module http to write a simple web server.
// p114.mjs
import http from 'http';
let server = http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.write("Hello browser");
res.end();
});
// ...
Try to run the example p114.mjs. The HTTP server actually support several more kinds of events, and you can register event handler for each kind of event.
Event handlers are similar to the callback function of a periodic timer. They are also similar to event handlers in web browsers.
setInterval(
/* this callback runs every minute, after all the statements below */
()=>{ },
60*1000
);
// ...
// ...
// event loop
The example source files are available here.