Invoking background processes from NodeJS

Maybe you are in a situation in which you have to integrate your code with legacy systems, or with systems that are being developed at the same time but with different technologies, or maybe even console scripts. For these purposes, NodeJS has a powerful module, called child_process, which allows the invocation of processes just like if you were running them from the console. In this post, we explain some key points about how to invoke, synchronize and kill background processes launched from NodeJS.

Invoking processes with Spawn

Let’s start with a simple but complete example, let’s suppose we want to execute an “ls” command to list the content of the /usr directory. To achieve that, let’s write the following code in a new file:

const { spawn } = require('child_process');
const ls = spawn('ls', ['-lh', '/usr']);

ls.stdout.on('data', (data) => {
  console.log(`stdout: ${data}`);
});

ls.stderr.on('data', (data) => {
  console.log(`stderr: ${data}`);
});

ls.on('close', (code) => {
  console.log(`child process exited with code ${code}`);
});
  1. First we have to import spawn: const { spawn } = require(‘child_process’);

  2. Next, we call spawn with the necessary parameters to invoke our ls command, and we assign it to a constant: const ls = spawn(‘ls’, [‘-lh’, ‘/usr’]);  The constant ls is now the reference to the object that represents the child process we have just invoked.

  3. By default the pipes for stdin, stdout, and stderr are established between the parent NodeJS process and the child process. Those pipes have specific and platform limited capacities.

  4. We detect the close event of the invoked process as well as its return code: ls.on(‘close’, (code) => {});

This our little program output:

$ node server.js

stdout: total 0
drwxr-xr-x  972 root wheel    30K 5 jun 18:35 bin
drwxr-xr-x  304 root wheel   9,5K 22 feb 13:29 lib
drwxr-xr-x  249 root wheel   7,8K 5 jun 18:35 libexec
drwxr-xr-x    7 root wheel   224B 25 feb 10:31 local
drwxr-xr-x  239 root wheel   7,5K 22 feb 13:25 sbin
drwxr-xr-x   46 root wheel   1,4K 22 feb 13:25 share
drwxr-xr-x    5 root wheel   160B 5 feb 05:41 standalone

child process exited with code 0

Spawning detached processes 

When a child process is invoked the parent keeps waiting for it to finish before finishing itself. Sometimes, especially with long running processes, you might find interesting to have both processes executing independently from each other. This way, the NodeJS process may finish while the invoked may keep running. To achieve this, we just need the unref() function and the detached option.

Let’s see how to do it: 

const { spawn } = require('child_process');

const subprocess = spawn(process.argv[0], ['child_program.js'], {
  detached: true,
  stdio: 'ignore'
});

subprocess.unref();

Thanks to invoke the unref() function the parent event loop does not include its child so the parent will finish no matter whether its child has already finished or not.  The problem is if we want the child to keep running once the parent has finished, if so it’s necessary to indicate an stdio configuration that is not dependent from the NodeJS process. This can be achieved by ignoring the standard i/o at all or by dumping it to an external file, like the one used on the following example:

const fs = require('fs');
const { spawn } = require('child_process');
const out = fs.openSync('./out.log', 'a');
const err = fs.openSync('./out.log', 'a');

const subprocess = spawn('prg', [], {
  detached: true,
  stdio: [ 'ignore', out, err ]
});

subprocess.unref();

Killing the child processes

Let’s suppose we have a child process which fails but it doesn’t finish by itself, or maybe it gets stuck and we want to kill it on a given timeout. In those cases we need the parent process to kill it, and this is possible with the ‘detached’ option.

To do this, the NodeJS parent process must send a signal to its child indicating the kill. We can do this using the kill() function including the desired signal, if we don’t specify one, SIGTERM will be sent by default. 

const { spawn } = require('child_process');
const grep = spawn('grep', ['ssh']);

grep.on('close', (code, signal) => {
  console.log(
    `child process terminated due to receipt of signal ${signal}`);
});

// Send SIGHUP to process
grep.kill('SIGHUP');

Invoking background processes 

We are going to use the previous  concepts to put it all together in the same example of invoking background processes.

In this example, we have a parent process, which is a NodeJS server. Let’s suppose its domain layer needs to produce and consume data given by the execution of a few programs (e.g. written in C), that we will invoke them as child processes. These children are long-process-1 and long-process-2.

exports.invokeChildren = params => {
  const childProcess = spawn('long-process-1', [params], {detached: true}).childProcess;
  const anotherChildProcess = spawn('long-process-2', [params], {detached: true}).childProcess;

  childProcess.stdout.on('data', async data => {     });
  childProcess.stderr.on('data', async data => {     });

  return childProcess.pid;
}
exports.invokeAnotherChildren = params => {
  const anotherChildProcess = spawn('long-process-2', [params], {detached: true}).childProcess;

  anotherChildProcess.stdout.on('data', async data => {     });
  anotherChildProcess.stderr.on('data', async data => {     });

  return anotherChildProcess.pid;
}

Those functions provoke the execution of the child processes and run in a totally independent way. A real example could be an API call to an endpoint that causes the invocation of one or many of these functions.

To make sure that a process does not run forever due to a non controlled exit or because it’s taking longer than expected, it is a good practice to kill the invoked processes on a timeout. This can be done right after the spawn, let’s see the example again with this functionality:

exports.invokeChildren = params => {
  const childProcess = spawn('long-process-1', [params], {detached: true}).childProcess;
  const anotherChildProcess = spawn('long-process-2', [params], {detached: true}).childProcess;

  // Stop job after max time
  const timeout = setTimeout(async () => {
      const error = { pid: childProcess.pid, code: -1, message: 'Process killed on timeout'};  
      logger.error(error);
      process.kill(childProcess.pid);
  }, 20000);

  childProcess.stdout.on('data', async data => {     });
  childProcess.stderr.on('data', async data => {     });

  return childProcess.pid;
}

After 20 seconds running we want to kill the process, to keep a record of this, first we build an error object and pass it to a logger, then we use process.kill() to kill the process.

Conclusions

Despite we have not covered all the details and functionalities, I hope this post is useful to see the power that child_process module offers. With a few simple concepts we can integrate the execution of any external process into our application, the possibilities are infinite.

Leave a Comment

Responsable » Solidgear.
Finalidad » Gestionar los comentarios.
Legitimación » Tu consentimiento.
Destinatarios » Los datos que me facilitas estarán ubicados en los servidores SolidgearGroup dentro de la UE.
Derechos » Podrás ejercer tus derechos, entre otros, a acceder, rectificar, limitar y suprimir tus datos.

By completing the form you agree to the Privacy Policy

¿Necesitas una estimación?

Calcula ahora