To use the examples in this book, you will want to have a Node.js server up-and-running so you will be able to leverage the many samples contained within this book. I would suggest by visiting the Installing Node.js page where you can download and install the latest version for your operating system of choice.
How to contact
To comment or ask technical questions about this book, send an email to:
For thousands of more recipes, visit our online version of the site at Node.js Recipes.
Find us on Facebook: https://www.facebook.com/nodejsrecipes/
Find us on Twitter: https://twitter.com/nodejs_recipes
The Recipes
Managing lots of callback recursion in Nodejs
Problem
In Nodejs, there are virtually no blocking I/O operations. This means that almost all nodejs IO code involves many callbacks. This applies to reading and writing to/from databases, files, processes, etc. A typical example of this is the following:
var useFile = function(filename,callback){ posix.stat(filename).addCallback(function (stats) { posix.open(filename, process.O_RDONLY, 0666).addCallback(function (fd) { posix.read(fd, stats.size, 0).addCallback(function(contents){ callback(contents); }); }); });};...useFile("test.data",function(data){ // use data..});
I am anticipating writing code that will make many IO operations, so I expect to be writing many callbacks. I'm quite comfortable with using callbacks, but I'm worried about all the recursion. Am I in danger of running into too much recursion and blowing through a stack somewhere? If I make thousands of individual writes to my key-value store with thousands of callbacks, will my program eventually crash?
Am I misunderstanding or underestimating the impact? If not, is there a way to get around this while still using Nodejs' callback coding style?
Problem courtesy of: Maciek
Solution
None of the code you show is using recursion. When you call useFile
it calls posix.stat()
, which returns, and useFile
terminates as it has run to completion. At some later time, when the call to posix.stat()
has completed within the underlying system and the results are available, the callback function you added for that will be executed. That calls posix.open()
, and then terminates as it has run to completion. Once the file has been successfully opened, the callback function for that will execute, calling posix.read()
, and will then terminate as it, too, has run to completion. Finally, when the results of the read are available, the innermost function will be executed.
The important point is that each function runs to completion, as the calls to the posix.*()
functions are non-blocking: that is, they return immediately, having caused some magic to be started off in the underlying system. So each of your functions terminates, and later an event will cause the next function to execute; but at no point is there any recursion.
The nested structure of the code can give one the impression that the stuff inside will have to finish before the stuff outside can get to its own end point. But in this style of asynchronous event-driven programming it makes more sense to see the nesting in terms of deeper => happens-later-than.
EDIT: Try adding some logging statements immediately before the end of each nested function; this will help to illustrate that the order in which they complete is from the outside inwards.
Solution courtesy of: NickFitz