Rate limiting in Node.js with Promises

I’ve been re-familiarising myself with Node.js recently, and there’s a lot to catch up on since I last used it in anger. With the support for ES6 I’ve been looking through the changes listed and playing with features here and there to get acquainted with what can now be done. While many seem very nice when used in toy projects, I haven’t yet used them for a large enough system to say which ones may end up more trouble than they’re worth. Enter Promises.

I won’t go into the mechanics of Promises here, as there are already a good number of articles and references out there already, but instead I’ll look at a particular use case that I had. I was working on a small script to call a rate-limited REST API and thought this might be a good time to try a Promise-based library rather than the callbacks I’d used in the past. Fortunately rest exists, straightforward to use and with decent documentation to boot. Calling the API is as easy as:

Once the libraries are imported, the rest client is wrapped in Interceptors that each change how the call will be made, first adding a MIME type, prefix for the path, then an extra header. Finally we return client(options), which initiates the call and returns a Promise.

This was working pretty well by itself, but I still needed to introduce some rate limiting. To simplify my experiments I thought I’d create a very basic Promise that did nothing but log to the console, and then introduce the rest call later, so I set that rest call aside and started out with a very basic Promise:

As an initial limit, I thought one call every two seconds would be fine for testing. Wrapping setTimeout in a Promise, I can create a delay function:

To get the call happening at most once during each interval was fairly straightforward. Looking at Promise.all, it resolves only once all its promises have resolved or is rejected as soon as any one of those is rejected. This meant that I could call it with my task and the delay:

So now if I could just figure out how to repeatedly call it I could have the task running at most every two seconds, or longer if the task itself exceeds that time frame. I did find some suggestions on how to do this, and implemented a small loop using recursive calls:

This worked quite well, logging and then delaying for two seconds, then calling loop() again once Promise.all had resolved. Great! But this still bugged me a little, and there was a niggling doubt in the back of my mind. I wanted this to keep looping, after all, and I wasn’t confident enough in my understanding of the mechanics of promises to be sure that it wasn’t going to blow the stack, or create some issue with memory leaks with infinite Promises. Each loop created a new Promise that was resolved through calling and returning loop again. Doing some more digging, it did appear that I still might have a problem. This was confirmed when I found an interesting issue in the node.js issues list.

Reading through the issue, it did seem that using this pattern with Node.js native Promises would indeed cause a memory leak, as it continues to create a resolve chain of Promises until the loop has completed, which in this case would be never (or when the program stops). For a loop with a definite time frame it’s a non-issue as the memory is freed once resolution occurs, but for indefinite or long running loops it’s potentially problematic. As discussed in the github issue you can get around this by removing the return, which means that the loop continues but the Promises aren’t chained through resolution.

If you want to use any result as a Promise that solution obviously won’t work, but I found that bluebird‘s implementation of Promises did not have this behaviour and could be run indefinitely without increasing memory. It is able to do this by using an implemenation that varies slightly from the Promises/A+ spec that Node.js adheres to but not in a way that affects the semantics of the operations in which I was interested. It also provides a number of handy features for simplifying Promise-based workflows, including promisifying callback-based functions and composition of Promises. I didn’t delve too much into those for this exercise though, but at least I did finish my working loop:

You can also look at this bluebird issue to see some more examples. To get an idea of how the Node.js and bluebird promises compare, you can remove/reduce the delay, make the counter incremental and then add:

The Promise const can be commented out to use the Node.js Promise, or left in to use bluebird’s.

This entry was posted in Programming and tagged , , , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *