Computing the minimum number of Heroku dynos from the New Relic API

In a previous post, we described how to query the API for New Relic, a service that we use to monitor our Rails apps.  In this post, we give an example of how this data can be used to adjust the number of front-end server units (“dynos”) for Heroku, a Platform-as-a-Service (PaaS) that we use to host our Rails apps.

Let’s do some math.  Here are two numbers that can be read from the New Relic API:

• Response Time (ms) =  milliseconds per request
• RPM = requests per minute

For simplicity, let’s convert these both into seconds and define:

• RTS = seconds per request, which is given by Response Time (ms) / 1000 
• RPS = requests per second, which is given by RPM/60 

The basic idea is that at the minimum, there should be enough dynos to handle the responses.  If we take the reciprocal of the Response Time (s), that gives the # of requests/second that can be handled by one dyno.  Suppose we have D dynos - then the capacity of our system is:

D / RTS 

Meanwhile, the number of requests per second we are actually seeing is RPS.  To have enough capacity to handle these requests, we must have

D / RTS > RPS

so that a minimum on the number of dynos is given by

D > RTS * RPS       

That’s it!  Expanding this out gives:

min dynos > (Response Time in ms)/1000 * RPM/60

This gives a simple formula for the minimum number of dynos needed for a Heroku app, just based on the New Relic monitoring data.  Very cool!

Of course, in practice there is variation in the traffic pattern and in the response time, which makes it necessary to use a higher number of dynos than this minimum.  As a result, we need to introduce some other adaptive algorithms that look at other inputs, such as the Heroku queue depth and queue wait, and response time for an HTTP GET.

______

John