Update 5 (June 23, 2015):
After posting the bad performance of Flask to their issue tracker User Methane posted a server config that boosts flask to 11751 requests/sec.
https://github.com/mitsuhiko/flask/issues/1512#issuecomment-114454800
Update 4 (Jan 10, 2015):
I added the sourcecode to a github repo: https://github.com/tschundeee/fibonacci-web-lang-benchmark
Plus I added ruby’s sinatra which ranges around 615.87 Requests/sec using Ruby 2.2.
Update 3 (Sept 22, 2014):
Rewrote the golang version using no external dependency. Now I got more than 63300 Requests/sec! Towards the python side I also had a little chat with @miguelgrinberg who was first confused to see Flask perform so bad compared to the others. After some tests he confirmed my results and he came to the conclusion that this might be a problem with Flasks underlying Werkzeug.
Update 2:
User asdf` in the #pocoo irc channel on freenode (Flask-Channel) redid a Python Twisted Framework Implementation that boosts req/s to 11k on his machine. Golang is the fastest again after setting GOMAXPROCS to 7 (7 out of 8 because 1 is reserved for wrk).
Update 1:
I added Sparkjava to the tests and even though it consumes more memory (about 140MB) it handles 48631 requests per second while Golang handles 37756 requests per second
The Task
Today I decided to do a little benchmark of some of my favorite web stacks. So I created a web service that calculates the corresponding fibonacci number to a request parameter. Go’s performance is way higher than node’s and python’s.
The Code
A request to for example http://localhost:3000/10 calculates fib(10) and returns the result (55 in this case) as plain text. I think it’s better to have a little computation per each request because it is more realistic than just serving “Hello World”.
Node + ExpressJS
I chose ExpressJS as it is node’s de facto web framework.
var express = require("express");
var app = express();var fib = function (n) {
if (n === 0) {
return 0;
} else if (n == 1) {
return 1;
} else {
return fib(n — 1) + fib(n — 2)
}
};app.get("/:number", function (req, res) {
var number = req.param("number");
var result = fib(number);
res.send("Node + Express<hr> fib("+number+"): "+result);
});app.listen(3000);
Python + Flask
For Python I chose Flask because it is very popular among Pythonistas (Geeky word for Python Programmers) and comparably small in size.
from flask import Flask
app = Flask(__name__)@app.route('/<int:number>')
def index(number=1):
return "Python + Flask<hr>fib("+ str(number) + "): "+ str(fib(number))def fib(n):
if n == 0:
return 0
elif n == 1:
return 1
else:
return fib(n — 1) + fib(n — 2)if __name__ == '__main__':
app.run(debug=False)
Python + Twisted
User asdf` in the #pocoo irc channel on freenode (Flask-Channel) redid a Python Twisted Framework Implementation that boosts req/s to 11k on his machine. Thx for this input.
import re
from twisted.web.server import Site
from twisted.web.resource import Resource
from twisted.internet import reactor
def fib(n):
if n == 0:
return 0
elif n == 1:
return 1
else:
return fib(n-1) + fib(n-2)
class FibonacciResource(Resource):
isLeaf = True
def __init__(self, num):
self.num = int(num)
def render_GET(self, request):
return "Python + twisted<hr>fib(%s): %s" % (self.num, fib(self.num))
class Router(Resource):
def getChild(self, path, request):
if re.match('\d+', path):
return FibonacciResource(path)
return self
reactor.listenTCP(5000, Site(Router()))
reactor.run()Golang + Gorilla Pat
As golang comes already with a nice net/http package one doesn’t really need a framework to handle requests. What golang lacks in my opinion is a routing system that supports url variables for it’s request handlers. Here I chose Gorilla Pat because it is small and for this test I don’t need additional stuff like subrouting, subdomains, etc.
UPDATE: I now used the original Pat Router and it’s way faster now.
package mainimport (
"fmt"
"net/http"
"strconv"
"runtime"
"github.com/gorilla/pat"
)// fib returns a function that returns
// successive Fibonacci numbers.
func fib(n int) int {
if n == 0 {
return 0
}
if n == 1 {
return 1
}
return fib(n-1) + fib(n-2)
}func IndexHandler(w http.ResponseWriter, r *http.Request) {
string_number := r.URL.Query().Get(":number")
number, _ := strconv.Atoi(string_number)
fmt.Fprintln(w, fib(number))
}func main() {
runtime.GOMAXPROCS(runtime.NumCPU()-1)
r := pat.New()
r.Get("/{number:[0-9]+}", IndexHandler)
http.Handle("/", r)
http.ListenAndServe(":4000", nil)
}
Sparkjava
This framework is a pretty lean package that makes use of Java8 Lambda support. It comes bundled with Jetty 9 even though it can also be deployed on other servers.
import static spark.Spark.get;public class HelloWorld {
public static void main(String[] args) {
get(“/:number”, (req, res) -> {
int number = Integer.parseInt(req.params(“:number”));
return “Sparkjava: fib(“ + number +”) = “ + fib(number);
});
} public static int fib(int n) {
if (n == 0) {
return 0;
} else if (n == 1) {
return 1;
} else {
return fib(n — 1) + fib(n — 2);
}
}
}
The Result
I tested with wrk with the following settings:
wrk -c 64 -d 30s http://localhost:PORT/10 1a. Golang + bmizerany Pat + GOMAXPROCS(7):
51684 Requests/sec => 1550508 requests in 30s1b. Golang + bmizerany Pat:
42222 Requests/sec => 1266661 requests in 30s
Average Latency 1.52ms1c. Golang + Gorilla Pat (using Gorillas Muxer)
37756 Requests/sec => 1132689 requests in 30s
Average Latency 1.71ms2. Sparkjava:
48631.24 Requests/sec => 1458768 requests in 30s
Average Latency 1.29ms3a. PyPy2.7 Python + Twisted:
12633 Requests/sec => 379001 requests in 30s3b. Python + Twisted:
3425 Requests/sec => 102781 requests in 30s3c. Python + Flask:
11751 Requests/sec => 16393 requests in 30s
Average Latency 55.54ms4. Node + Express:
8962 Requests/sec => 268866 requests in 30s
Average Latency 7.14ms
As one can see Spark’s & Go’s results are in another league than the other two. At least this is the case on my System.
Setup:
Hardware: Retina Macbook Pro i7@2.7GHz and 16GB RAM
Software: OSX 10.9.3
Java 8 + Sparkjava 2.0.0
Golang 1.3
Python 2.7.7 + Flask 0.10.1
PyPy 2.3.1 + Twisted 14.0.0
Node v0.10.29 + Express 4.4.4
A note about memory usage:
All of the three technologies didn’t consume much memory during the tests. Node was about 17MB, Python 14MB, PyPy about 76MB and Go about 5MB of RAM. Sparkjava consumed about 140MB of RAM even though this is pretty normal for the JVM.
Conclusion (Updated)
If you need raw computing performance for the web Golang might be for you. Sparkjava is also among the fastest in this test and the JVM is battle proved. Node + Express and PyPy + Twisted are also not slow. Python with Flask is slow compared to the others.
UPDATE: Flask is fast now using meinheld + gunicorn.
If you read this far you should follow me on twitter: @BijanRahnema ☺
