Headless Chrome vs PhantomJS Benchmark

At SERP API, being able to provide real results the fastest is a daily concern. However, running full browsers is an expensive task and finding the best solution is not easy.

Trends seem to favor more and more using Headless Chrome over PhantomJS when an automated browser is needed. The rumor is that the new headless mode of Chrome is both faster and less memory intensive than PhantomJS. Unfortunately, I wasn’t able to find any benchmarks to back up or to rebut this statement. So, let’s just do this!

We are going to use Ruby Selenium WebDrivers to run both PhantomJS and Headless Chrome. Results might vary slightly if you are using other drivers, but we expect anyway the vast of majority of the charge being the browser itself not the library wrapper.

The benchmark loads 1,000 times the Rails default page and checks for integrity. I’ve run it on my 2017 MacBook (1.4 GHz and 16 GB of Ram) with Ruby 2.3.3p222, Chrome 60.0.3112.113 and PhantomJS 2.1.1.

Here are the results:

As you can see, Headless Chrome finishes 55% faster while consuming 38% less memory than PhantomJS. Headless Chrome seems also more stable in performance when running the benchmark again and seems to be the big winner here.

Here the source code of the main benchmark logic:

URL_TO_TEST = "http://localhost:3000"
TEXT_TO_VERIFY_PROPER_LOAD = "Yay! You’re on Rails!"
BENCHMARK_ITERATIONS = 1000

namespace :benchmark do

  task :phantomjs do
    puts "== Starting PhantomJS Driver =="
    driver = Selenium::WebDriver.for :phantomjs
    puts "Benchmarking (#{BENCHMARK_ITERATIONS} times):"
    time = Benchmark.measure do
      BENCHMARK_ITERATIONS.times do 
        driver.navigate.to URL_TO_TEST
        unless driver.find_element(tag_name: "body").text.include? TEXT_TO_VERIFY_PROPER_LOAD
          raise "Page Not Properly Loaded"
        end
        print '.'
      end
    end
    puts "\nTime taken: #{time}"
    puts "== Quitting PhantomJS Driver =="
    driver.quit
  end

  task :headless_chrome do
    puts "== Starting Headless Chrome Driver =="
    headless_chrome_capabilities = Selenium::WebDriver::Remote::Capabilities.chrome(chromeOptions: { args: [ "--headless" ]})
    driver = Selenium::WebDriver.for :chrome, desired_capabilities: headless_chrome_capabilities
    puts "Benchmarking (#{BENCHMARK_ITERATIONS} times):"
    time = Benchmark.measure do
      BENCHMARK_ITERATIONS.times do 
        driver.navigate.to URL_TO_TEST
        unless driver.find_element(tag_name: "body").text.include? TEXT_TO_VERIFY_PROPER_LOAD
          raise "Page Not Properly Loaded"
        end
        print '.'
      end
    end
    puts "\nTime taken: #{time}"
    puts "== Quitting Headless Chrome Driver =="
    driver.quit
  end
  
end

On GitHub: https://github.com/hartator/benchmark-headless-chrome-vs-phantomjs/blob/master/lib/tasks/benchmark.rake

Full GitHub if you want to run or modify the benchmark code yourself: https://github.com/hartator/benchmark-headless-chrome-vs-phantomjs