Deploying a Selenium/PhantomJS app to Heroku in 4 to 5 (and a half) steps.

The Phantom(JS) of the Opera

So we wanted to find a way to just test that our different WebApps and WebPages were up and running without having to check them ourselves every hour or be told by our customers (which is always embarrassing).

The options were to either use an existing service or write something ourselves, and since I already had built something similar in the past with one of my previous coworkers (which I thank for the knowledge shared) I thought “I can do it!”.

And I did. Even though it was painful.

I have written my code in Ruby and used Selenium WebDriver and PhantomJS to be able to perform all my tests without needing an “actual” browser, this way all the tests are performed by Selenium on PhantomJS and we just get to see the results in the terminal.

I am going to list the steps and have a verbose intro and a summary for each of them. Depending on your level of confidence with programming you might want to skip the intro if you already know your stuff.

If you want to get a basic layout, you can fork my repo on GitHub and try it out :)

Step 1. The Gems

Intro

I have used very few gems to set everything up, I’m going to tell you a bit about each of them.

• slack-notifier: (see it in step 3.2) I wanted my app to notify me on Slack whenever something went wrong with my tests, so I got the slack notifier gem and built a very small and cute bot that looks awesome. The slack-notifier gem is totally optional, if you don’t have a passion for slack notifications or coolness in general you can skip this. I don’t judge. In case you are cool though, here is where you can find all the details about building your own slack bot using the slack-notifier gem.

I am particularly proud of the little ghost, because.. phantom.. HA!

• phantomjs: You will need this. This is what allows you to run all of your tests and actually navigate without a browser. You can see PhantomJS as a fake invisible browser that your code can use as if it was a person typing the address on Chrome or Safari or whatever you would use to do it manually. If you don’t want to use PhantomJS because *reasons I don’t care about*, you can instruct Selenium to use a real browser to perform all of the tests you need. But I’m not going to get into that because *reasons you don’t care about*.
• selenium-webdriver: (see it in step 2 — summary) Selenium does all the cool stuff. The webdriver is the invisible little user that is going to allow you to go to a page ( .get ), search for stuff ( .find-element ), click on stuff and other nice things that you would do yourself if you were testing manually for something to work properly. You need this.
• dotenv: (read more in step 2 & step 3.0) So I have had the need to use dotenv because I didn’t want my slack URI to be in my code, so what dotenv does is allowing you to store and use information that is only going to be visible to you in your local environment (if you do yourself a favour an gitignore it). You don’t necessarily need this, especially if you are not going to build a very cool slackbot that tells you cute things with a ghost emoji.
• rake: (see it in step 4) Rake is a task manager, you write in a rakefile what you want to happen to your other files and when, then you execute your rakefile and it takes care of all of your tasks for you. If you don’t want to have to instruct Heroku on several different commands to be executed in order to run all of your tests, you might want to have rake and write a rakefile.

Summary

Gemfile:

‘https://rubygems.org'
ruby ‘2.3.1’
gem ‘slack-notifier’
gem ‘phantomjs’
gem ‘selenium-webdriver’, ‘~>2.53.4’
gem ‘dotenv’
gem ‘rake’

Now $ gem install bundler and $ bundle install in your project root and you are almost done with step 1.

The only thing missing that is not a gem but still needs to be installed is the executable for PhantomJS. You can’t expect all of this magic to happen without having to install the headless browser, even though you don’t see it, it is still a browser! So $ npm install -g phantomjs and let’s move on.

Step 2. The Helper

Intro

You will need a helper in case you plan on having more than one test file, so that you can move the pieces of code you intend to reuse on a separate file that can then be called in each of your test-files. In the helper you are going to have all of your require statements.

In my case, I needed to require the following:

require ‘phantomjs’
require ‘selenium-webdriver’
require ‘time’
require ‘dotenv’
Dotenv.load

I added the Dotenv.load at the end because apparently Ruby does not know what to do after requiring dotenv and you have to tell it to load it too or the ENV variables are not going to be recognized and the slack-URI will not be set as an environmental variable. If you don’t plan on using the slackbot you won’t need to worry about this unless you plan on having other things you want to keep private.

My helper hassetup, run and teardown methods.

• The setup is going to define my driver and the kind of browser that the driver is going to use: @driver = Selenium::WebDriver.for :phantomjs , if you plan on using another browser just put the name of the browser you intend to use in place of :phantomjs . After running into some troubles during my tests since I couldn’t find my elements in the browser window, I took a snapshot of it and found out that the size of my phantom browser window was for mobile, so my elements were hidden and my tests would fail even though the page was up, so I resized the window like this: @driver.manage.window.resize_to(1300,500) and then everything worked.
• The run method has everything that is going to happen when the file loads, if you don’t want to print out a nice message that is going to tell you how long your test took, you can skip start_time and end_time and puts

def run
  start_time = Time.now

  setup
  yield
  teardown

  end_time = Time.now
  puts 'Total test duration: ' + (end_time - start_time).round(2).to_s + ' seconds'
end

• The teardown is just going to terminate the driver with @driver.quit .

So basically we define our driver in the setup, we quit it in the teardown and we put setup and teardown in our run method so that they happen in order and allow us to yield our code blocks.

Summary

test_helper.rb in short:

require 'phantomjs'
require 'selenium-webdriver'
require 'time'
require 'dotenv'
Dotenv.load 
def setup
  @driver = Selenium::WebDriver.for :phantomjs
  @driver.manage.window.resize_to(1300, 500)
end 
def teardown
  @driver.quit
end 
def run
  setup
  yield
  teardown
end

Now you can require ‘test_helper.rb’ in your test files and whatever code block you are going to have is going to be passed to yield and follow the same flow as instructed in the helper. 🎉

Step 3.0 (OPTIONAL). Create Your SlackBot

This doesn’t even require an intro or a summary. Your slackbot.rb file should look something like this:

require 'slack-notifier'

notifier = Slack::Notifier.new ENV['SLACK_URI']
notifier.ping "Something's wrong! Go to <#{@appLink}|#{@appName}> for details! "

slack-notifier is the gem we talked about in step 1. ENV is the environmental variable “key” used by our gem dotenv and allows us to access the variables written in the .env file. (this is how you write and environmental variable in your .env file = SLACK_URI=https://yourLinkGoesHere ). ping is a method coming from the slack gem that sends the notification to the SLACK_URI of your choice.

Step 3.1. Write Your Tests

Intro

Create a folder that is going to include all of your test files. Each one of your test files will needs to have require ‘test_helper.rb' at the first line of the file in order for the code you write in it to be executed in the test_helper and for your driver to be defined.

In the examples below you will see the file including the variables and methods for using the slackbot and the one that doesn’t need them.

In the slackbot example I have saved my app’s name and link into variables that I want to use in the slackbot.rb file (half a step up). This is due to the fact that I have different apps I want to test and I want the test file to define the variable @appName and @appLink with wathever I am going to test at the moment.

In the examples reported in this section’s summary you will see a begin — rescue block. I decided to use a Ruby Exception because this way, if the driver is not able to get the link or the element is not found, the file’s execution will not crash and will instead print a failure message (or in the case of the slackbot, send a notification to my channel 👻). I could have probably used Selenium Expectations but this seemed easier to me.

Other people might have different opinions about this and I’m fairly new to the whole programming thing so don’t just take my word for it 😸

For those who are very new to coding (newer than me at least 😄), I want to clarify that I am able to use the @driver variable in the test-file because I defined it in the helper and I imported the helper in my test file, so everything contained in it is going to follow. The same concept applies for @appName and @appLink , which I define in my test files and then use in my slackbot file after loading it in the environment where the variables were defined.

Summary

Example of test-file with slackbot:

run do
  @appName = 'MyApp'
  @appLink = 'https://myapp.com'

  begin
    @driver.get @appLink
    @driver.find_element(yourSelectorOfChoice: 'example')
    puts 'Congrats! Your website is up!'
  rescue
    puts 'Whoops! Something went wrong here :('
    load './slackbot.rb'
  end
end

Example of test-file without slackbot:

run do

  begin
    @driver.get 'https://myapp.com'
    @driver.find_element(yourSelectorOfChoice: 'example')
    puts 'Congrats! Your website is up!'
  rescue
    puts 'Whoops! Something went wrong here :('
  end
end

yourSelectorOfChoice is whatever kind of selector you choose to use together with find_element .

Step 4 (OPTIONAL). Write Your Rakefile

Intro

If you want to keep your Heroku scheduler clean, you will want to have a Rakefile in your project, so that you can execute all your different tests by only calling one action.

I ran into several troubles while setting up my Rakefile due to the fact that I kept giving the different files wrong paths, but after much struggling I manage to find a tree structure that made everything work.

In case you don’t want to go over my repo, this is how the tree structure looks like.

Summary

So this is how a rakefile can look like!

desc "Goes to myapp.com"
task :tests_myapp do
  ruby './tests/test-file.rb'
end

task :default => [:tests_myapp]

To add more tasks to is just make another task and add it to the list of tasks like this:

desc "Goes to myapp.com"
task :tests_myapp do
  ruby './tests/test-file1.rb'
end
desc "Goes to myotherapp.com"
task :tests_myotherapp do
  ruby './tests/test-file2.rb'
end
task :default => [:tests_myapp, :tests_myotherapp]

etc..

Step 5. Deploy To Heroku

Intro

This part caused me a real headache. I just did not know why PhantomJS would not work once added my project on Heroku and writing my command in the scheduler to run my tasks. I googled a lot but I somehow managed to always open the wrong StackOverflow question 😅

The reason was that I needed to add a webpack to make the PhantomJS executable available to Heroku, since I only installed it on my computer and Heroku had no clue how to get to it!

Once I got it, the implementation was more than easy.

Summary

Step-by-step:

• Go to Heroku and log in.
• Click on apps and create a new one
• Connect your new app to your GitHub repository
• Click on Overwiew and install a Heroku add-on called “Scheduler”

With the rakefile:

• Go into the Scheduler and add bundle exec rake -f tests/rakefile.rb or whatever the path to your rakefile is going to be from your project root.

Without the rakefile:

• I have not done this myself, but I guess you would need to add a new job for each test-file and then runruby tests/test-file.rb instead of rake.

Then:

• Go to your terminal and install the Heroku CLI
• Connect to your Heroku app and add the PhantomJS webpack by typing in your terminal:

$ heroku buildpacks:add https://github.com/stomita/heroku-buildpack-phantomjs -a your-project-name

Now deploy and be happy! Your tests should run according to the Scheduler (every 10 minutes, hourly or whatever you chose) and you will see the messages in the Heroku logs!

Happy pinging! ❤