FROM THE ARCHIVES OF PRAGPUB MAGAZINE JANUARY 2011
Everyday JRuby
Part 2: Sharing Your Software
By Erin Dees
Erin continues her series series on Everyday JRuby by showing how to build a game and share it with others.
People are using JRuby for amazing things worldwide. They’re also using it at smaller scales to make their day-to-day work a little easier. This series is an exploration of everyday situations where JRuby comes in handy.
In the previous installment, we discussed how to extend a large engineering environment like MATLAB using JRuby (and its statically typed cousin, Mirah). The plugin we ended up with was easy to deploy, because it didn’t have any dependencies other than the basic JRuby runtime.
For this article, we’ll look at a slightly more complicated case. We’ll build a data-backed Web application that uses Java libraries, Ruby gems, and static template files. When it comes time to share our app, we’ll need to make sure all the pieces get packaged up correctly.
Six Degrees
At work, my colleagues and I occasionally need to throw together a simple Web front end for some engineering data. But for this article, let’s look at something much more interesting (and less likely to get me fired): the movies.
We’re going to implement an old party game known as “Six Degrees of Kevin Bacon.” In this game, players choose an actor and then try to find as direct a connection as possible between that actor and Kevin Bacon by looking at movie roles. For example, we can get from Kevin Bacon to Danny Glover in two steps: Kevin Bacon starred in JFK with Kevin Costner, who in turn starred in Silverado with Danny Glover.
Connecting to the Database
The standard “how to write a blog server” tutorial has you stash your data in a relational database like MySQL. But since finding connections between actors is essentially a graph traversal activity, we’ll use Neo4j, an open source graph database, instead.
For this exercise, you’ll run Neo4j as a standalone server and connect to it from Ruby, just as if you were handed a server address and login credentials on a real-world project. Download and extract Neo4j 1.2 from the official site. Launch the background server, and leave it running for the remainder of the experiment:
$ ./bin/neo4j start
On the Ruby side, we’re going to use a dependency management tool called Bundler to install Ruby libraries. First, install Bundler:
$ jruby -S gem install bundler
Now, create a file called Gemfile in your project directory with the following contents:
source 'http://rubygems.org'
gem 'neography', '~> 0.0.7',
:git => 'git://github.com/undees/neography.git',
:branch => 'warbler'
gem 'jruby-openssl', '~> 0.7'We’ll be using a Ruby wrapper around Neo4j called Neography. The :git notation means that we need a fork of Neography that’s been tweaked to match the latest Neo4j API. When this change makes its way into the official gem (which may happen by the time you read this), you can remove this modifier.
To install Neography and its dependencies, run the following command:
$ jruby -S bundle install
Now, we’re ready to build a small Ruby API for storing actors and movies in a database. Put the following code in lib/database.rb:
require ‘bundler/setup’
require ‘neography’
require ‘cgi’
class Database
def neo
server = ENV[‘NEO4J_SERVER’] || ‘localhost’ # Line 1
@neo ||= Neography::Rest.new ‘http://', server
end
def find(type, name) # Line 2
hit = neo.get_index type, ‘name’, CGI.escape(name)
# get_index will return nil or an array of hashes
hit && hit.first
end
def find_or_create(type, name) # Line 3
# look for an actor or movie in the index first
node = find type, name
return node if node
node = neo.create_node ‘name’ => name
neo.add_to_index type, ‘name’, CGI.escape(name), node # Line 4
node
end
def acted_in(actor, movie)
neo.create_relationship ‘acting’, actor, movie # Line 5
end
endOn line 1, we default to connecting to the local machine, but allow a remote server name to be set in an environment variable. On lines 2 and 3, we create a couple of helper methods that take care of the details of object creation and retrieval. These details include marshaling strings into the format required by Neography, storing data into two indices named actor and movie, and so on. On line 5, we connect an actor to a movie (which implicitly creates a link from the movie back to the actor).
The alert reader will notice that we’re comparing movies by title only. That means that this script can’t tell the difference between, say, the 1939 and 1994 versions of Love Affair. The app will erroneously describe Warren Beatty and Irene Dunne as co-stars.
To distinguish different movies with the same title, we’d need a unique identifier for each film. As we’ll see in the next section, the data source we’re using doesn’t directly contain that information.
Populating the Sample Data
We’ll fill the database with a Creative Commons licensed flat file from the Freebase project. A warning for those who love to read through raw data files: there are over 60,000 movie titles in this file, and not all of them are work-safe.
Here are a couple of sample rows from performance.tsv, with arrows in place of tabs. The first non-empty column is a unique ID for the performance (not the movie, alas). The next two are the ones we’re interested in: the actor name and film title.
->/m/0jybnb->Kevin Bacon->Footloose->->Ren McCormick->
->/m/0k2ckt->Val Kilmer->Real Genius->->->To get the data into Neo4j, we just loop through the rows looking for actor and movie names, then use our Database object to make one actor/movie connection for each performance. Put the following code into Rakefile in your project root:
desc ‘Import movie data into Neo4j’
task :import do
require ‘lib/database’
File.open(‘performance.tsv’) do |f|
db = Database.new
f.gets # header
f.each_line do |line|
_, _, actor_name, movie_name = line.split “\t”
next if actor_name.empty? || movie_name.empty?
actor = db.find_or_create actor’, actor_name
movie = db.find_or_create ‘movie’, movie_name
b.acted_in actor, movie
end
end
endNow, run your import task like so:
$ jruby -S rake import
This process took a while on my old machine, so we watched a Christmas movie while I waited for it to complete. (Die Hard, if you’re curious. You can get from Bruce Willis to Kevin Bacon in only two steps, by the way.)
Traversing the Graph
Neo4j ships with a function to find the shortest path between two nodes — between two actors, in our case. It would certainly be more fun to dust off our Dijkstra and write our own traversal than just to learn the quirks of someone’s API. But fetching all those graph nodes across the network and creating a bunch of intermediary Ruby objects would almost certainly slow down the final program. With a heavy heart, add the following code inside your Database class:
def shortest_path(from, to)
from_node = find ‘actor’, from
to_node = find ‘actor’, to
return [] unless from_node && to_node
acting = {‘type’ => ‘acting’}
degrees = 6
depth = degrees * 2
nodes = neo.get_path(from_node, to_node, acting, depth)
[‘nodes’] || []
nodes.map do |node|
id = node.split(‘/’).last
neo.get_node(id)[‘data’][‘name’]
end
endThe API is now fully-featured enough for you to play a command-line version of the game:
$ jirb -rubygems -rlib/database
No Extensions Found: /Users/username/.neography
jruby-head :001 > db = Database.new
=> #<Database:0x4e26d560>
jruby-head :002 > db.shortest_path ‘Kevin Bacon’, ‘Bruce Willis’
=> [“Kevin Bacon”, “The Magic 7”, “Michael J. Fox”,
“Die Hard”, “Bruce Willis”]From here, it’s fairly straightforward to wrap a Web interface around the game.
Showing the Data
We’ll serve the data through the Sinatra web framework. Though templates are overkill for this project, we’ll use the Haml template language for the presentation. Doing so gives us an excuse to see how our packaging tool handles static files. Add the following lines to your Gemfile:
gem ‘sinatra’, ‘~> 1.1’
gem ‘haml’, ‘~> 3.0’…and then re-run Bundler to install the libraries:
$ jruby -S bundle installThe Web application is pretty simple. Create a new file called lib/degrees.rb with the following code:
require ‘bundler/setup’
require ‘enumerator’
require ‘sinatra’
require ‘haml’
require ‘lib/database’
db = Database.new
get ‘/’ do
from = params[:from] || ‘Kevin Bacon’
to = params[:to] || ‘’
path = if to.empty?
[]
else
db.shortest_path from, to
end
previous = path.shift
@results = path.each_slice(2).map do |slice| # Line 6
movie, actor = slice
result = %Q(#{previous} was in “#{movie}” with #{actor})
previous = actor
result
end
@from = CGI.escapeHTML from
@to = CGI.escapeHTML to
haml :index
endWe store the two actors’ names and the path between them in the @from, @to, and @results instance variables, so that the template can display them.
Our database wrapper hands us an array of alternating actor and movie names. The loop on line 6 steps through this array two items at a time and builds the final list of actor/movie/actor links.
The template just needs to display the results, along with a form for entering the next two actors’ names. Here’s what that looks like in Haml (this code goes in lib/views/index.haml):
%h1 Six Degrees
%ol
- @results.each do |result
%li #{result}
%form(action=”/”)
%input(name=”from” value=”#{@from}”)
%input(name=”to” value=”#{@to}”)
%input(type=”submit” value=”Search”)You can run the app directly like this:
$ jruby -rubygems lib/degrees.rb…but it will be handy later on to launch the app through the Rack interface. Rack is a wrapper that presents your app to the rest of the Ruby and Java ecosystem through a standard interface. You’ve already got Rack — it came along for the ride when you installed Sinatra. All you need to do is to create a configuration file in your project root called config.ru:
require ‘rubygems’
require ‘lib/degrees’
set :run, false
set :environment, :production
run Sinatra::Application…and launch again using the rackup command:
$ jruby -S rackupYou should be able to point your browser at http://localhost:9292 and see something like the figure.
Sharing Your App
Now that we have everything running on our development system, how do we package the software for deployment? Our system contains several different kinds of files:
- The Ruby files that make up the program
- Additional static files (templates, in this case)
- Supporting Ruby libraries
jarfiles containing Java code used by the Ruby libraries- The JRuby runtime
The goal is to package all these items up in a single file, hand it to our end user, and say, “Run this.” There are a few different ways to do this:
- Extract all our Ruby, Java, and other files into a staging area, then use the jar command to combine them into a single file.
- Partially automate the process with the Rawr packaging tool.
- For Web apps, completely automate the packaging process with Warbler.
All three of these approaches can be automated to some degree, but some require more custom code than others. Let’s take a closer look at the three techniques.
Do It Yourself
Doing your own packaging used to be the only game in town for JRuby deployment. You’d extract jruby-complete.jar to a staging area, copy in all your Ruby gems, and then recursively go through and extract any jars contained in those gems.
You’d then write and compile a little Java stub program that would launch your main Ruby program, something like this:
import org.jruby.embed.ScriptingContainer;
public class Launcher {
public static void main(String[] args) {
ScriptingContainer container = new ScriptingContainer();
container.runScriptlet(“require ‘lib/degrees’”);
}
}There’s a bit of grunt work here, but it’s not as bad as it sounds. JRuby has gotten pretty good at loading jars within jars, so you can often get away with just throwing a mixed directory tree of rb and jar files into the final package.
This approach gives you fine control over the package layout — something that comes in handy for things like Java service providers. But if what you’re doing falls into the more common use cases, it makes sense to use an automated tool to help you package your software.
Package with Rawr
Rawr is a library for packaging JRuby projects. It downloads the JRuby runtime, provides a prefab Java launcher, compiles code, and builds executables for the Mac and Windows platforms. To use it, you’d run the following commands from your project directory:
$ jruby -S gem install rawr
$ jruby -S rawr installThis would give you a Rakefile containing common packaging tasks, plus a build_configuration.rb file where you’d specify details like the program name and list of included files.
Rawr looks in your project directory for code to package, so you’d have to copy your Ruby dependencies there yourself. One easy way to do this is to rerun the bundle install command and pass it a destination directory. As we’ll see in a moment, there are alternatives to Rawr that don’t require this extra step.
Package With Warbler
For the specific case of Web applications, there’s a packager called Warbler that provides a more fully automated experience. In a single command, it does everything we need to prep our Six Degrees program for sharing. Warbler leans on two Ruby standards we’ve already seen: the Rack Web interface and the Bundler packaging tool.
Because Warbler isn’t a dependency that needs to get packaged along with the program, we just install it using plain old RubyGems:
$ jruby -S gem install warbler
Warbler’s output is a war file, which is an archive suitable for deployment on standard Java Web servers. These files don’t usually run on their own, but it’s possible to create one that does. If you pass the executable option on the command line, Warbler will embed a standalone Java Web server in the war file and connect it to our app:
$ jruby -S warble executable war
Warbler has several configuration options for including libraries and static files. By default, it brings in all the libraries and dependencies in your Gemfile, plus all the files in lib. Since we kept to those conventions, we don’t need to do any extra configuration.
You should now be able to copy the resulting file to your target system, set the NEO4J_SERVER environment variable to point to the database, and run the program:
$ export NEO4J_SERVER=ip.address.of.database
$ java -jar jruby-degrees.warBy leaning on JRuby and adhering to a few conventions for laying out Ruby projects, we’ve turned packaging into a one-liner. No struggling with MySQL adapters, tripping over conflicting Ruby versions, or wondering if the end user’s machine has gcc installed. Just a simple file copy.
That’s a good stopping point for this exercise. You’ll find the example source code at http://github.com/undees/pragpub. As a next step, you might choose a test framework and write a simple functional test for the app. In the spirit of the Six Degrees game, may I suggest Bacon?
About the Author
Erin Dees grew up in sunny Dallas, Texas, but eventually fled to the cooler climes of Portland, Oregon, and then on to Ontario. She got hooked on programming as a child trying out a Timex Sinclair 1000 at school, and has been creating software ever since. She’s hacked on everything from tiny embedded systems that need an oscilloscope to debug, to scaled-out web services that handle hundreds of thousands of requests every minute.
When Erin is “away from keyboard,” you can find her race-walking across Ontario wearing brilliantly hued and possibly aerodynamic garb. She also likes to play tabletop board games, so bring your meeples and meet her at the table!
Erin tweets as @undees.
