Getting Started w/ Ruby on GCP

The “Missing Tutorials” series

Writing a short series of ‘getting started’ posts for those of you, like me, who may get to the point of wanting to write code against a Google service, having a language chosen but then, having not written code for a week or two, I’m stalled by “How exactly do I get started?”

Ruby

OK. I’ve used Ruby twice. Both times were because (and I don’t know why) it appears to be the preferred language for monitoring solutions. I think I used it with Vagrant and with collectd… amiright?

Setup

To avoid supporting the myriad ways you will arrive at this point, I’m just going to tell you what I’m running (Linux), ruby 2.3.3p222 (2016–11–21) [x86_64-linux-gnu] and gems (2.5.2).

PROJECT_ID=[[YOUR-PROJECT-ID]]
LANG=ruby
mkdir -p ${HOME}/${PROJECT_ID}/${LANG}
cd ${HOME}/${PROJECT_ID}/${LANG}
https://rubygems.org/gems/googleauthX

Google Ruby Packages

API Client Libraries

Google

https://developers.google.com/api-client-library/ruby/

gems

gems “google-api-client”

gems “googleauth”

and… this is rather nice but you can also:

gem search ^google
*** REMOTE GEMS ***
...
google-api-client (0.13.4)
...
googleauth (0.5.3)
...

rubydoc:

http://www.rubydoc.info/github/google/google-api-ruby-client/Google/APIClient

Gemfile:

source 'https://rubygems.org' do
  gem 'googleauth', '0.5.3'
  gem 'google-api-client', '0.13.4'
end

Cloud Client Libraries

Google

https://cloud.google.com/ruby/
https://cloud.google.com/ruby/apis
http://googlecloudplatform.github.io/google-cloud-ruby/#/
https://github.com/GoogleCloudPlatform/google-cloud-ruby

Google Cloud Storage (GCS)

As is happening across the languages for the Cloud Client Libraries, Google is moving from “all services” package to “each service” packages. When you review the Cloud Client Libraries documentation for Ruby, please ensure you use the service-specific packages and select the correct documentation:

Google Cloud “google-cloud-storage” for example

In order to facilitate writing code that operates on GCS Buckets and Objects, the following script will create (hopefully) a uniquely named bucket for you and populate it with 10 differently named but identical content images. Please set $FILE to a value that paths to a small file that you wish to use as a template.

BUCKET=$(whoami)-$(date +%y%m%d%H%M)
FILE=[[/Path/To/Your/File]]
gsutil mb -p ${PROJECT_ID} gs://${BUCKET}
Creating gs://${BUCKET}/...
for i in $(seq -f "%02g" 1 10)
do
  gsutil cp $FILE gs://${BUCKET}/${i}
done
gsutil ls gs://${BUCKET}
gs://${BUCKET}/01
gs://${BUCKET}/02
gs://${BUCKET}/03
gs://${BUCKET}/04
gs://${BUCKET}/05
gs://${BUCKET}/06
gs://${BUCKET}/07
gs://${BUCKET}/08
gs://${BUCKET}/09
gs://${BUCKET}/10

Solution #1: Using API Client Libraries

Experience with other languages leads me to expect there to be a Ruby tool to help with package management. I was aware of gems as the packages of Ruby code (and thus RubyGems.orgs) but, unlike other languages, it appears there’s a separate tool (I found) Bundler to help create deployments.

Gemfile:

source 'https://rubygems.org' do
  gem 'googleapis', '0.5.3'
  gem 'google-api-client', '0.13.4'
end

and then:

bundle install --path vendor/bundle
Fetching gem metadata from https://rubygems.org/.........
Fetching version metadata from https://rubygems.org/.
...
Fetching googleauth 0.5.3
Installing googleauth 0.5.3
Fetching google-api-client 0.13.4
Installing google-api-client 0.13.4
Bundle complete! 2 Gemfile dependencies, 22 gems now installed.
Bundled gems are installed into ./vendor/bundle.

third.rb:

require 'googleauth'
scopes =  ['https://www.googleapis.com/auth/cloud-platform', 'https://www.googleapis.com/auth/devstorage.full_control']
authorization = Google::Auth.get_application_default(scopes)
require "google/apis/storage_v1"
gcs = Google::Apis::StorageV1::StorageService.new
gcs.authorization = authorization

When the ‘require’ wouldn’t work with “google/apis/storage”, I was left to check the API Client Library sources:

https://github.com/google/google-api-ruby-client/blob/master/generated/google/apis/storage_v1/service.rb

I was receiving an error:

bundle exec ruby first.rb
first.rb:9:in `require': cannot load such file -- google/apis/storage (LoadError)
 from first.rb:9:in `<main>'

Here it’s clear (line #28) that the include must be:

require 'google/apis/storage_v1'

After changing that, I’d guessed the Google::Apis::StorageV1 correctly and received no errors when:

bundle exec ruby third.rb

The sample “Drive” code on the GitHub page was almost sufficient help to get me to a point where the code lists the Buckets and Objects in a project:

PROJECT_ID = [[YOUR-PROJECT-ID]]
BUCKETNAME = [[YOUR-BUCKETNAME]]
require 'googleauth'
scopes =  [
    'https://www.googleapis.com/auth/cloud-platform',
    'https://www.googleapis.com/auth/devstorage.full_control'
]
authorization = Google::Auth.get_application_default(scopes)
require 'google/apis/storage_v1'
gcs = Google::Apis::StorageV1::StorageService.new
gcs.authorization = authorization
buckets = gcs.list_buckets(PROJECT_ID)
buckets.items.each do |bucket|
    puts bucket.name
end
objects = gcs.list_objects(BUCKETNAME)
objects.items.each do |object|
    puts object.name
end

This appears to work. I’m sufficiently unfamiliar with Ruby that I’m not comfortable tweaking it too much. If anyone has feedback on better practices, I’d appreciate them.

Proceed to “Testing” below to try this out.

Solution #2: Using Cloud Client Libraries

GCS includes Ruby samples for the Cloud Client Libraries. So, beyond the setup, I’m not going to reproduce that content here.

https://cloud.google.com/storage/docs/listing-buckets#storage-list-buckets-ruby

Using RubyGems:

https://rubygems.org/gems/google-cloud-storage

google-cloud-storage v1.4.0

fourth.rb:

PROJECT_ID = 'dazwilkin-170828-medium'
BUCKETNAME = 'dazwilkin-1708291403'
require "google/cloud/storage"
storage = Google::Cloud::Storage.new project: PROJECT_ID
storage.buckets.each do |bucket|
  puts bucket.name
end
bucket = storage.bucket BUCKETNAME
files = bucket.files
files.each do |file|
    puts file.name
end

Then

bundle install --path vendor/bundle
Fetching gem metadata from https://rubygems.org/..........
Fetching version metadata from https://rubygems.org/.
Resolving dependencies...
Fetching public_suffix 3.0.0
Installing public_suffix 3.0.0
Using bundler 1.15.4
...
Fetching googleauth 0.5.3
Installing googleauth 0.5.3
Fetching google-api-client 0.13.4
Installing google-api-client 0.13.4
Fetching google-cloud-core 1.0.0
Installing google-cloud-core 1.0.0
Fetching google-cloud-storage 1.4.0
Installing google-cloud-storage 1.4.0
Bundle complete! 1 Gemfile dependency, 26 gems now installed.
Bundled gems are installed into ./vendor/bundle.

To test this, please proceed to the next section “Testing”.

Testing

I recommend using Application Default Credentials to test your code. This enables config changes only for running the code as an authorized user locally, on App Engine, Compute Engine, and Container Engine. You must:

gcloud auth application-default login

You may also|additionally, authenticate with a service account. To do so, you must create a service account with sufficient permissions (the role ‘roles/storage.admin’ will be sufficient to make the buckets.list and objects.list calls. You must then also set the environment variable:

GOOGLE_APPLICATION_CREDENTIALS=/path/to/your/key.json

Then:

bundle exec ruby third.rb
bundle exec ruby fourth.rb
${BUCKET}
01
02
03
04
05
06
07
08
09
10

You may delete the bucket (and its objects) when you’re done. Be *very* careful that you specify the correct bucket when you perform this delete. It deletes all the objects and then the bucket:

gsutil rm -r gs://${BUCKET}

Done.