How to fetch & store your files in GCS (Google Cloud Storage) using Symfony

Dear reader,

Don’t close the tab , you’ve come to the right place.

In this article, we are going to tackle a very interesting subject which is storage in the Cloud.

Nowadays, various Cloud Storage Services exists (Amazon S3, Azure, Google, etc…) among them their is GCS (Google Cloud Storage) which we are going to focus on in this article.

Files in the Cloud are stored as objects not as blocks (traditional way). An object is composed of :

  • Data
  • Unique identifier : which represents the access path to data.
  • Metadata : contains information about the object (type, ….)

Accessing/storing files can be established via an API, which offer more easy and elegant way to manage files.

For more details, read this great article about object storage and its advantages and drawbacks.

As you have noticed in the title, we are going to interact with GCS using Symfony, so let’s go 😛

Prerequisites

  • You must have PHP and Composer installed on your computer. If it’s your first time to do that, check out theses articles that will help you setup PHP and Composer.
  • A Google Cloud Platform account (free trial or paid membership, it doesn’t matter).

The outcome of this article will be a REST API that exposes three endpoints:

  • Listing release notes;
  • Creating a release note;
  • Download a release note for given id;

First things first, let’s create a new Symfony application using composer:

composer create-project symfony/framework-standard-edition sf_gcs "3.4*"

After that, we are going to install a few dependencies into our project, we will tackle the configuration just after that.


Dependencies installation

The first one is Gaufrette Bundle which provides a filesystem abstraction layer.

To install it just run the following command in your terminal (in the project directory):

composer require knplabs/knp-gaufrette-bundle

The second dependency is VichUploaderBundle, this bundle makes file upload easy.

The installation is same as before :

composer require vich/uploader-bundle

In order to make some HTTP calls to our API, we are going use FOSRestBundle which provides various tools to rapidly develop RESTful API’s & applications with Symfony.

The installation is done with a simple command:

composer require friendsofsymfony/rest-bundle

In order to interact with Google Services (Cloud Storage in our case), we will need the Google APIs Client which enables you to work with Google APIs such as Google+, Drive, or YouTube on your server. The good this that is maintained by Google, so no worries ;)

The installation is straight forward :

composer require google/apiclient:"^2.0"

Configuration

Now after installing the necessary dependencies, we need to enable them in our app and make some configurations.

In your app/AppKernel.php add the lines (bold ones) to your bundles array:

$bundles = [
new Symfony\Bundle\FrameworkBundle\FrameworkBundle(),
new Symfony\Bundle\SecurityBundle\SecurityBundle(),
new Symfony\Bundle\TwigBundle\TwigBundle(),
new Symfony\Bundle\MonologBundle\MonologBundle(),
new Symfony\Bundle\SwiftmailerBundle\SwiftmailerBundle(),
new Doctrine\Bundle\DoctrineBundle\DoctrineBundle(),
new Sensio\Bundle\FrameworkExtraBundle\SensioFrameworkExtraBundle(),
new FOS\RestBundle\FOSRestBundle(),
new JMS\SerializerBundle\JMSSerializerBundle(),
new Knp\Bundle\GaufretteBundle\KnpGaufretteBundle(),
new Vich\UploaderBundle\VichUploaderBundle(),

new AppBundle\AppBundle(),
];

The JMSSerializerBundle comes with FOSRestBundle which helps in the serialization/deserialization process that why we enable it too.

In order to make these bundles work together, we must configure them in app/config.yml file

Here is a sample configuration :

The configuration of the Gaufrette bundle is divided into two parts: the adapters and the filesystems.

Configuring the Adapters

Gaufrette uses adapters to distinguish how the files are going to be persisted, in our case we are using google_cloud_storage adapter which takes as parameters :

  • service_id The service id of the \Google_Service_Storage to use. (required)
  • bucket_name The name of the GCS bucket to use. (required)
  • detect_content_type: if true will detect the content type for each file (default true)
  • options A list of additional options passed to the adapter.
  • directory A directory to operate in. (default '')
  • acl Whether the uploaded files should be private or public (default private)

Configuring FileSystems

The defined adapters are then used to create the filesystems.

Each defined filesystem must have an adapter with its value set to an adapter's key. The alias parameter allows us to define an alias for it (release_notes_filesystem in this case).


We also need to add reference to our buckets within the application, for that let’s add some parameters to app/config/parameters.yml.dist.

google_cloud_storage_scope: Access Rights (READ, READ_WRITE, FULL_CONTROL)
google_cloud_storage_private_key_location: relative or absolute path to credentials file (P12 or JSON (Recommended way))
release_notes_bucket_name: The name of the buckets we want to interact with.

P.S : Values are retrieved from environment variables.

Service configuration

As you have noticed in the adapters configuration, the service_id is ‘app.google_cloud_storage.service’ which is an alias to the service that communicates with Cloud Storage.

The ServiceFactory role in simple, here is how it looks:

Another important thing that your entity must have a file field annotated so that VichUploader can do the mapping in the incoming request. (line 72 on the below gist).

After this, the only thing left is to create a repository, service and a controller where you will put your logic to store and fetch data.

Let’s not forget that you will need a database with a release_notes table.

You can use DoctrineMigrationsBundle to manage database changes in your application.

That’s it, full source code is available in my github repository.

Don’t hesitate to comment for any further explanation and details.