What the hack?

In this article I’ll walk you through some of the technologies and fun we had during the 2nd edition of eMAG Hackathon on June 13, 2015. I’d like to share our journey with you, starting with Symfony, moving on to the burger van, writing some APIs with the FOSRestBundle, taking a hell of a ride with the Oculus Rift simulator and ending with the Android SDK deep into the night (true last man standing experience).

Our team (Black Team from the eMAG Marketplace application), had a very practical idea, based on a real need we had during the Scrum plannings: a mobile application to handle the meetings and estimations, while keeping record of each team’s and user’s progress. Yes, we’re doing Scrum all over eMAG, even in the business department . I won’t go into details like why we do Scrum or how we do it. It’s out of scope for now, but I promise to get back to you in the near future with the whole Scrum approach at eMAG.

Need for speed

Well, yes! We had a need and we needed the speed. Our specs for the MVP version were:

• Mobile app (Android for starters) with the following features
• Log in the user
• View his / her team
• Join a Scrum planning meeting
• Send his / her estimation for each sprint backlog item in story points (we use Scrum Poker with Fibonacci numbers in real life so why not use the same system for the mobile app?)
• And the mother of all Scrum apps: a VIOLATION button, capable to horn all team’s devices when such an awful event occurs
• Web platform
• API to communicate with the mobile devices
• Log in the user
• Define teams, each team having an owner
• Start a meeting for a team
• Start an estimation round for each sprint backlog item; if the estimations are widely spread, restart the round for a second and last time
• Control of the mobile app during the meeting: invite team members to a meeting, switch the mobile device screen to the meeting screen and let the users pick their estimation using their touch screens
• Show some stats after the meeting ends: estimations, rounds, total estimations in story points, user estimation accuracy

In this article I’ll focus on the backend application, meaning: Symfony setup, APIs and Security. We had a general idea on how things should be done, so we did a…

1-day Sprint

Yeah, I know the book says 1 week to 4 weeks for each Sprint, but we had only one day and we just adapted to what we had. We started with some flipchart drawings, mocking the mobile screens, leaving out the ones that were too costly. After one hour of moving back and forth between less work, some work and lots of work, we had some stories ready to go into production. We had only one Android developer in the team (bummer), while the rest of four were PHP developers, so we had no trouble in assigning the stories to each dev. We took good care of our Android guy, giving him constant back-rubs, making sure he’s properly hydrated and happy.

(yeah, don’t mind the guys in the background, they’re just playing with the flipchart)

During the 24 hours of Hackathon, we did some reviews and refinements, leaving out more features as we ran out of time due to serious distractions. You’ll see what I’m talking about in bit.

Care for a Symfony?

We built the backend using Symfony, because that’s our framework of choice at eMAG. With Symfony set, we decided to use FOSRestBundle to build the API needed by the mobile devices, together with JMS Serializer, FOSUserBundle and WSSE authentication.

First, add the required packages in Composer:

[snippet id=”9"]

and enable the bundles in Symfony’s Kernel:

[snippet id=”10"]

We created a new bundle for our application, called ScrumBundle, using the Symfony command:

[snippet id=”11"]

Now, we need to configure these bundles and let me tell you, it wasn’t a very straightforward task. We needed to put together bits of information taken from the Symfony Cookbook, the bundles’ GitHub documentation, tech blogs, StackOverflow and many other sources, including the mighty code. Some docs were outdated, some details were missing, but this is what we came up with:

FOSRestBundle https://github.com/FriendsOfSymfony/FOSRestBundle

[snippet id=”12"]

Basically, the configuration above:

• accepts requests over the API in JSON, XML and HTML (format_listener.rules), for URLs starting with /api
• responds HTML for all requests NOT matching the /api URL pattern (fallback_format: html)
• reads the parameters from request body (JSON, XML) or form-data key — value pairs
• returns custom ScrumException (‘ScrumBundle\Exception\ScrumException’: true) in prod environment, otherwise a generic 500 Internal Server Error is returned on API
• disables CSRF validation for the users having ROLE_API_M because we can’t read CSRF tokens over API (actually we can, but it brings unnecessary overhead)

FOSUserBundle https://github.com/FriendsOfSymfony/FOSUserBundle

Next, we configured FOSUserBundle:

[snippet id=”13"]

With the configuration above, we’re telling FOSUserBundle to:

• use the Doctrine driver
• use the main firewall (see bellow the security section)
• use the custom user class ScrumBundle\Entity\User, extending the FOSUserBundle User
• use the custom registration form type scrum_user_registration because we added some new properties to the User entity which are not defined in the FOSUserBundle default User, like name and device_id
• use the custom profile form type scrum_user_profile to fit the new properties’ requirements

scrum_user_registration and scrum_user_profile are the service IDs we defined for the custom form types.

JMS Serializer http://jmsyst.com/libs/serializer

JMS Serializer is a library capable of dealing with serializing / deserializing data of any complexity. What we tried to achieve is to automatically serialize any object and return the resulting string over the API. Since we don’t want to expose all object properties (especially for the User entity) over the API, we had to implement the ExclusionPolicy(“all”) and to expose only what was necessary. You can read more about the exclusion strategies over here: http://jmsyst.com/libs/serializer/master/cookbook/exclusion_strategies. Because we can’t control the User entity from FOSUserBundle (don’t modify any files in vendors directory, duh!), we implemented the following config for JMS:

[snippet id=”14"]

This tells JMSSerializerBundle to look in the app/serializer/FOSUB directory for configuration metadata:

[snippet id=”15"]

Now, all User properites from FOSUserBundle are not exposed, except email and enabled. The serializer accepts the group property, which allows us to expose the properties on certain groups. In our case, the API group is called API-m (stands for API mobile).

Nelmio API doc https://github.com/nelmio/NelmioApiDocBundle

Nelmio bundle automatically creates documentation for our API, it’s highly flexible and has a built-in sandbox to test your API. Enabling the bundle is very straightforward, it’s a one-liner:

[snippet id=”16"]

That’s it! We’ve pretty much passed the config stage and we can now move over to the fun part:

Burger Van FTW!

You can’t build a decent API without having a decent burger, especially when it’s in your backyard. It’s like coding with one hand. So here you go: our team waiting for the mighty burger:

Someone called Security?

Yep, you gotta have security these days, it’s not like back in the old days where a plain-text p@ssw0rd was enough. We wanted to allow our users to log in on the mobile devices as well as on the web platform. Same user, same credentials. Keeping the controllers thin, without authentication and authorization logic was solved with the WSSE protocol (Web Services Security Environment).

WSSE

WSSE is a family of open specification security, designed for SOAP services, but it can be applied to other applications as well. One of the advantages of WSSE is that it blocks replayed attacks, given that it uses a one-time nonce cryptographic random string, which is set to expire in 5 minutes (by default). If the nonce is used once, it can’t be used again. You can read more about WSSE here: http://www.xml.com/pub/a/2003/12/17/dive.html.

Symfony 2 can handle WSSE by default and we just followed this guide, which is self-explanatory: create a custom token class, a listener which will find the X-WSSE header in the request and trigger the authentication and an AuthenticationProvider. Now, the firewall section in our security.yml looks like this:

[snippet id=”17"]

As you can see, we secured with WSSE only the /api-m URL pattern. You can now access the User entity in the controllers with $this->getUser().

Let’s recap a bit

Our backend application has these main features now:

• framework able to support a REST API and automatic API documentation
• user authentication and authorization in place, stateless over API or stateful browser login

Let’s see now how we can create some API routes and controllers:

API routing and controllers

Thanks to the FOSRestBundle, you don’t need to manually create a route for each controller action. Let’s see how we managed the user login action from a mobile device.

First, add the general routing for our ScrumBundle, in the app/config/routing.yml file:

[snippet id=”18"]

We separated the web routes from the API-m routes:

[snippet id=”19"]

• scrum_api is for API routes
• scrum_dashboard is for web based dashboard, where the users can login and, for example, start a planning meeting
• scrum_public is for general access to the web site Icon

Did you notice the new routing parameter type? This is FOSRestBundle specific and it helps the application to automatically generate the proper routes.

Since we’re talking about the API routes, here’s our routing file:

[snippet id=”20"]

Each route entry corresponds to a different controller. You gotta love Separation of Concerns!

Enough with the YML files, let’s see some actual PHP code! Well, you’ll be disappointed, as I said earlier, keep your controllers thin. Thin as this:

[snippet id=”21"]

Lots of annotations, I know. Ok, this user controller has two actions:

userCheckAction()

Checks if a user is successfully logged in and saves his device ID (each mobile device has its own device ID, we’ll use it to publish notifications and control the device’s screen). Step by step explanation:

• line 21: defines the route for this action and the method allowed (POST). Since the scrum_api route has the prefix /api-m and the scrum_api_m_user route has the prefix /user, the complete route for this action is /api-m/user/check
• lines 23–31: the configuration for the Nelmio ApiDoc. Possible returned HTTP status codes and the view under which the documentation is available. Nelmio supports different views groups for the documentation, so you can have a public API doc, a private one and so on. The default url for Nelmio ApiDoc is /api/doc, but using the API-m view, our documentation is available under /api/doc/API-m
• line 33: requires that the parameter device_id be present in the request, either in the body or in the form-data
• line 40: updates the device_id for that user (if it changed), using the scrum.user service
• line 42: returns a HTTP 200 response

Thanks to all the configuration above, we just have a two-liner controller method, capable of authenticating the user and updating a User property!

userTeamsAction()

Gets the teams where the user is a member:

line 49: the route /teams and the method allowed (POST). The complete URL is /api-m/user/teams

lines 51–54: this is new! It configures the View object returned in response with the following two JMS Serializer parameters: serializerGroups (serialization group is API-m as declared above) and serializerEnableMaxDepthChecks to take into account the annotations on the entities to go “n” levels deep when serializing objects. For example: our user is linked to a Team entity, which is linked to an Organization entity. On this particular route, we don’t want to expose the Organization, but the Team only. We can control this by using the @MaxDepth annotation on the entities, but the View has to be configured to take this setting into account.

lines 56–61: the ApiDoc configuration, same as above

lines 67–70: the user has a ManyToMany association with the Team entity. Here we get the teams linked to the user and return them as an ArrayCollection. FOSRestBundle and JMS Serializer takes the weight off our shoulders and the response is something like this:

[snippet id=”22"]

OMG, I’m flying!

Enough code! Let’s put on the Oculus 2 goggles and have a ride on the virtual roller-coaster. Have you ever tried these gadgets? Oh man, it’s the real deal! It’s just amazing how a pair of glasses and 4 motors mounted under your seat can induce such a vivid and real experience. Check the photos below:

As I said, you can’t build a proper application without this kind of fun.

Season finale

Hey, wait, no! We can’t have a finale just yet, right? The best part of the eMAG Hackathon experience is that we’re still developing this application and trying to bring it to a point where we can sell subscriptions for the service. Yeah, eMAG will have FREE lifetime subscription! So no, it’s not a finale, we’ll bring you soon the next episode, where we integrate Web Sockets and ZeroMQ into our app for real-time, blazing-fast data transfer.

Have I mentioned we won the Popularity Prize with our app? Well, here it is: