How to Manage Office Logistics Effectively — Smart Display Case Study

Booking a room can sometimes be a challenge, especially when you are in a hurry and your meetings pop up unexpectedly. We all have situations in which we try to make a room reservation on the phone while moving from one place to another, which is not the most convenient option. That’s why at one of our hackathons we came up with an app that will allow us to make room reservations instantly, without having to open our computers and with minimal interaction with our phones.

Smart Display — Seamless room booking app with voice assistance and face recognition

Thinking about the ideal application for booking a room, we have set ourselves three goals:

• The app should be placed in the center of the office, so that everyone can use it comfortably
• Booking a room must be super fast, easy and intuitive
• There should be an option to check which room is available right now so that we won’t need to walk through the whole office and look for it.

Based on these criteria we deducted that it implementing voice commands and voice interpretation for both advanced and basic functions was a bad option. We didn’t want to force users to spend additional time interacting with the application, so we stuck with a standard clickable interface for quick actions.

Another issue that arose was the question of security: users of the app will be able to access potentially confidential data e.g.:

• Personal agenda of some users is confidential, so it should be accessible only by the owner,
• We do not want users to do room reservations from somebody else’s account. To book a room under a particular user’s name we have to know the user’s identity.

Before we let users perform such actions we have to make sure they are eligible for it, so users should log in before accessing restricted data. Two questions naturally came to our minds:

How to log in quickly?

At NG we use a dedicated tool to generate secure passwords. So for our new app we wanted to make it simpler without discouraging the users. Instead of the complicated password we wanted to use something different, maybe our access cards, fingerprints or… our face? We decided on the last one and it has many advantages:

• Logging in by face is simple, fast and intuitive, it doesn’t require users to create complicated logins and remember them;
• It doesn’t require any additional hardware — your smartphone, tablet or computer camera will do fine;
• If done correctly, it is more secure than a password.

How to log in securely?

Users should not log in with their email credentials on a tablet that is commonly shared in the office. This is why registration (setting a PIN and an optional face photo) is done on the user’s personal device such as a smartphone or laptop. Later, they can log in using face recognition or email and a PIN used exclusively in Smart Display.

Implementation

Although the app was designed for Android only, we chose to build it with React Native for multiple reasons:

• RN is especially good for rapid development and prototyping, which we confirmed in React Native vs Swift — Performance and Development Comparison,
• It was battle tested in production environments with multiple successes,
• It makes it possible to share portions of JS code on web frontend,
• It doesn’t preclude building an iOS version in the future (thanks to the cross-platform character of RN),
• Our developers are familiar with RN.

For the backend we chose Ruby on Rails. The main reasons were:

• RoR has good support of third-party gems (libraries) for services we target, such as Google Calendar API or Slack API,
• It’s also often associated with quick prototyping and rapid development, which fits the Smart Display,
• It’s a popular technology well known by our BE developers,
• We used RoR with success in many other projects.

For the Machine Learning part we used a VGG-based deep learning model implemented in Keras.

During registration, the user can provide a face photo to the machine learning engine for facial features to be extracted and stored in a database.

When the user wants to log in with face recognition, the app takes a photo of his face and sends it to the backend, which forwards it to a machine learning app running on the Google Cloud Platform. On GCP, facial features are extracted and compared to the ones stored in the database that were generated during registration. When GCP finds a matching user who is eligible to log in, you can access your personal calendar and book a room.

Final results, user test preparation

The final app looked like this:

https://www.youtube.com/watch?v=rZQkg-VuE3g

After finishing the MVP we decided to test our app in one of our offices in Kraków. To do so we:

• Ran an internal Bug bash and caught some bugs which we quickly fixed,
• Prepared registration instructions and a short video on how to use the app,
• Launched the app in the NG Cracow office,
• Shared a feedback form through which users could share their opinion and give us their ideas of improvements.

It is important for us to gather real users’ opinions about the app. In the feedback form we asked: what is done well in this app, what could be improved, and what other functionalities would you find useful in your daily work. Currently the app is being tested and we are collecting feedback which we will use in planning our next development steps.

Summary and the next steps

So far we’ve prepared our app for tests and we are working on improvements while gathering feedback. So what are our next steps?

There are many ideas and features that we plan to implement within this app. One of them is to prepare a personalised dashboard for each worker and display flight info for our Account Managers and project workflow for our developers. We also want to connect this app with our internal bot and make it feel more alive by adding voice recognition.

This solution, like many other ML applications, allows you to perform tasks faster, easier and safer. Technologies such as face recognition can be used in many industries like healthcare, banking, and manufacturing, to name a few. If you have questions about this project or want to see how Machine Learning can improve your business, just write to us at hello@netguru.com.

Originally published at https://www.netguru.com.