How to use the cloud to build applications faster

Photo by Egor Lyfar on Unsplash

It seems like everybody wants their apps to run faster and faster. Hundreds of posts on application performance are published every year. But articles about creating applications quickly are pretty rare to find. That’s a pity because when push comes to shove this is what counts for business. Development productivity is the knowledge which allows you to tell a CTO/tech lead apart from a regular developer. Let’s talk about it then. In this post, we describe 3 modern software development approaches you can use to deliver features like a well-oiled assembly line.

Speed of delivery is key to winning in the market. The time of “build -> launch -> measure & adapt” cycle is a key metric for the majority of software produced today. That’s why today’s engineering teams come up with more and more sophisticated tools and practices to reduce time to ship. These include continuous deployment processes, build automation tools, coding guidelines, and static code analysis tools. All of this to create a controllable environment for smooth software delivery lifecycle with dozens of coders.

Nonetheless, while these artifacts are useful, it seems like some teams focus too much on optimizing them. Refining build pipelines or developer environments doesn’t always lead to cutting time to market — sometimes it can even have a negative impact on delivery speed.

Our experience shows that turning your focus to 2 aspects of software development lifecycle brings better results to shipping speed. These are:

• achieving code simplicity
• building a rapid feedback setup

While they can be approached from many angles, I’d like to focus on one in particular — API-first SaaS platforms.

API-first building blocks instead of your clunky POC

Let’s start with a short intro of these solutions. API-based or API-first platforms provide a particular piece of functionality wrapped with an API that is easy to integrate with. You can use these endpoints to implement a well-adapted solution without the need of starting from scratch.

There are a few hard-to-miss features of API-based solutions:

• the main business value comes from the API and webhooks, and the user interface (if there is any) plays a secondary role, and is used mostly to administer the account
• they provide a thorough developer documentation
• they have dedicated teams to take care of security and availability

Profit? The business gets the speed and flexibility at the same time. And savings of course. As a user of such a service, you usually pay for the actual usage, the number of requests for example.

So what’s that got to do with 2 goals of fast delivery we’ve mentioned?

The first one is straightforward. You just need fewer lines of code to implement the solution the task is about. Sure, there’s some learning curve at the beginning, but overall the time you spend on familiarizing yourself with a new API — usually with decent documentation and examples — is shorter than what it takes to create a feature with a regular “trial and error” approach.

Unless you’re a seasoned developer in a particular area, the solution provided by the platform will be more robust. And even if there are some errors or confusion, the support team will help you handle the case.

The second goal naturally stems from the first one. If you implement the feature quicker, then you can test it with users sooner (thanks Sherlock). But to gather feedback properly, you also need a way to collect and visualize data for the metric the business is after. API-based platforms might be helpful here again because, usually, they provide detailed usage statistics (also available through API) and sometimes even whole reporting modules.

So all in all, putting a feature together with API-based platforms is a better approach than your quick-and-dirty proof of concept.

OK, let’s see an example.

Case: imagine an early stage but rapidly growing online marketplace which connects suppliers and customers. The process of matching ones with the others is done manually by operations agents. With increasing traffic, the company needs more and more agents to process the orders. They decide to ask the tech team to automate this activity. Matching takes into account several parameters: time slot, location, customer preferences, supplier category, supplier rank, and of course the amount of money they make on every order.

Now the goal of the project project is to reduce the time of agents while keeping the profitability of the matching function.

The first idea which came from the management was to commission a machine learning algorithm and connect it to the CRM platform. But the CTO was skeptical. He knew it would be an expensive ride. Ultimately, he was afraid it would take a while to develop a solution which yields results similar or better than agents. He forced an alternative approach which focused on unburdening the operations team in short term.

He analyzed the data and found out some patterns. Then, he decided to replace the human factor with SMS. The engine analysed incoming orders and sent the “Do you accept this XYZ job? Yes/No” message to several suppliers. The process sent SMSes according to a pattern including aforementioned parameters. Whoever accepted first, won the order.

This was quick to implement and worked out very well! Of course, the solution didn’t cover 100% of cases but it significantly reduced the ops team effort by 60%, gaining more resources for other operational tasks. It was a matter of days, not weeks or months.

Now the key part is that the quick delivery wouldn’t have taken place without the help of API-first platforms.

First, instead of building the SMS gateway, they team went for Twilio. Second, instead of implementing a cron-like functionality, they used EasyCron. Third, to calculate geo-distances, they tapped into Google Maps API. The whole solution was basically just a loop and a bunch of ifs, and several HTTP requests.

In summary, they quickly managed to get an important feature live and achieved business goals in no time. On top of that, the code responsible for this functionality was super simple and thus open for modifications (which they did later — automating 90% of order allocation).

But API-first platforms have their caveats too:

• The most important thing you need to verify as part of your SaaS research is limits. API-first vendors impose thresholds to simply make money as well as to protect their service from unexpectedly high traffic. Make sure then the limits are aligned with your use case budget.
• The second is a fallback plan — depending on the impact on the business, your system should be accordingly prepared for the outage either of a vendor’s platform or yours. Features like system down/maintenance alerts and webhooks which repeat failed requests help in such cases.
• Make sure the provider you want entrust offers a support policy which goes along with your business case.
• No bigger company will allow for integration with a 3rd party system if it doesn’t provide enterprise-grade features. Prepare a list of what’s required in your org.

Recommended reading :

• API-first SaaS collection
• How to use API-first platforms to build your websites faster
• An introduction to ecommerce APIs for non-developers

Business apps is copy-and-paste-and-change-one-thing

A seasoned business application developer would say that business software products are more or less copy’n’paste. This has certainly been a leitmotif for Salesforce founders. Having this in mind, they’ve built a CRM platform which powers hundreds of thousands business apps around the world. So why has Salesforce managed to attract so many customers and why are we talking about this at all?

Salesforce started as a SaaS CRM solution. They were one of the first players on the market and their sales pitch back then was “NO SOFTWARE”. They aimed at attracting companies trying to automate their sales processes which were fed up with sluggish on-premise software. Their SaaS offer turned up to be a massive success.

But fast-forward to the second decade of the 21st century, we can see that Salesforce is not “NO SOFTWARE” anymore. Actually today, they claim that 50% of their profits comes from the usage of Salesforce API. What happened to “NO SOFTWARE” then?

Well, SF realized that although 80% of CRM application works the same, companies still needs software services to integrate other systems and customize data flow.

With this observation, they started to give companies the ability to configure the setup. It started off with custom fields, simple triggers, lookups, reports etc. These capabilities turned out to be in high demand, so the SF team decided to push forward in this direction.

They extended configuration options by adding:

• programmable software environment with their proprietary language called APEX
• created Lightning UI framework to create new views right in Salesforce
• exposed API and webhooks-like functionality to connect external systems

In other words — they bet on software.

With this setup, you can automate helluva lot of sales processes. But it’s not only about sales. Through acquisitions and development, SF can now cover various departments making the whole business tick. The best thing about that? Developer productivity.

With a handful of tools, a single experienced SF developer can provide more value than a dozen of coders building a business app from zero.

And we’ve been there. Remember the fast-growing online marketplace company from the previous example? They bet on Salesforce almost from day one. After 3 months of running on Google Spreadsheets, they transferred their small-back-then departments to SF and never regretted this decision. In hindsight, SF allowed them to grow for the next 3 years. How did it work?

Salesforce was treated as a single source of truth, there was no database other than that. Part of functionality was developed with APEX and Lightning right in Salesforce and the other part using external system in Node.js connected with API. By connecting other API-based platforms through the API, they achieved very high developer productivity.

The other Salesforce features that led to high speed in software delivery:

• When the company change the business model and, in turn, the database model, they could react super quickly by exporting data through Data Loader, rearranging them in Excel(!), and putting them safely (with the right format) back to Salesforce.
• Whenever they were hitting limits, they did cost-benefit analysis on how to proceed with a feature — either to extend quota or to figure out a work around. For example, when data analytics queries started to utilize a large part of quota, they replicated the internal database to a Postgresql instance (through Heroku Connect) and changed the data source accordingly.
• Whenever the business asked for a new feature, there was a high chance that AppExchange marketplace had some solution

But Salesforce is not all bells and whistles. Similarly to MS Sharepoint, the SF developer experience leaves a lot to be desired. Cumbersome IDE, poor support for multi-developer environments, poor source control technologies integration, long deployments — this all makes a software developer cringe. Fortunately, they’ve noticed their bad and have been fixing it with Salesforce DX.

Recommended reading:

• Building an online marketplace from scratch [series]
• The Salesforce Platform: The Return of the Citizen Programmer
• What is Quick Base? A low-code database platform for citizen developers

Quick and not-so-dirty internal apps

Excel supports (too) many business processes across departments. Most of them are fed up with data with a regularly scheduled database export, some of them are even connected to a DB directly. Although Excel is a real powerhouse for various analytics, both off-hand charts and robust year-to-year summaries, it’s still one of the most criticized approach to business intelligence. Reason? It’s error-prone, it’s hard to track changes, and finally, sharing among users and roles is hindered.

You can of course use 2 previous approaches to replace them, but often either it’s not worth the effort or the new solution won’t be flexible enough. So, what can a CTO do in this case?

One way forward is to tap into SaaS like Metabase, Redash or Retool. What they offer is the controllable access to your database with out-of-the-box reporting and charting capabilities. Their user interface is intuitive and less error-prone than a regular spreadsheet. And when Excel is simply used as a list online, you can replace it with tools that are build just for this reason, such as Airtable or more advanced Smartsheet.

Whatever tool you use, in most cases it’ll be way faster to mimic or even overtake Excel than in case you’ve decided to implement it from scratch. Of course, you need to consider initial user onboarding effort, but it’s worthwhile because this approach will increase the productivity of internal apps users in the long run.

The cloud is here, use it

Your role as a CTO/tech lead/even regular developer is to solve business problems with the best yet the most cost-effective tools you find out there. Today, SaaS tools have become mature enough and developer-friendly to put them into your toolbelt. Depending on your decision-making level in your company and the problem specifics, you can outsource the solution to appropriate flavour of SaaS. You’ll get simpler and maintainable code which will lead to faster feature experiments, and ultimately, to better productivity in your team — hopefully better rewarded by the management :)