Surviving & thriving in the age of software accelerations
It has almost been a decade since Marc Andreessen made this prescient statement. Software is not only eating the world, but doing so at an accelerating pace. There is no industry that hasn’t been challenged by technology startups with disruptive approaches.
- Automakers are no longer just manufacturing companies: Tesla is disrupting the industry with their software approach to vehicle development and continuous over-the-air software delivery. Waymo’s autonomous cars have driven millions of miles and self-driving cars are a near-term reality. Uber is transforming the transportation industry into a service, potentially affecting the economics and incentives of almost 3–4% of the world GDP!
- Social networks and media platforms had a significant and decisive impact on the US election results.
- Banks and large financial institutions are being attacked by FinTech startups like WealthFront, Venmo, Affirm, Stripe, SoFi, etc. Bitcoin, Ethereum and the broader blockchain revolution can upend the core structure of banks and even sovereign currencies.
- Traditional retail businesses are under tremendous pressure due to Amazon and other e-commerce vendors. Retail is now a customer ownership, recommendations and optimization business rather than a brick and mortar one.
Enterprises need to adopt a new approach to software development and digital innovation. At Velotio, we are helping customers to modernize and transform their business with all of the approaches and best practices listed below.
In this fast-changing world, your business needs to be agile and fast-moving. You need to ship software faster, at a regular cadence, with high quality and be able to scale it globally.
Agile practices allow companies to rally diverse teams behind a defined process that helps to achieve inclusivity and drives productivity. Agile is about getting cross-functional teams to work in concert in planned short iterations with continuous learning and improvement.
Generally, teams that work in an Agile methodology will:
- Conduct regular standups and Scrum/Kanban planning meetings with the optimal use of tools like Jira, PivotalTracker, Rally, etc.
- Use pair programming and code review practices to ensure better code quality.
- Use continuous integration and delivery tools like Jenkins or CircleCI.
- Design processes for all aspects of product management, development, QA, DevOps and SRE.
- Use Slack, Hipchat or Teams for communication between team members and geographically diverse teams. Integrate all tools with Slack to ensure that it becomes the central hub for notifications and engagement.
Businesses need software that is purpose-built for the cloud model. What does that mean? Software team sizes are now in the hundreds of thousands. The number of applications and software stacks is growing rapidly in most companies. All companies use various cloud providers, SaaS vendors and best-of-breed hosted or on-premise software. Essentially, software complexity has increased exponentially which required a “cloud-native” approach to manage effectively. Cloud Native Computing Foundation defines cloud native as a software stack which is:
- Containerized: Each part (applications, processes, etc) is packaged in its own container. This facilitates reproducibility, transparency, and resource isolation.
- Dynamically orchestrated: Containers are actively scheduled and managed to optimise resource utilisation.
- Microservices oriented: Applications are segmented into microservices. This significantly increases the overall agility and maintainability of applications.
Cloud native is disrupting the traditional enterprise software vendors. Software is getting decomposed into specialised best of breed components — much like the micro-services architecture. See the Cloud Native landscape below from CNCF.
Process and toolsets need to change to enable faster development and deployment of software. Enterprises cannot compete without mature DevOps strategies. DevOps is essentially a set of practices, processes, culture, tooling and automation that focuses on delivering software continuously with high quality.
As you begin or expand your DevOps journey, a few things to keep in mind:
- Customise to your needs: There is no single DevOps process or toolchain that suits all needs. Take into account your organisation structure, team capabilities, current software process, opportunities for automation and goals while making decisions. For example, your infrastructure team may have automated deployments but the main source of your quality issues could be lack of code reviews in development team. So identify the critical pain points and sources of delay to address those first.
- Automation: Automate everything that can be. The lesser the dependency on human intervention, the higher are the chances for success.
- Culture: Align the incentives and goals with your development, ITOps, SecOps, SRE teams. Ensure that they collaborate effectively and ownership in the DevOps pipeline is well established.
- Small wins: Pick one application or team and implement your DevOps strategy within it. That way you can focus your energies and refine your experiments before applying them broadly. Show success as measured by quantifiable parameters and use that to transform the rest of your teams.
- Organisational dynamics & integrations: Adoption of new processes and tools will cause some disruptions and you may need to re-skill part of your team or hire externally. Ensure that compliance, SecOps & audit teams are aware of your DevOps journey and get their buy-in.
- DevOps is a continuous journey: DevOps will never be done. Train your team to learn continuously and refine your DevOps practice to keep achieving your goal: delivering software reliably and quickly.
As the amount of software in an enterprise explodes, so does the complexity. The only way to manage this complexity is by splitting your software and teams into smaller manageable units. Micro-services adoption is primarily to manage this complexity.
Development teams across the board are choosing microservices to develop new applications and break down legacy monoliths. Every micro-service can be deployed, upgraded, scaled, monitored and restarted independent of other services. Micro-services should ideally be managed by an automated system so that teams can easily update live applications without affecting end-users.
There are companies with 100s of micro-services in production which is only possible with mature DevOps, cloud-native and agile practice adoption.
Interestingly, serverless platforms like Google Functions and AWS Lambda are taking the concept of micro-services to the extreme by allowing each function to act like an independent piece of the application. You can read about my thoughts on serverless computing in this blog: Serverless Computing Predictions for 2017.
Digital transformation involves making strategic changes to business processes, competencies and models to leverage digital technologies. It is a very broad term and every consulting vendor twists it in various ways. Let me give a couple of examples to drive home the point that digital transformation is about using technology to improve your business model, gain efficiencies or built a moat around your business:
- GE has done an excellent job transforming themselves from a manufacturing company into an IoT/software company with Predix. GE builds airplane engines, medical equipment, oil & gas equipment and much more. Predix is an IoT platform that is being embedded into all of GE’s products. This enabled them to charge airlines on a per-mile basis by taking the ownership of maintenance and quality instead of charging on a one-time basis. This also gives them huge amounts of data that they can leverage to improve the business as a whole. So digital innovation has enabled a business model improvement leading to higher profits.
- Car companies are exploring models where they can provide autonomous car fleets to cities where they will charge on a per-mile basis. This will convert them into a “service” & “data” company from a pure manufacturing one.
- Insurance companies need to built digital capabilities to acquire and retain customers. They need to build data capabilities and provide on-going value with services rather than interact with the customer just once a year.
You would be better placed to compete in the market if you have automation and digital process in place so that you can build new products and pivot in an agile manner.
Big Data / Data Science
Businesses need to deal with increasing amounts of data due to IoT, social media, mobile and due to the adoption of software for various processes. And they need to use this data intelligently. Cloud platforms provide the services and solutions to accelerate your data science and machine learning strategies. AWS, Google Cloud & open-source libraries like Tensorflow, SciPy, Keras, etc. have a broad set of machine learning and big data services that can be leveraged. Companies need to build mature data processing pipelines to aggregate data from various sources and store it for quick and efficient access to various teams. Companies are leveraging these services and libraries to build solutions like:
- Predictive analytics
- Cognitive computing
- Robotic Process Automation
- Fraud detection
- Customer churn and segmentation analysis
- Recommendation engines
- Anomaly detection
Companies are creating data science teams to build long term capabilities and moats around their business by using their data smartly.
Re-platforming & App Modernization
Enterprises want to modernise their legacy, often monolithic apps as they migrate to the cloud. The move can be triggered due to hardware refresh cycles or license renewals or IT cost optimisation or adoption of software-focused business models.
Software is getting more intelligent and to enable this, businesses need to integrate disparate datasets, distributed teams and processes. This is best done on a scalable global cloud platform with agile processes. Big data and data science enables the creation of intelligent applications.
How smart applications can help your business?
- New intelligent system of engagement: intelligent apps surface insights to users enabling the user to be more effective and efficient. For example, CRMs and marketing software is getting intelligent and multi-platform enabling sales and marketing reps to become more productive.
- Personalisation: E-Commerce, social networks and now B2B software is getting personalised. In order to improve user experience and reduce churn, your applications should be personalised based on the user preferences and traits.
- Drive efficiencies: IoT is an excellent examples where efficiency of machines can be improved with data and cloud software. Real-time insights can help to optimise processes or can be used for preventive maintenance.
- Creation of new business models: Traditional and modern industries can use AI to build new business models. For example, what if insurance companies allows you to pay insurance premiums only for the miles driven?
Security threats to governments, enterprises and data have never been greater. As business adopt cloud native, DevOps & micro-services practices, their security practices need to evolve.
In our experience, these are few of the features of a mature cloud native security practice:
- Automated: Systems are updated automatically with the latest fixes. Another approach is immutable infrastructure with adoption of containers and serverless.
- Proactive: Automated security processes tend to be proactive. For example if a malware of vulnerability is found in one environment, automation can fix it in all environments. Mature DevOps & CI/CD processes ensure that fixes can be deployed in hours or days instead of weeks or months.
- Cloud Platforms: Business have realised that the mega-clouds are way more secure than their own data centers can be. Many of these cloud platforms have audit, security and compliance services which should be leveraged.
- Protecting credentials: Use AWS KMS, Hashicorp Vault or other solutions for protecting keys, passwords and authorizations.
- Bug bounties: Either setup bug bounties internally or through sites like HackerOne. You want the good guys to work for you and this is an easy way to do that.
As you can see, all of these approaches and best practices are intertwined and need to be implemented in concert to gain the desired results. It is best to start with one project, one group or one application and build on early wins. Remember, that is is a process and you are looking for gradual improvements to achieve your final objectives.