Sharing is Caring: Building Shared Services Effectively
Since childhood, we’ve been told that sharing is caring. Now that we’re all grown up and working, we realize that sharing is more than just caring — sharing is saving (money), sharing is aligning (business needs), sharing is reducing (duplicated efforts), and sharing is reusing (applications and services).
In the fast-paced world of software development, the innumerable benefits of such sharing tend to get lost in the constant pressure of feature requests, tight timelines, and product roadmaps. In this post, I’ll be going over our approach to Shared Services at Dow Jones, discussing the benefits we’ve seen and the difficulties we’ve faced. I hope this information will reinforce your existing Shared Services model, or assist in formalizing a new one.
“Shared Service” is a very overloaded term in the software industry — it can mean different things to different people. For the purposes of this blog post, a Shared Service is an application, service or set of services that provides common functionality to multiple consumers with varying technical and business requirements. This could be a backend API that supports multiple applications, a front end application that is used across multiple sites, or a team responsible for overseeing a specific area of application development like InfoSec or CloudOps.
At Dow Jones, we’re known for our world class journalism: The Wall Street Journal, MarketWatch and Barron’s are just some of the many products that Dow Jones owns and operates. Each of these brands has a website with its own identity, its own style and its own customer demographic. But if you peel away the styling on each site, you’ll notice that there are many commonalities. Under the covers, they all rely on many of the same Shared Services.
I am the engineering manager for one of these Shared Services, the Dow Jones Consumer Content Team. We’re responsible for the main service that serves content to all the major brands of Dow Jones, used by dozens of applications and the list continues to grow every day. As such, I’ve become very familiar with what it means to manage a service that is core to many differing business needs.
This juggling act of requirements can be difficult sometimes. In order to make our lives easier and reduce stress, we strive to follow these five principles during our application development lifecycle.
1. Reliability
Our Shared Service is crucial for dozens of applications in the Dow Jones universe. These applications are hosted all over the world and expect that our service is always available. To ensure the utmost availability, we operate in an active-active mode in two regions across the US. We replicate all data across these regions as well for disaster recovery scenarios. We also follow best practices during our development cycles, which ensures that the quality of our code is as high as possible. We strive to continually harden our Shared Service, building reliability into both our infrastructure and coding practices.
2. Usability
If no one is able to quickly and easily start using your service, it may as well not be there. Generally, it’s more upfront work for teams to switch from their own custom solution to your shared one. In the long run they will be better off for it, but corners inevitably get cut due to deadlines, and shared services tend to take a back seat. It can take some convincing to get teams started using your service. Our aim was to make it as easy as possible to transition onto our platform. We lowered the barrier of entry by documenting thoroughly and providing quick-start guides and examples. For API documentation, we rely on auto-generated Swagger docs. This documentation has allowed for teams to easily adopt our service, with very little work on our side.
3. Visibility
No matter how hard we try to prevent them, bugs will happen. In order to quickly address and resolve the issue, we need high visibility into our application. Following Dow Jones best practices we use centralized logging (Splunk) and monitoring (New Relic), but we also have built various custom tools that provide further insight. Some of these tools are built for our eyes only, while others are user friendly and intended for teams like emergency operations to become familiar with. This visibility reduces time to resolution of our bugs and gives customers of our service the means to inspect and debug if desired as well.
4. Maintainability
Testing, testing, testing. The previous generation consumer content service was around for ~20 years. If we intend to be around for even half that long, we must make our service maintainable. To do this we rely heavily on unit, integration and contract testing. Unit testing is the first line of defense against bugs — it validates the most basic units of work within our application. Integration testing is our second line of defense — it exercises our various APIs and code flows throughout our entire system. Contract testing is our third line of defense but might be the most important for our use case. Our API response format is a contract with all 20+ integrating applications. The format is extensible by design and grows as we add new features and functionality. When making changes to this format we can inadvertently break this contract, and customers’ applications in turn. Contract testing is our way to gut check that we are not going to break any customer integrations. The customer provides us with an API that accepts our content, and validates that any format changes we’ve made will not break their application. Running these tests during our CI process provides the confidence we need to continually deliver new features with minimal fear of introducing bugs.
5. Agility
While we maintain our own technical roadmap, at the end of the day we’re still beholden to our customers. With dozens of customers whose needs are in constant flux, it is important that we operate in an agile fashion. We communicate with these teams regularly in order to align their roadmaps and deadlines with ours. The autonomy we have as a Shared Service allows us the flexibility to focus on our technical roadmap while also building out business features. Our agility allows us to react quickly to important initiatives when they arise.
The Good, The Bad, & The Ugly
Even if your team has not yet adopted a Shared Services model it’s still useful to understand the ups and downs that Shared Services teams experience, as you likely rely on one or more of those teams yourself already. Here are some of the pros and cons that I’ve found over the years:
Pros
Better cross team collaboration
Centralizing common functionality of your business means that teams need to work together efficiently in order to build nearly anything. It is a great way to foster a culture of learning and trust within a company, as developers must rely on their peers and the services they’ve built.
Increased domain expertise
This one may seem obvious but it cannot be overstated. For applications & systems that have a large or complex problem domain its extremely beneficial to have a team that knows the domain inside and out. Some domains may take a year or more to learn and many more to master. Having a team dedicated to this allows for core functionality to be fully understood and reasoned with.
Increased ownership & accountability
There is a sense of pride and accomplishment in building software that is widely used by other teams. In my experience, that sense is directly proportional to the level of ownership & accountability the team ultimately has. This lends itself to many great things including higher quality code, more engaging discussions around architecture, brainstorming sessions that encourage innovation and push boundaries, and all-around improved team morale.
“Wow! Shared services seems like a great idea, there couldn’t possibly be any downsides!”
Well, there are some to be aware of:
Cons
Teams become very specialized
While domain expertise is generally a good thing, pigeonholing your team’s skillsets is not. When the team becomes an expert in something, it can be difficult to branch out and use other skills. Our team mitigates this by working on related projects that engage different sets of skills. This could be related to the visibility tools we mentioned earlier, or innovative side projects related to our core competency. We also encourage spending a few hours each week on personal career growth. Common areas include learning a new language or framework, studying for AWS or Google cloud certifications, or honing general soft skills.
You become the dependency
I wish I could say that we’re able to meet every demand on time, however sometimes we can’t. It can be stressful knowing you’re the dependency everyone is waiting on. We try to mitigate this by communicating with our customers effectively, early and often. During the discovery phase it’s useful to be curious and to ask questions. Often times questions that we ask during meetings lead to new requirements that may have otherwise gone unnoticed. General agility helps here as well, in cases where requirements come in late but deadlines cannot be moved.
Budgets can be challenging
The side effect of building a widely-used shared service is that the availability and usage becomes second nature for teams. This can lead to product teams taking the underlying services for granted, and project funding slipping through the cracks. Depending on the organization structure, getting funding for such a shared service can be challenging. We’ve been successful using a combination of evangelism and awareness, however we’ve also been fortunate enough to have the backing of the Technical Leadership Team at Dow Jones. Without their sponsorship, we wouldn’t have been able to gain the adoption & financial backing we needed to be successful.
Ephemeral objectives
With Shared Services, there is usually no big launch like that of a new website, no red carpet event for your new brand or product, sometimes there may not be much recognition at all. It’s imperative for morale that everyone on the team understands the direct impact that they have and the important work that they do. Be transparent about the vision for the product, discuss usage statistics and plans for the future, celebrate milestones amongst yourselves, be proud of what you have accomplished as a team.
The Shared Services model works well for us and others at Dow Jones — it’s truly in our DNA. It may not exactly fit your particular situation, but I hope that you’ve gained an understanding of how it works in ours. If you have any questions around any of this, or would like to learn more, please feel free to reach out at chris.nelson@dowjones.com.
