Getting technical — starting to develop an architecture and roadmap for NCVO’s future technology

There’s a theme that runs through most events you’ll attend or pieces you’ll read about digital leadership or digital transformation. It tends to go something like this:

Digital is not about technology. It’s about people.

I wholeheartedly agree. Technology on its own achieves nothing. What matters is how people use it. And that’s why successful digital leaders and change programmes focus on culture, behaviours, skills and confidence. This was the focus of NCVO’s digital strategy between 2014 and 2016, when we did things like launch our internal digital skills programme. And I always take every opportunity to use the Co-op’s definition of digital:

Applying the culture, practices, processes & technologies of the internet-era to respond to people’s raised expectations.

Except, and here’s the thing, sometimes a digital strategy is hugely technical. The definition above talks about ‘technologies’ after all — not on it’s own, but it’s in there.

In the last six posts in this series about NCVO’s technology strategy I’ve shared our experience of mapping out our technology, doing qualitative and quantitative user research, and testing prototypes. All to try and understand what to do about the big, complex, scary problems that I outlined in my first post.

The next phase was to map out an architecture and a roadmap. And when it came to meeting the needs of our users and of NCVO, these turned out to be very technical.

Enter Jaggeree

I’ve taken every opportunity to mention that I was supported through this project by the fab team at CAST. We’d agreed from the start that they would bring in a ‘technical architect’, and around a year ago we were introduced to the fantastic Chris Thorpe (aka ‘jaggeree on Twitter’).

We had some quite specific questions for Chris about technology choices. These included:

How should we integrate our websites and digital products with our CRM?

We knew that we had a problem with our integrations. We were (and still are) using a web database, that pre-dated our CRM. It provides a useful air gap between websites, which may be at risk of intrusion attempts, and the CRM. It replicates data with the CRM but can become out of sync, particularly because it has a different data structure and different data validation rules. This causes lots of issues for users (particularly members who want to access restricted content), and we spend a lot of time resolving these problems for our users.

Chris’s suggestion was to build our own API to replace the web database. The API would be flexible and secure. For example we could easily give and revoke credentials to our digital partners, and control in a granular way which fields an application could read or write to. I work with a fantastic CRM Manager (James) and Chris proposed that an API could be maintained in-house by someone with his skills.

Chris and James have been working on our API on and off since last summer (and if you’re interested in the technical stuff, Chris is writing the code in Javascript and James is building serverless functions using Azure functions and Microsoft Flow — selected because our CRM is Microsoft Dynamics).

What is the best way to approach online payments?

We knew that our users wanted to pay for products and services online, and that we were spending many hours moving data around manually because we had implemented online payments for some products with no integrations to our business systems.

Chris’s first response was to tell us that our current finance system (Great Plains) was going to be a blocker if our vision was to become more efficient. We have been working on this since the summer and are hoping to migrate to a cloud-native modern finance system by October.

An early diagram showing how online purchasing could work

What are the best (and the good enough, and the risky) technologies for our digital products?

We knew that we were hampered by having so many different Content Management Systems (at the time: Joomla, Plone, Wordpress, Tumblr, a proprietary CMS, and a .NET web application). We also sensed, from talking to other charities and agencies, that they might not all be particularly popular with developers anymore, which felt like a big risk.

Chris proposed that we move towards selecting ‘best in class technology’ and adopting a modular architecture consisting of one publishing technology (because NCVO primarily publishes content) and micro-services built with whichever technology was best for the job. In order to make sense of this for users, Chris also proposed building ‘a layer of shared services’. This needs unpicking and explaining further, so I’m going to come back to this in my next post…

Towards an architecture, a vision and a roadmap

After several stretching but fascinating months working with Chris, we had a clear idea of what we needed to do.

The following is pulled directly from the business case that I wrote for our trustees:

Our technology is now getting in the way of us scaling our impact in a sustainable way
Our digital estate has grown organically through a combination of acquisition of products through mergers and short-term delivery of websites. It is complex and increasingly expensive to maintain, let alone improve it to meet the increasing expectations of our users. Our users don’t know about the breadth of what NCVO offers and aren’t finding the content or services that they need.
The technology behind the scenes that joins up data between different systems is patchy, unreliable, and not secure enough. Staff spend many hundreds of hours each month on manual processes that we struggle to automate with our existing set up. This is time that could be spent on things that matter — representing, supporting and connecting our members.
We know what we need to do
We need to make some fundamental changes to our technology. We need to create a leaner system that allows us to scale our impact in a sustainable way.
We need to change the architecture of our technology — the tasks that individual digital services perform, how they work together in an efficient and secure way, and the specific coding languages and framework that they are built on. As a result of this process we will:
· Rationalise the number of websites/digital products experienced by users, retiring some websites by incorporating their content or services into core products
· Break up websites into a number of less complex pieces, building digital services which provide a specific functionality or information delivery, built on top of a layer of shared services (authentication, user information, subscription management, search, payment).

When you need a CTO

As I set out in my first post, NCVO has an annual turnover of £7.5m and 12 of our 100 hundred staff work on developing digital products, content strategy and production, CRM and data. That makes us a very large organisation compared to the vast majority of the sector and our members (97% of charities in England have an annual income of under £1m).

But we’re still far too small to have a CTO (Chief technology officer) type of person. And yet our technology strategy would not have been possible without that kind of expertise.

I’m keen to work with CAST and others to work out how charities of all sizes can access this kind of independent, invaluable technical expertise when they need it.