Building a technical foundation for the future of local news

Our “just enough tech” approach to supporting new local news organizations is built for affordability, efficiency, and third-party integrations.

by Tyler Fisher, Deputy Director, Technology at News Catalyst

A history of code additions and deletions over the last two years of work

When we started the Tiny News Collective in 2020, we were responding to a crisis in journalism: as local news institutions failed, the barriers to replacing them were too high. We spoke with founders of small newsrooms around the country about starting their organizations, and what we heard consistently was that technology is a hurdle to building their businesses. We built the Tiny News Platform to address the challenge of finding, configuring, and maintaining the technology necessary to support a local news organization.

The Tiny News Platform is a complete publishing solution built specifically for small news publishers.The platform allows members of the Tiny News Collective to start their new businesses without the high cost and effort of spinning up a technology stack, freeing founders to focus instead on audience and revenue development. We want to share more about the platform’s architecture and why we made the decisions we made.

To date, the Tiny News Platform has been built by the staff of News Catalyst, a Temple University project funded by the Knight-Lenfest Local News Transformation Fund. However, we are beginning to hire the permanent technology team for the Tiny News Collective that can take this foundation and build upon it. Right now, we’re hiring a Director of Technology to lead the permanent technology team. If you find yourself interested in this platform and this work, please apply.

Aligning technology with mission and values

In conversations with news startups and organizations that support local news, we identified a serious gap in support for one of the most common types of local news startups: the solo or small group organization. These founders that start with little funding and not much capacity beyond their own labor are deeply committed to their missions, connected to their communities, and also incredibly short on time. They’re responsible for everything from business operations to building the tech stack to producing the content, and they simply do not have time to expend on things that are not essential in the early days of getting started. With little funding, they also don’t have the ability to offload work.

We wanted our services and especially our technology to remove as much of the burden as possible from these founders so they can focus on product development, connecting with their audiences and experimenting with revenue strategies.

With that in mind, we prioritized putting together a tech stack that includes everything a founder could need, making it as fast and as accessible as possible while ensuring that the founders had complete ownership of their work and would be able to migrate to a new system after they’d outgrown ours.

Determining the platform’s initial feature set

We began our process of building the Tiny News Platform by doing user research. We spoke with founders of small news organizations about their technology, and we learned that they frequently spend months and thousands of dollars exploring different technologies. Consistently, we heard that the technology necessary for starting a news organization was too expensive, too complicated, and too fragile. With this input, we resolved to build a platform more affordable than anything else on the market, pre-configured for best practices, and deeply integrated with powerful and trusted third-party platforms.

We also took note of the technology small news organizations felt they needed to succeed. In these conversations, we heard key needs like publishing on multiple platforms, raising money from both readers and local businesses, keeping track of members, and measuring success. To meet these needs, we identified core pieces of technology every new news organization should have available to them: web publishing, newsletters, analytics, membership/donation, CRM, advertising, and comments.

Since building all of those pieces of technology from scratch would be impossible, we opted to think about integrations as a key aspect of our platform and to find third parties to partner with. We now work with brilliant partners in Letterhead and MonkeyPod, who allow us to offer newsletters, direct advertising, donation-based membership, CRM, and accounting to our Tiny News Collective members. Working with these partners let us focus on the area we felt we could best innovate: web publishing.

Why not WordPress?

WordPress is the most popular CMS on the internet, and it is likely the most common CMS among small news publishers as well. It’s so ubiquitous in publishing that many people assume we’re using WordPress somewhere in the backend of the Tiny News Platform. But when we spoke with small news organizations, we consistently heard about the complicated nature of working with WordPress, so we chose to go in a different direction.

Maintaining a system like WordPress means keeping up-to-date with plugins, version upgrades, and theme management. For non-technical users, this can be daunting and even lead to catastrophic failures when something unexpected breaks. Some WordPress-based systems, like Newspack, will handle the technical minutia of hosting and configuring a WordPress for you, but they are often priced out of reach for the smallest of news publishers.

WordPress is built so you can grow. It runs small blogs and underpins organizations as big as Al Jazeera. Its flexibility is its core feature. With some resources and technical staff, this flexibility is great because it allows WordPress to be anything you want it to be. In fact, a key feature of the Tiny News Platform is the ability to export all content to WordPress because we think it is the best platform for organizations of a larger size. But we didn’t feel it worked for our members when they were getting started.

Another knock against WordPress is the cost and complexity of maintaining the infrastructure it requires. The Tiny News Collective is built to support hundreds of news organizations. In a WordPress ecosystem, that would mean running hundreds of WordPress servers and MySQL databases. We have a small technical team of three people, and we didn’t feel comfortable managing that level of operational complexity. While WordPress does support multisite networks on one instance, we didn’t believe that tying together our members in a network like this was appropriate for the level of ownership we want them to have over their brand and domain.

Over the past few years, rapid advances have been made in serverless technology, allowing complex systems to exist without the need for running and managing servers. We wanted to explore the possibility of building a serverless publishing system that was easier to maintain and more affordable than WordPress servers and databases.

A simpler way to publish: Google Docs

We know that Google Docs is a common platform for drafting and editing content, and moving content from Google Docs to a CMS is always an extra step. We decided to start from the premise that we could allow users to publish content directly from Google Docs. This pushed us to work with Google Apps Script as a way to provide a custom add-on to Google Docs where we could handle publishing.

The result is a sidebar that allows a user to add metadata to a Google Doc. In the end, all a CMS needs to know about a piece of content is its title, summary, authorship, taxonomy, and time of creation. From this information, you can build a complete article object with headlines, descriptions, datelines, and SEO/social-optimized <meta> tags, as well as relate the article to authors, categories, and tags across the website.

We install a custom Google Docs add-on, available to pull up next to a Google Doc.

In this model, the sidebar handles metadata, but the Google Doc itself handles all of the content. We strove to use as much of the Google Doc natively as possible. That means no special formatting like in Markdown or ArchieML. All text formatting, such as bolds, italics, links, and lists, is done using the Google Docs formatting toolbar. Even images, alt text, and captions can be added natively in the Google Doc. Our only mild concession was handling embeds: to embed a YouTube video, a tweet, or some other piece of social media content, simply put the URL to the content in its own paragraph.

Adding images is as simple as drag-and-drop.

The trick here was transforming the Google Docs object available via the API into something that makes sense for a content database. We chose a data format based on the format used to store content in the Slate.js rich text editor. The taxonomy used in Slate is flexible enough to handle custom content types, and it gives us a backup plan if someday Google shuttles a service or API we depend upon for our platform. Storing content in JSON like this also makes it simpler to build out the webpages using React. We wrote a parser that transforms the Google Doc object into our custom format we keep in our database.

Designing for more equitable journalism

The central goal of the Tiny News Collective is to make news entrepreneurship accessible and achievable to more people from a much broader range of professional, educational, and lived experiences than are currently represented in our media ecosystem. Inherent in this work is ensuring our platform empowers more equitable journalism. This work will never be finished, but we’ve built some features from day one that we think distinguish us from other publishing systems.

For example, within our sidebar Google Doc add-on, we give users the ability to track the diversity of their sources in real-time. By default, we allow users to track sources by race, ethnicity, age, gender, sexual orientation, and location (by zip code). It’s an opt-in system, so users are not required to track sources on every story. But if they choose to do so, we take this data and provide a dashboard for analyzing source diversity over time.

When publishing, you can also add source demographic information.

We aggregate all source demographic information into an easy-to-read dashboard.

We believe that by building features that support more equitable journalism directly into our tools, we can help news organizations hold themselves accountable for covering the communities they represent. We hope to build more features like this into our tooling in the future.

From Google Docs to static site

The next step was figuring out how to store all of this information and communicate it both to our sidebar add-on and our frontend. We decided to use a GraphQL API provided by Hasura backed by a Postgres database hosted on AWS. GraphQL makes interacting with our highly relational database simple: we simply describe the data we want at any given time, and Hasura handles the rest. It’s even possible to use Hasura’s CLI to develop against a database locally and build SQL migrations with every change so that the API and database are completely reproducible. This is incredibly handy for us since we maintain separate staging and production environments.

Finally, to tie this all together, we use next.js as our web framework, deployed on Vercel. Next.js is a perfect match for our system, as we can build cheap and fast static websites off of our GraphQL API, and build serverless API endpoints to handle processing our Google Docs content. Next.js’s innovative incremental static regeneration system lets us build new static pages on demand without building the entire site.

In short, our Google Docs publishing system works like this:

1. Users create a document in a specified shared drive we establish for them and write content natively in Google Docs.
2. When ready to publish, users pull up the sidebar add-on while in the document and add the necessary metadata.
3. Users publish the content, which sends the metadata from the sidebar and the document object from Google to a next.js serverless API endpoint.
4. Our serverless API endpoint collates the metadata and the document and inserts them into our database via GraphQL.
5. Using next.js’s incremental static regeneration, we create or update the page for the article, as well as any associated pages like the homepage, author page, or category page.

A frontend built for speed

Publishing content is only one part of the business puzzle for a small news publisher. To grow toward sustainability, many founders are also eager to attract newsletter subscribers and donors, display ads, and build an audience for their web presence. The place where all of these concerns come together is the website itself. Even with the slickest publishing system in the world, without a website that effectively communicates your priorities, you’re sunk.

We worked with Alley, a design and technology consultancy, to design a simple but effective template that forms the basis for all of our members’ websites. It emphasizes newsletter subscriptions, donations, and ad placement while providing a beautiful home for native content. While our platform cannot extend beyond this template, we believe this is a benefit. Rather than micromanaging every little design decision, our members can get to work on the things that matter at the very beginning of the news business process: acquiring audience and building revenue.

The information architecture of our website supports all of the pages you would expect to have on a small news website:

• A homepage that promotes the latest stories and allows users to choose what stories are featured at the top. The homepage also promotes a newsletter, allows ads, and solicits donations.
• Article pages for all article objects. These article pages support ads and promote a newsletter at the bottom of all articles, a placement found to be most effective across the industry.
• Static pages for evergreen content. Rather than attributed to an author and category, static pages are for things like about pages, contact pages, and other informational pages that aren’t strictly journalistic content.
• Category, tag, and author index pages that list all articles associated with those objects.
• A donate page for soliciting donations from readers
• Complete newsletter archives built off of the Letterhead API

While our template is relatively fixed, we offer many customization options so every site can feel uniquely its own. We package these options as part of a product we call the TinyCMS.

In the TinyCMS, users can:

• Upload logos and favicons
• Customize typography and colors
• Write copy for microtext around the site: newsletter promotions, donation prompts, site descriptions, and more
• Choose featured articles and layouts for the homepage
• Build custom navigation
• Manage author, category, and tag records
• Check on key metrics critical to measuring success

We build all of this as a static website via next.js. This has two primary benefits. First, static websites are lightweight and inherently capable of handling high traffic loads. Publishers aren’t punished for success with slow load times and crashing sites when a story goes viral. Second, static websites are cheap to host. On Vercel, we only pay $20/month per developer seat for the entire collective. We can deploy as many static pages as we want. If we go over the standard bandwidth limits, we pay extra, but only marginally so. Because next.js bakes out complete HTML representations of every page, we still get the SEO benefit of server-side rendered pages, unlike many static websites built with React or other SPA-focused libraries.

Historically, the major downside of static websites is the build process required to update content. To update one page, the entire site must be built all over again. Next.js has brilliantly gotten around this issue with its incremental static regeneration system. With this system, individual pages get rebuilt as requests for them come in, set on a timer that works like a cache.

The results are clear: our websites are highly performant according to the Core Web Vitals scores Google now uses to rank search results. We regularly see Lighthouse scores of all 90s and 100s across our page types.

Building for the future

The current version of the Tiny News Platform is just the first version of what we can build. We have plenty of ideas about what we can build in the future. Come join us if you want to help make local news ownership accessible to all!

Support for News Catalyst and the Tiny News Collective comes from the Knight-Lenfest Local News Transformation Fund and the Google News Initiative.

The Tiny News Collective believes a world where everyone can participate in creating relevant, accurate and culturally conscious local news and information is one where everyone can more fully engage in civic life, make more informed decisions, and better understand the world around them.

For the latest news about the Tiny News Collective, sign up for our newsletter.

Interested in founding a news organization and not sure where to start? Sign up as an interested news founder for resources and information about the Tiny News Collective.