6 things I learned the hard way leading business systems

Rich Archbold
15 min readFeb 13, 2023

--

When I was starting out in Business Systems, I could find very little writing on the web to help me understand what I was getting myself in for, or how to get started, especially from the perspective of someone coming from a product/infrastructure engineering background.

My first year in Business Systems was the most difficult and humbling of my professional career to date.

It pushed me further outside of my comfort zone than I’ve ever been before.

I’m glad I did it, but it was definitely the hardest thing I’ve ever done in a work capacity and it exhausted me on a number of occasions until I could get a handle on things.

This blog post is my attempt to rationalize and share what I’ve learned about running a business systems organization in the hope that it will help others starting out in this role be sustainably successful and avoid burnout.

Introduction to Business Systems

A business systems organization builds and operates the technology platform and solutions that brings a company’s GTM strategy to life. It is therefore a critical organization within larger B2B SaaS businesses.

It is responsible for the systems and tools that enable customers to discover, purchase and pay for products and services.

It also builds and operates the complementary technology tools needed by sales, marketing, finance, customer service and legal teams to support customers during their purchase journey.

The business systems team is usually made up of three key functions:

  1. Growth/Pricing & Packaging/Billing
  2. GTM Systems
  3. Data Engineering

This is a diverse set of teams with several different job functions such as Business Systems Analysts, Business Systems Engineers, 3rd Party Software Systems Admins, Software Development Engineers, Data Engineers, Data Analysts, Program Managers, Product Managers and Designers.

It’s quite a different organization than you usually experience in a product engineering world and it takes time and investment to properly understand its makeup and unique ways of working.

My journey to Business Systems

Prior to 2019, most of my 20 or so years of software engineering experience had been in the fields of infrastructure, cloud and backend engineering. I had led product engineering teams and groups too, but it wasn’t where I had most of my experience or where I’d done my best work.

Sometime in 2019 I started getting involved in Go To Market (GTM) Engineering by responsibility for fixing some thorny issues in Intercom’s billing system.

I quickly realized (a) just how complicated and difficult it was to run a subscription billing system and (b) how difficult it was to work with GTM stakeholders when I didn’t have sufficient understanding and empathy for how their jobs worked.

However, I also quickly came to appreciate how important this space was and that there were lots of valuable new set of skills to learn and experience to gain, so I was hooked.

In November 2020, shortly after Intercom hired its first CFO, I moved out of Product Engineering and switched to report to our CFO to form and lead Intercom’s first incarnation of a Business Systems organization.

In this role I lead our:

  1. GTM Applications function, responsible for Sales, Marketing, Finance and CS technology needs. The main technology platforms in this space were Salesforce, Marketo and Intercom.
  2. Billing function, responsible for all subscription management activities such as account provisioning, metering, usage calculation, invoicing and all other billing related activities. The main technology platforms in this space were a homegrown ruby on rails billing system and Zuora.
  3. Data engineering function, responsible for scaling and operating our data warehouse and all of the associated ETLs, and the creation and maintenance of some higher level core data assets that critical company reports were built on. The main technology platforms in this space were Redshift and Tableau.

I also provided a lot of engineering leadership cover to Growth Engineering teams and in particular was heavily involved in the implementation of Intercom’s pricing and packaging initiatives.

Lessons Learned

Lesson 1: Deep knowledge and experience of how Sales, Marketing and Finance organizations work is more valuable to you in this role than a deep knowledge and experience leading software development teams

In the role of Business Systems Leader, you are usually responsible for figuring out what to build as well as actually building it. You need to wear the metaphorical PM, Design and Eng leadership hats.

If you don’t already have a deep understanding of how sales, marketing and finance teams work in your industry and the most common technology solutions that are used to support them, then you likely won’t be able to perform any of your leadership duties well or fast enough.

You might think that because you are smart, curious, have strong computer science fundamentals knowledge and a track record of getting stuff done that you can figure it out as you go …

However the complexity of this role is in the awareness and understanding of the plethora of important, intersecting and interdependent business processes that take place throughout the buyer journey, not in the technical complexity of how any one of them are implemented.

There is a large amount of incompressible wall-clock time that needs to be spent experiencing and learning these processes so that you can properly support them that can’t be shortened.

To succeed in this role you must become an expert on your:

  1. Lead to Opportunity process
  2. Opportunity to Cash process
  3. CRM and its integrations
  4. Billing System
  5. Finance’s core analytical data assets

The more detail on the above topics that you know before you go into this role, the faster you will ultimately get up to speed. Only when you are sufficiently familiar with all of the above 5 topics will your software engineer fundamentals become an asset to you.

Understanding to some decent engineering depth how your Lead to Cash process works, how you bill and invoice customers and how finance measures your business is actually really, really valuable context to have, no matter what engineering leadership role you have. Regardless of whether you are in Product, Platform or Infrastructure engineering these are some areas that I strongly recommend that you learn about.

Lesson 2: Strong operational quality of your core GTM data objects is the key to achieving excellence throughout your GTM funnel

One of your organization’s responsibilities is to ensure relevant data is collected, recorded and updated as business flows through your sales and marketing pipeline.

There are a few critically important core data objects in this pipeline: Leads, Accounts, Contacts, Product Catalog, Opportunities/Deals, Contracts, Invoices.

If you can’t rely on these objects being accurate then everyone’s productivity (and morale) will be negatively impacted, you won’t be able to forecast, report or analyze your business accurately (which creates compliance and business risk) and individual customer experience will suffer.

It is therefore critically important that core GTM object data quality can be relied upon at all points in the funnel.

Some of the main causes of data quality errors are:

  • Weak Object design. Objects designed and implemented without the uniqueness constraints. Object attribute meanings and uses are not documented appropriately.
  • Weak Object attribute management: Weak syntactic or semantic validation of data inputs to ensure data is type is correct and data is complete and correctly formed.
  • Weak Lead and Account Enrichment: Leading to incomplete or duplicate entries.
  • Buggy CPQ implementations: Leading to inaccurate Quotes, Contracts or Account-level financial data.
  • Buggy Billing Systems: Leading to inaccurate Invoices.
  • Weak data integration between systems. Leading to low or high-grade chaos or friction.
  • Lack of investment in data quality monitoring and repair. Leading to a silent accumulation of huge data quality debt.

Some tips to avoid data quality issues include:

  • Make maintaining good data quality a part of your culture. Educate everyone on the importance of core object data quality. Make good data quality a key objective of all initiatives that you undertake.
  • Ensure your team has strong engineering leaders that are skilled and experienced at designing, implementing and maintaining strong data quality through a distributed system.
  • Invest in staffing and initiatives that systematically measure, improve and safeguard your data quality, such as keeping your data schema and system architecture up to date, automated data quality checks, regular data audits, improving semantic and syntactic data validation checks and improving your point-to-point data integrations.
  • Treat data quality issues like you would application/website outages, prioritize fixing them above other work, conduct a post-mortem afterwards and implement the lessons learned.
  • Create KPIs and goals around data quality and review them regularly with your team.

A lot of skill, discipline and hard work is required to achieve and maintain good core object data quality, but believe me, it’s worth the investment. It helps you avoid near-term failure and makes long-term, sustainable ​​success achievable.

Lesson 3: A comprehensive and well managed work intake, prioritization and communication process is essential to survival

A business systems organization serves many different teams, each with a fast evolving set of unique needs and requests.

There will always be a lot more work than you have resources to work on.

It is very important that you instantiate and maintain an effective and efficient process for interfacing with all of these stakeholders, gathering, fleshing out and prioritizing their requests and communicating back to them the prioritization decisions and progress updates.

If you don’t, you will get swamped dealing with the ad-hoc requests for new work and updates on prior work requests.

Setting up and operating the necessary intake and stakeholder management processes isn’t an easy job, nor is it a part-time one.

If you don’t prioritize this work highly enough or if you don’t have enough of the right people who can focus on it fully then you will fail.

PM’s and BSA’s are usually the ones who flesh out stakeholder requests into actionable work items, but they usually don’t have enough bandwidth to also design and maintain all of the other processes that are needed for strong, holistic stakeholder management. My advice is to hire some TPM’s or build out a PMO function to help with all of the other operational work required for great stakeholder management.

Lesson 4: Don’t cut-corners or compromise your values in pursuit of (short-term) revenue gains.

In the GTM space, we often undertake important, ambitious, complex projects that have revenue targets against them.

These projects often have aggressive target delivery dates assigned to them that are baked into revenue forecasts.

The unfortunate truth is that the delivery dates often prove to be unrealistic, because they are set before the full scope of the work required to deliver them is understood.

In such circumstances, the delivery teams are often pushed to offer trade offs on how they might go faster.

The delivery teams are sometimes pushed to make temporary compromises around technical quality, employee experience or customer experience in an attempt to preserve the amount of revenue the company can earn.

Once the project is completed and the revenue is in the bank, delivery teams will then circle back and fix any compromises that were made in the first place.

While this might sound like a reasonable approach to take, it’s been my experience that the happy path described above almost never plays out.

  • The revenue targets are not achieved (in part due to the compromises made, in part due to the size and complexity of project)
  • And (in part because the revenue targets were not achieved) the delivery teams don’t get time to properly go back and fix the compromises.
  • And even if they do, some amount of employee and customer trust is lost once they experience the compromises.

As difficult a pill as it is to swallow, you’re better off admitting that you will miss the original goal and rescoping it to something that you can deliver on in a way that doesn’t compromise your values and principles relating to technical quality, employee and customer experience.

Some tips to avoid getting into this situation include:

  • Involving the right engineering folks as early as possible, and don’t commit to delivery dates until they have a chance to do appropriate scoping, planning and estimation.
  • The scoping, planning and estimation phase should include a “futurespective”, where delivery risks are flushed out and accounted for.
  • Whenever projects of this nature do run long, and compromises are made, conduct retrospectives to understand where things went wrong and incorporate the lessons learned into your planning processes going forward.

I’m not saying you should never make compromises to hit an important revenue target, but it’s worth knowing that the cost of compromising engineering quality, customer or employee experience is almost always significantly higher than you imagine, they are harder to undo and I’ve rarely, if ever, thought they were worth it.

Lesson 5: Practical tips for onboarding into the role

Understand the why, how and what of your company’s monetisation strategy at the highest level.

  • Find out who owns pricing and packaging and ask them to talk you through your pricing strategy.
  • Talk to the head of RevOps or Sales Ops and ask about your revenue goals for the year and what needs to happen for you to achieve them..

Develop some first-hand customer empathy for your buying journey.

  • Do some “secret shopping”. Sign up for your company’s products and test out the onboarding process as a customer would experience them from the outside.

Develop stakeholder empathy at the frontline, manager and leader level.

  • Do some ride-alongs with all key front-line positions of marketing, sales, support and finance.
  • Once you’ve done these, conduct manager and leader interviews to get their perspective on what’s working well and not so well.

Learn how your core business processes and business systems work.

  • Talk to your business analysts to uncover how your lead nurture, lead qualification and routing, new business, expansion, renewal, collections, forecasting and quarter close processes work at a business and process level.
  • Talk to your engineers and admins to understand how these processes are brought to life through technology. Ask for systems and workflow diagrams, get people to whiteboard it all out for you. Ask what’s working well and not so well. Ask about how changes are designed, developed, tested, deployed and monitored.
  • Assess your level of technical debt and the strength of your developer environments and development best practices.

Get to know your core data models and core data integrations points to help you assess your data quality strengths and weaknesses.

  • Talk to your engineers and data analysts to understand what your core data models are, how they flow through operational and analytical pipelines and learn where your quality issues are, and what stakeholders most want fixed or improved.

Assess your work intake, prioritization and roadmapping processes.

  • Talk to your product and program managers to understand where roadmaps come from, how they are prioritized and whether the process is working well.
  • Understand whether your roadmap process is flexible enough to account for the occasional ad-hoc or late-breaking request.
  • Talk to your key stakeholders and ask them how happy they are with the process and what they feel needs to be improved.

Get to know your team, assess their size and strength relative to the job at hand.

  • Talk to as many people in your organization as possible, to assess their skillsets, strengths, areas for growth, performance and engagement levels.
  • The Business Systems ecosystem can be large and complex, look to see how many, if any, leaders, managers and individual contributors have a solid understanding of the overall systems and data architecture. These people usually have a staff or principal title. If you don’t have folks like this in your organization, or if you do have them but they don’t have the right breadth and depth of understanding of their domain then it’s usually a warning sign that you either have a staffing gap that needs to be filled or that you need to afford people more time to understand and master their domain. A lack of sufficient, primed principal / architect roles in your organization is something that will definitely hold you back from making strategic progress.
  • The Business Systems function can often be a fast, paced, high pressure environment, that can easily lead to burnout if not managed well. Pay close attention to the morale, engagement and culture that you experience. Ask about sustainability of the work to assess how your org is doing and ask people what they think needs to be done to improve it. Does everyone feel like they are getting the support and recognition that they feel they and their teams deserve?

Lesson 6: Practical tips for ongoing leadership and management in the role

Once you’ve finished your onboarding it’s time for you to define (or refine) the mission, vision, strategy and values for your org, to set some goals and create (or refine) some core processes that will help you achieve sustainable success.

One way that you can frame the mission of a business systems organization is to … Enable our business to grow better every day, by providing our GTM teams, and where appropriate our customers, with the comprehensive, valuable, easy to use technology solutions throughout the full lifecycle of the buyer journey.

Some important factors we need to keep in our minds at all time are:

  • Customer satisfaction and experience. Are the services and experience we build and operate loved by customers?
  • Stakeholder satisfaction and experience. Are the tools and data capabilities we provide to our internal stakeholders comprehensive, valuable, reliable and easy to use?
  • Engineering excellence. Are our core systems and data sets sensibly architected, excellently and consistently implemented and relatively easy to operate and extend? Do we adhere to good engineering standards?
  • Team health. Are our teams whole, stable, happy and high performing? Do we have measurably strong retention, engagement, growth and progression within our teams? Can we rely on our teams to meet their commitments and deliver good incremental business value?
  • Bottom-line business value. Is our organization delivering on its top line business goals and doing so in a lean, efficient manner? Do we have control of our costs and are we actively managing them?

Some metrics that can help you track your performance on the above themes are:

  • Customer NPS relating to their purchase experience.
  • Stakeholder NPS surveys relating to the tools and services that you provide them.
  • Lead Enrichment reliability
  • Lead Routing timeliness
  • % Sales/Marketing Email Deliverability
  • % of Sales workflows that complete without requiring a support case to be opened to business systems
  • P90 time to respond to cases opened by internal stakeholders to each of your teams
  • Time (in days) to close the books with Finance at the end of each quarter
  • % of ETL jobs that complete successfully, within a specified timeframe, without manual intervention
  • % of accounts with unique, complete, accurate data
  • % of contracts without accurate, complete data
  • # of issues raised to us that are caused by data quality issues
  • # of compliance audit exceptions
  • Engagement, attrition and promotion rates of employees in your organization
  • Manager effectiveness scores of managers in your organization.

Some strategies that can help achieve positive trajectory and results with these success factors and metrics include:

Systematically invest in stakeholder empathy and alignment.

  • The more you understand the goals, core processes and ways of working of each of your stakeholders the better you can partner with them and deliver for them.
  • Hold regular 1:1s with your stakeholders, ask to attend their team meetings, invite key stakeholders to present at your team meetings, create processes that ensure your team members are systematically continuously deepening their understanding of the needs of their stakeholders.

Develop and maintain a strong shared, documented understanding of your core business process and the technical systems architecture and data flows that support them.

  • Inertia and bad decision making arise when people have the wrong or missing information. Given that the business processes that a Business Systems organization supports involve and impact many different functions it’s too easy for knowledge silos to occur.
  • Ensure all core business processes (Lead ingestion/enrichment/qualification/routing, Sales-owned new business, Sales-owned expansion/renewals, Self-serve sign-up, Billing+Invoicing, quarterly forecasting+reporting, etc) all have a dedicated owner within your organization and that someone always has expert knowledge of the business process and technical implementation of these processes, and that knowledge is durably recorded and scalably communicable, and that it is regularly communicated to all key stakeholders.

Develop and follow strong changement management processes to minimize the accrual of bugs, process and technical debt.

  • Bugs and technical debt creep into a system one change at a time. In today’s environment, where the desire for fast and frequent change is demanded, it’s very easy for large amounts of technical debt to accumulate if individual bugs are not caught and fixed before (or very shortly after) they get into your production systems.
  • Leveraging your strong, documented shared understanding of the business process that a change is impacting, put in place standardized QA testing templates and allocate sufficient time to execute them as part of any material production change.

Streamline your intake, prioritization and communication process to eliminate the need for interrupts and ad-hoc requests.

  • Hire at least one awes Senior (Technical) Program Manager to own the design and operation of your intake and stakeholder communications process.

Systematically invest in GTM core object data quality.

  • Architect your GTM tech stack to not only deliver the required stakeholder facing functionality, but also to ensure reliable, timely data quality throughout the systems.
  • There are a few critical, standardized, core data objects in a GTM pipeline (Leads, Accounts, Contacts, Product Catalog, Opportunities/Deals, Contracts, Invoices) and it is critically important that their data quality can be relied up at all points in the system and that replication of these objects between systems can be relied upon in a timely manner.
  • The greater the number of pieces of unique software solutions and data stores that your GTM system contains, the harder it is to maintain good data quality throughout the system.
  • Bad data quality leans to friction, inefficiency, bad decisions, inaccurate reports and bad customer experience. It is not an overstatement to say that weak GTM data quality can, overtime, ultimately tank your business.
  • Aim to have as few software components as possible, with as few data stores as possible and make sure data integration points are robustly implemented as possible. Don’t assume the happy path for data integrations. Put in place appropriate metrics, monitoring and alarming and dedicate staff to ensuring your data integrations continue to work as required and the data quality of your core data assets are at the level you need them to be.

Over communicate your work in ways that best resonate with your stakeholders.

  • Make sure you and your team are writing and sending appropriately detailed written status reports at regular intervals.
  • Don’t hide behind these status reports, no matter how detailed and accurate they are. It is vitally important that you regularly present in person or via VC to your stakeholders, restating your vision and strategy, providing meaningful updates, doing demos and asking for feedback.
  • You won’t be able to properly serve Sales, Marketing or Customer Support unless you invest in building real human connections with them.

--

--

Rich Archbold

VP of Engineering @ Hubspot (ex Intercom, Facebook, Amazon)