Risks of the City of Toronto’s Financial Systems Transformation Program
Background
I think Toronto is looking for an automated management control system. For example here’s a diagram from the “closeout” document for Toronto’s failed FPARS system (FPARS stands for Financial Planning, Analysis, and Reporting System).
Notice that goals flow down from City Council (at the left), are decomposed into services, and then finally performance measures flow back up (at the right) to close the loop.
It looks to me like Toronto is doubling down on this plan with its relatively new Financial Systems Transformation Program (FSTP) which has been awarded to Deloitte for design and implementation (~$44M). I infer this parallel by reading the “Lessons Learned” section from the closeout document (page 14) together with the Deloitte award document.
Here is a comparison of the mandates:
FPARS: The vision for FPARS was to build a service-based, performance-focused organization by improving how we plan, budget, and evaluate our services with clean financial, complement, and performance information organized by service. (page 3 of the FPARS Closeout document).
FSTP: This includes standardizing finance processes, modernizing its finance service operating model, and streamlining the underlying financial platform to ensure consistent access to timely financial information in an efficient and effective system. [and later, page 7] …enable future transitions to service-based, multi-year budgeting. (page 6, bottom of the Deloitte award document).
The key word here is “service”, as in “organized by service” (FPARS) and “service-based […] budgeting” (FSTP).
The City of Toronto means something very specific by this term “service”, though it’s not clearly highlighted in the documents. From page 8 of the Closeout document:
Program maps for each division were developed that categorized services provided, with a further breakdown to specific activities and sub-activities by service to enable the ability to directly link cost and budget information to services. The individual program maps, as developed by the project team, are a hierarchical representation of the division’s customer centric service model and are based on the Municipal Reference Model (MRM), Government of Canada Strategic Reference Model (GSRM) and the Ontario Public Service Reference Model (PSRM). [emphasis mine]
For some reading about this, see these links
- MRM (Municipal Reference Model)
- GSRM (Government of Canada Strategic Reference Model)
- PSRM (Ontario Public Service Reference Model)
- also, GFOA (Government Finance Officers Association) on Recommended Budget Practices from the National Advisory Council on State and Local Budgeting referenced later (page 9) of the Closeout document
The basic idea is to rely on a layer of abstraction for understanding government services, based on measurable outcomes to services that users recognize and consume. The Toronto civil service interpretation is that the services should be reported independently of the underlying organizational structure. The organizational structure supports services in this view by performing, in aggregate, the “activities” required to deliver those services, and is therefore of secondary importance.
Moreover the service abstraction layer is to be formally, and apparently statically, codified and institutionalized, making it difficult to change.
Here’s a diagram from a presentation given about this in 2017 by the Financial Planning Executive Director at the time, Josie LaVita:
“Programs” are City Divisions and Agencies. Those blobs in the middle are services. There is intentionally not a one-to-one link between them. The plan seems to be, for reporting purposes (like budgets), to abandon the familiar city programs in favour of the new services. I call this an “orthogonal ontology” — the ontology is the classification scheme, and the scheme is orthogonal to the programs (organizational structure) of the city, meaning that services and programs intersect, but services are independent concepts.
In fact, one of the “lessons learned” from the FPARS experience, according to the FPARS Closeout document, is that services identified by a City inventory of services thus far and classified as sub-categories (dependent categories) of City Divisions are inherently wrong. From page 8:
The program maps were developed on a divisional basis and as a result, outline the services provided by the division, not services that lead to a measurable outcome
I don’t agree with this of course. Divisional services are perfectly capable of being measured, but it reveals the mindset here — de-couple services from organizational units.
This de-coupling of services from organizational units is a two-edged sword. On the one hand, it provides the flexibility of imposing a “pure” orthogonal services model on the classification scheme. On the other hand, it risks weakening accountability for results, as well as generating confusion.
In any case, the managers of this transformation program have set themselves an enormous challenge. I think they’re being over-zealous about it, and there are easier and better ways to achieve the same goals. In the past I’ve spoken with some people involved with MRM and GSRM, and I was given the opinion that it’s quite possible to be over-enthusiastic about the concept of a “pure” service based system. One knowledgable individual even told me that MRM was no longer considered best practice, owing to the extra burden it places on people and systems.
There’s another, more insidious problem. The aim of the City has been to deeply and formally embed the services hierarchy into the systems that are used to collect and report data — the budgeting and accounting systems. Clerks are required to classify inputs to the correct services, together with the standard requirement to correctly classify inputs according to common accounting rules. This adds to confusion and non-compliance among City staff (I’ve spoken to a few about this over the years). That then adds to the already pervasive problem of extremely poor data quality and consistency in the City’s various financial systems. From the Closeout document (bottom page 13):
Throughout the FPARS lifecycle, it was known that the quality of data at the City required improvement. As a result, data as it stood may not have been reliable for use in measures, analytics and visualizations, as it could provide incorrect conclusions.
This is classic bureaucratic understatement. The data is a mess. And the extra and unnecessary burden of having to double-classify all data entries is partly to blame. There are better, easier ways of classifying costs to services (for example by classifying City cost centres — see below).
While the goal of linking services to City Council strategies and ultimately citizen’s goals and missions is laudable, I think the implementation Toronto has in mind is fraught with peril. This peril has already been proven once with the failure of FPARS, and that failure risks being repeated with FSTP.
I should add that in 2013 Toronto’s Auditor General issued a scathing report about the FPARS project management, which should have generated real alarm and bold, decisive corrective action within senior management ranks. Instead management let FPARS problems fester for another eight years until it failed. It wasn’t just an FPARS failure. It was also a senior management failure. Now, instead of focussing on fixing those management problems, the current senior management has chosen to outsource responsibility to Deloitte. So it’s a missed opportunity as well.
For still more perspective on the failure of FPARS, you can see what I wrote about many of FPARS’ problems in 2019, here.
Risks
Budgets must be useful to all readers. Budgets must be interesting. For the City of Toronto, readers include councillors, senior managers, middle and junior managers, City staff, the business community, the civil society community, and well informed members of the public. Creating a system that successfully serves all these audiences is hard, but critical to a functioning democracy in my opinion.
Toronto’s stated vision is to have a state of the art financial reporting and analysis system for its audiences through FSTP. Unfortunately there are a lot of risks associated with the FSTP plan as currently constituted. Here are the main risks that I see. (See the following main section “Options” for some ideas that may help).
1. The use of the Waterfall-to-Colossus method
It’s clear from the budget and timeline of the FSTP project that the intent is to implement it all as a single large product. First do design, then move to implementation, then maintenance. This is called the waterfall method (one thing follows another). It has long been recognized that this is not the optimal way to develop large projects. The main reason is that it doesn’t allow enough flexibility to integrate discoveries made during implementation, into the design. It doesn’t have quick feedback loops built in to the extent needed.
In fact the problems and “lessons learned” cited in the FPARS Closeout document is a case study in the need for more agile, incremental, development methods. The problems cited were themselves not a cause for shame. To the contrary problems are to be expected. But the lack of capacity to use those discoveries to modify the development constructively was a huge problem. Senior management has wrongly chosen to follow the same rigid waterfall method that contributed so much to the problems of the FPARS system.
An example of clear evidence of the benefits of smaller, therefore inherently more agile, development modules is a survey done of government projects. See here for details. Here’s an extract:
I’ve seen recommendations that government software contracts should never exceed $2M. FSTP has a $44M budget.
There is, in particular, a screaming need to work out in a constructive way co-operation with City Divisions and Agencies to resolve the problems of data quality, standards compliance, and consolidation discipline. The City seems to want to rely on imposed compliance programs for these. Surely it would be much better to rely on one or more small “pilot” programs, in partnership with one or more Divisions and/or Agencies, to discover and develop monthly routines that succeed in transferring high quality data to a centralized City database. Successful approaches could then be applied universally (though still in partnership with the entities involved).
For example local units, down to the cost centre, could habitually use monthly or even weekly financial reports, to compare budget and actual numbers, examine trends, and consider comparisons with others and experiences on the ground. Those reports would then constitute feedback to data-entry, and would presumably motivate corrections and continuous quality improvements. Quick, close-to-the-ground feedback on data entry should promote accurate data collection.
Agile development is better than waterfall.
2. Ontologies are hard; Orthogonal Ontologies are harder
Ontology is the definition of concepts in a domain, with their properties and relationships.
By “orthogonal” here I mean intersections of ontologies with organizational hierarchies, with independence of the ontologies. This means that services would be defined to be orthogonal (intersecting but conceptually independent) to organizational units.
A taxonomy (the implementation goal) is the formal classification, including principles, relationships, and terminology, of hierarchical ontological concepts in a domain.
Ontologies, which try to capture the nature of things, are rooted in culture, language, and understanding. They cannot exist outside of their contexts. This is what makes them hard to develop. My experience is that they take time to evolve. But evolve they do, and they must, particularly in early years. They must be relatable, and reflective of the reality in which they operate. They (like budgets) must be useful. Ontologies are organic. Good ontologies cannot be bureaucratic.
The ontologies that the City seeks must by the City’s definition be rooted in measurable user outcomes, and moreover must through “activities” find complete coverage in existing City programs (Divisions and Agencies). Corollary-wise, no organizational activities can exist which are not part of a service.
Finally the ontologies must be expressed as taxonomies — formal definitions using familiar (across many audiences) language.
Let’s see how the City has done so far. Have a look at 2021 service levels here. There are 123 pages of tables of service details, including activities, sometimes types and subtypes, and ordinal measures by year. To me, in general the outcomes are process outcomes, not service outcomes reflecting measurable changes (hopefully improvements) in the circumstances of the service users. Also the services are grouped by City Division and Agency, precisely the structure that the Closeout document found to be in error.
As an example, I noticed that more than one of the services have some version of this statement:
Arts Services [ed: or fill in your favourite service] is committed to increasing its service levels after monitoring the delivery closely and using synergies and efficiencies to enhance the customer experience to our citizens and provide better results.
It’s hard to find any concrete meaning in this kind of statement, and therefore hard to infer good understanding on the part of the writer.
The services in the referenced service level document are (against policy) currently grouped by Division and Agency. When they are de-coupled (if that ever happens) they would need their own higher classification scheme. I presume this process hasn’t been started yet, or at least hasn’t developed to the point of release. The City writings suggest that the service hierarchy should be formally tied to City Council strategies. This would be another challenge, given that it assumes City Council has a coherent and comprehensive set of strategies. Good luck with that. (It may be helpful in this regard to start with a carefully crafted set of strategies. See for example Doughnut Economics by Kate Raworth, or the Doughnut Economics Action Lab — DEAL.)
If the City insists on a detailed ontology, I believe that classifying the 13,000 or so City Cost Centres would be sufficient (thus obviating the need for input clerks to classify entries by service as well as accounting categories). But even so creation of a taxonomy would have to be done in a technical environment conducive to change, and supporting multiple concurrent schemes.
At the bottom of page 8, the Closeout document states:
To comply with the best practices established within the MRM to achieve a customercentric service model, the divisional program maps need minor review and refinement to ensure divisional services are being identified at the appropriate level.
This is an extreme understatement, in my opinion. I have a very hard time seeing this process of refining service definitions coming to a successful outcome any time soon.
3. Briefly…
i. Technical “solutionism” can sideline human interests
Solutionism is the belief that every problem has a technological solution. Here’s an article Stop ‘Solutionising’ and start focusing on the user that briefly describes the benefits of continuously focussing on users as design drivers over technical solutions. After all, good service definitions and classifications depend on people. Production of good data depends on people. And successful use of services depends on people.
The FSTP has quite a technical focus. For example the benefits of the program from page 6 include (summarized):
- A single authoritative source of financial information…
- Reduction of manual processes…
- Standardized accounting structure…
- Stronger governance of key financial processes…
- A reliable, automated, and robust financial platform…
Also, on page 1:
The new version of the core financial platform provides the City with the opportunity to review and build more efficient and effective business processes and consolidate the number of systems and financial information which will support an integrated and seamless delivery environment that will be critical for all City operations.
These are quite general attributes, which focus on the system itself, more than the human support and usability goals that could be reached with the assistance of the system.
User-centred design is better (see below).
ii. Tight timelines on large, complex projects invite short-cuts and compromises
The FSTP is large and complex, and as was learned with FPARS, there is likely to generate an ongoing stream of learnings, which should be fed back into the development process with a view to making changes or corrections. I can’t imagine such an ambitious project being well completed by 2023 as scheduled.
iii. The continuity problem
Analytics, which underlies opportunities for insights and decision support gained from a successful financial system, depends on presentation of data along timelines, and the ability to make comparisons and contrasts with similar contexts.
A radical transformation such as is being contemplated by the City in my view portends massive structural changes and instability in the data, and therefore at least for a time, difficulty in accessing the timelines and comparisons which are helpful in decision support.
With a complex layered classification system such as contemplated by Toronto, changes would, or should be frequent, particularly in early years, contributing to the continuity problem.
iv. Risk of hidden agendas
From page 6 of the Deloitte award contract, as part of a “benefits” list:
Standardized accounting structure through a redesigned Chart of Accounts.
Stronger governance of key financial processes through a centralized repository of financial policies.
In a report titled Value Based Outcomes Review: A Fiscal Modernization Agenda for the City of Toronto, by Ernst & Young from 2019, which I have always taken as a trial balloon raised by the City of Toronto’s City Manager Chris Murray, there’s a section on page 10 titled Fiscal Management Institutions:
Overall, the City is in a position where it is larger than many provinces in both size and ambition but lacks the institutions that are standard in provincial governments such as strong expenditure management functions for both capital and operating budgets. For example, a centralized treasury or management board function might enhance ownership of the overall operating budget, long-term outlook, and multi-year capital plan, and improve overall accountability for expenditure commitments over time. While accountability for individual objectives and budgets could continue to sit with operating functions, co-ordination across the full budget at a consolidated level could be strengthened and enable greater focus on City-wide priorities. [emphasis mine]
I would actually support this initiative, if it is done in a way that promotes collaboration and co-operation. But the difficulty in standardizing on the new services scheme might be another incentive for senior management to assume greater control. This should be watched.
Options
There are always options to improve development programs. Here are a few suggestions.
1. Modify Toronto’s organizational structure to align more closely to services
I think Toronto’s structure is currently quite good, actually, so I don’t see a huge change if this was pursued. Perhaps adding a Commissioner system for overseeing municipal missions would be helpful.
I’ve personally been using a taxonomy I developed over many years, and I think it’s pretty straightforward. Here’s a high level summary of expenses, for example:
Here’s an early version of my functional taxonomy, from 2017: Functionally, the Toronto budget isn’t that hard to understand
2. User-Centred Design
To repeat what I said above, budgets are only as good as the usefulness to their readers. A good place to start would be to make budgets and actual comparisons useful to smaller cost centres and organizational units within the civil service. Iterative feedback through consulting, updates.
3. Multi-dimensional budgeting
Given the number of audiences for Toronto’s budget, a multi-dimensional approach may be helpful. The generic dimensions that I recently used in an analysis of Stratford Ontario’s budget for instance, are
- A functional budget
- A resource allocation budget
- A taxation requirements budget
See the Stratford report for details. I’ve also created an interactive dashboard for Toronto’s 2021 budget which uses multi-dimensional budgeting.
4. Public Dashboard
The number of combinations and permutations of budget series is too high to present in a single report. So I think it’s essential to incorporate dashboard capability in any transformative system. The point of the dashboard would be to allow a wide range of users to select presentation criteria that most suits their needs.
5. Granular data
Most people are interested in, and often passionate about, very specific budget and financial issues. This calls for access to highly granular data. For Toronto this means providing data to the Cost Centre level. There are about 13,000 Cost Centres in Toronto, last time I asked.
6. Classify cost centres, not transactions to categorize data
If fine-tuned service classification is required, I can’t imagine that assigning Toronto’s 13,000 cost centres to a services scheme would fall short of requirements. This would have the advantage of separating classification schemes from the core data. Core data can be classified according to more conventional, standard accounting rules (using a Chart of Accounts).
7. Use management accounting techniques.
Management accounting is an accounting specialty designed to support decision-making by providing specialized reports to management. This specialty can be applied to production of classification schemes used to provide insights for budget readers. Management accounting relies, among other things, on statistical methods, and assumptions in decomposing aggregates according to specialized investigations. Much of what is contemplated for the services classification for Toronto could be accomplished through management accounting techniques.
8. Build modification capacity into taxonomy
As noted above, the classification schemes developed will of necessity have to change to keep up with user requirements. Therefore system support for evolution should be built in.
9. Pay attention to the continuity problem
Trend observation is a critical part of analytics (to gain insights). To be accessible, timelines need to have good continuity in classification schemes. With the depth of the changes contemplated by the City of Toronto, continuity will be broken. I believe it would worth paying some attention to this, with a view to retroactively re-imposing continuity to the more modern classification schemes. Also ongoing changes to classifications are likely, particularly in the early years, so standard methods for re-establishing continuity would be helpful.
Henrik Bechmann is a retired software developer with an interest in all things related to the Toronto budget. From 2015 to 2018 he was the lead of the budgetpedia.ca project. Twitter: ‘@henrikbechmann’
