How do we make Scotland the best digital state in the world?
There is a desire among the political class for Scotland to be a world leading digital state, and a fear of failing, a fear born of no clear road.
This proposal is such a clear road — starting small, working iteratively, and managing, containing and learning from the inevitable (small) failures along the way.
- This proposal is based on experience in both the Scottish public sector and the burgeoning internet community in Edinburgh.
- It proposes a whole scale change in approach to how public sector reform should be done — from an external, private sector driven approach to an internal one.
- It is premised on the fact that the cost of computing has become commoditised and crashed, and that the barriers to excellence are not human factors and not capital.
- The format and structure of legislation hasn’t changed since the introduction of large scale computer systems in the public sector — even though most administrative law will be computerised — this cannot be right.
- It proposes an iterative, experimental development of capabilities across the entire public sector, based on existing islands of excellence. This approach draws on cutting edge private practice, but is very far from simply copying it.
- As this work is based on human factors the burden of it will fall to the OpEx budget and not the CapEx, it is about enabling the existing staff and organisations accomplish more — better outputs and not more inputs.
- It proposes a small team with a graded set of experimental tasks. If these tasks are successful they should be able to achieve and demonstrate benefits quickly which can gradually be scaled and rolled out. Some of the experiments will fail to deliver results and the programme is designed to stop these early.
What stands in our way?
The dominant trend of technology has been the collapse in capital costs of computing. The most powerful computer in the UK in the 80s, the Crays at London Uni and the Weather Centre are 1/6000th of the power of an iPhone. The servers we bought at if.com in 1999 for £1.5m the pair are now in student’s backpacks, on the tables in coffee bars.
To put this in terms politicians will understand:
It flows from this that the barriers to Scotland becoming the best digital state lie in the human factors, and the human factors alone. From this it follows that the transformation of the Scottish State is down to OpEx and not CapEx expenditure — better use of existing resources, not new inputs and budget increases.
We know from the disastrous history of big bang computer projects that incremental and evolutionary change — mutating a working system to an incrementally better working system — is the only way to transform government services.
For too long the political class has looked over at the tech sector and dreamed of emulating it, the great conquering of a new market, the building of a new world. There was a time when the state did do that, but the age of imperialism is over. (The fruity end of the tech world, the Peter Thiel’s, dream of bringing this back by creating new countries through sea-steading, with new East India Companies to rule them.)
Our task is to use technology to deliver low-cost, efficient and effective public services to our citizens and private sector — and that is not an ignoble or minor task.
At the heart of this task is embracing complexity. A lot of simplistic nonsense about a bonfires of red tape and quangos, and privatisation has driven debate about the efficiency of the state for the last 30 years and needs to be put to bed.
Consider the iPhone. A partial list of the ‘regulations’ that pertain to it are:
- international agreements on use of radio frequency spectrum
- technical standards on telecoms interoperation
- technical standards on internet interoperation
- cryptographic standards
- safety regulations pertaining to the manufacture and testing of its potentially explosive batteries and components that contain toxic chemicals
- legal wrappers that enable users to purchase goods on it in most countries in the world — in 100+ languages
- integration with global payment systems
- integration with global tax systems
- end-user agreements in 100+ languages
- API specifications for 3rd party developers
- usability specifications for 3rd party developers
- UX/UI specifications for 3rd party developers — including presentation of UX/UI elements in 100+ languages
- Apple store approval and submission guidelines in a large number of langauges
Were you to print out this ‘red tape’ the pile would stand well over the height of a normal person. No single living person has read it all, let alone understand it, and not just because nobody understands 100+ human languages.
The source and origin of this red tape varies as well:
- international treaties
- individual nation-state law
- global treaty-based technical standards
- global (but non-statutory or treaty-based) technical standards
- internal Apple standards and regulations published urbis et orbis
If you start in a Russian-speaking village in the high north at the top of the Urals, you can find a person who will understand people from the next village and how they speak. You can walk Slav-to-Slav along a cline of comprehension across Russia, through Belarus or Ukraine into Poland, down via Slovakia and the Czech Republic, on and on until you find yourself amongst the remnants of Slavic-speaking Turks.
The Macedonians you meet can’t speak to the Ukrainians, nor the Latvians to the Slovenes, but each person can to their neighbours.
So it is across the Apple cline of comprehension: the Polish lawyers can speak to their other lawyers, and the bankers. The app developer can speak to the Apple store administrator.
Excellence in the management of complexity is a social, organisation and operationally cultural condition par excellence.
The protean impulse to ‘deregulate’ comes from a terror of incomprehension — the cut down world is ‘easier to understand’. The problem, of course, is that without the ‘red tape’ the iPhone is substantially a brick.
How does the Scottish public sector embrace complexity and yet deliver for citizens?
The thing that makes the iPhone an exemplar to follow for Scotland is the enormous disconnect between the user and the provider. All the regulation and red tape is for the professionals.
For the end user the iPhone is point, click, call, press and swipe.
The state must understand and organise itself around the excellence of complexity so that the citizen, and the private sector companies here can experience simple and straightforward interactions with the state.
It must be easy to find and understand information and to interact with the state at all levels.
The big challenge implicit in this is the depth of interaction with the state: “it was easy to pay my tax” is not enough.
“It was easy to change to the law to improve the computer systems” — now that is hard.
Every time the Parly passes a piece of administrative legislation a computer system is born or amended.
From that it follows if you wish to eliminate friction caused by poorly structured computerised administrative systems you may have to propose changes to the law.
Render Unto Caesar
The debate about how the state should computerised is both helped and hindered by comparison with the private sector.
It is absolutely right that the state must avail itself of a range of techniques developed in the private sector: lightweight software development processes, usability testing, product management techniques including personae, open APIs and open source software, even competitions and hackathons on public data.
But the processes and methodologies of the private sector don’t translate well.
Modern software techniques are embedded in a methodological matrix that talks about taking an idea, testing it against customers and problems and first seeking product market fit; and when that is found building out and scaling and growing a company, its revenues and users. It is also driven by a culture — a culture of entrepreneurialism, of risk taking and so on.
This methodology fits very poorly with the public sector.
The private sector must find customers whereas the state can and does compel. Many of the problems associated with state activities flow from this compulsion — systems that force companies to implement internal processes to feed them, to hire staff to handle them, to incur costs and delays. This is not, in an of itself, a bad thing: would the world be a better place if chemists didn’t have to maintain a poison register?
Companies seek to create and grow markets, whereas many state activities cope with market failure, or things like mental health that cannot be constructed on a market basis at all.
Companies like Facebook and Twitter compete for attention, stickiness, engagement –but this exactly what we want compulsory services like tax returns not to be. Once and done, bish-bash-bosh in the shortest time is the government’s core goal.
Some core features of modern software culture are wildly destructive when applied to governmental activities — the relentless focus on find-a-niche-and-service-it is antithetical to the notion of equality of citizenship — and legal obligations that the state does not favour some class of people over others.
Another major problem with blindly trying to implement public sector software by transposing private sector mores is the classical liberal notion that the citizen needs to be actively protected from the state, that state actors must be constrained and cannot be given a free license.
This particularly shows itself in the very different feedback loops that can be constructed for the construction and development of private and public sector software.
A company will establish a core financial loop (are people paying more for my service than it costs?) and supplement that with a range of soft metrics that enable that loop to be managed and developed (consumer satisfaction surveys, user engagement measurement, conversion funnels, user groups, usability tests and so on a so forth).
This is a long way from the public sector. Let us take one of the most degenerate cases (although it is a reserved not a devolved matter) income tax.
The structure of the income tax form, the data it collects and the degree of understanding it takes to collect it is largely statutory. What is the feedback loop there? How do I as a citizen impact it? The comparison with new feature in Twitters is stark.
Now take is another degenerate form of public sector software — the near-private sector system. An example would be purchasing from a leisure centre: buying a ticket for swimming, booking the gym or a sauna. The software systems here are not statutory systems. The legislative basis is a general permission to “promote the health of citizens” or some such — but changes to this software does not involve a general election.
A number of other systems (think medical records systems) are hybrids of these two extremes — some features (patient confidentiality) are specified in software, some (clinic codes) are specified by arms length bodies that have technical competence and are empowered by legislation to lay down rules, and some elements (clinic booking systems) are near-private sector.
These feedback loops are shown schematically below:
Excellence in the private sector comes from running through feedback loops at speed, improving the behaviour of computer systems incrementally. The huge and all encompassing challenge is how to measure and improve the operation of these feedback loops in the public sector. The answer is incrementally.
Traditionally the state uses audit and oversight as a way of establishing feedback loops — statutory bodies acting at arms length who provide independent information on the operation of state bodies — think of the National Audit Office.
It it worth thinking of what such organisations would be in the new world:
- National Statistics for computer systems covering information like: number of transactions, the value of transactions to the various parties, government, citizen, companies etc, the cost of executing transactions and so on and so forth
- National Usability: how users find systems, how easy systems are for users, error rates and costs of remediation and so on and so forth
- National Post-Mortem Department: identify why system failures happened, document where and how projects went off track/over budget, make recommendations on best practices to avoid failure and so on and so forth
There are other national organisations that might be expected to exist, for instance a national register of state APIs.
Simply stating what the feedback loops are, and how they must be speeded up will not be enough. Unless there is clear and trusted audit and statistical information there will be no pressure on the political class to act. These bodies are not ‘nice to have’ but very important parts of the public sector ecosystem that we have to create.
The Law And Computer Systems
It is worth looking at the degenerate case of income tax as an exemplar of the ‘long feedback cycle’.
There is a powerful technique in delivering high quality software called ‘paper prototyping’ where pictures of the unwritten software are drawn and usability testing is performed with them on selected customers prior to development or change of the system.
Imagine Scotland becomes independent and we have to develop a new Income Tax system. A Scotland that was a world-leading state could:
- write the first draft of the new tax code
- develop a paper prototype of the main citizen facing tax return
- test that prototype
- pass the feedback to the parliamentarians
- debate and amend the legislation in light of that feedback
- pass the amended legislation
- write and deploy the software
By writing the process out this simply we bring out one of the main hidden roadblocks to a world-class digital state — the misownership of specification of computer systems.
As well as a feedback loop problem, there is a comprehension problem: how does legislation specify computer systems?
It turns out that legislation and computer specifications are cousins.
A law sets out how decisions are to made, who is to make them, the various circumstance that constrain and inform those decisions and the processes by which those decisions are to be reviewed, appealed and enforced.
A computer specification sets out how computers are to make decisions, based upon data entered into them, how that data is validated, the sources of that data and how they are to be authenticated.
Currently the specification of computer systems in legislation is implicit — statements about decisions are made from which the specification of a computer system is deduced prior to its implementation.
As currently written laws are directly justiciable, courts rule on them, but computer systems consequent to laws aren’t, courts rule on outcomes and actions consequent to the operation of a computer system.
The format of legislation has not changed since the introduction of computerised administration, and that change will need to come. The implicit specification will need to be drawn out and done explicitly.
For a given piece of administrative legislation, many clauses would remain unchanged, but a special section pertaining to the computer systems would be added. This section might have a different format and structure, it may be shared between many statues, it might have a different process through the legislator, in the way the supply bills do. Perhaps the computer system clauses have to be signed off by the National Office of Data as correct, or some other quality assurance process.
This new format, these new procedures and ways of working remain to be discovered, implemented, tested and improved.
The broad outline of the can be deduced by reading existing legislation. Legislation largely defines computer systems in terms of the data they store, and the modalities of operations on these data items. These modalities fall into three distinct parts.
The first is simple data management.
Create
this states:
- who can create the data
- how they can create the data
- when they must create the data
- why the data should be created
Read
this states:
- who can read the data
- how they can read the data
- when they must read the data
- why the data should be read
In addition there may be additional statements on the provision of read-only access to data covering topics like:
- the provision of external searchable feeds for use by other state agencies
- mandatory indexing
- and so on and so forth
There is also a requirement to define what the data will be read for, in particular with relationship to its use in management information or intelligence systems or as the basis for reporting
Update
this states:
- who can update the data
- how they can update the data
- when they must update the data
- why the data should be updated
Delete
this states:
- who can delete the data
- how they can delete the data
- when they must delete the data
- why the data should be deleted
These aspects are not-specific to state computer systems, they are fairly generic.
The second set pertain to the problem of the state and compulsion; of justiciable computer systems; of co-ordination between different state organisations,
Definition
the specification includes:
- a single point of definition of the data. This might be: a piece of legislation, a person authorised to issue the data definition by way of tabled regulation eg secondary legislation
Audit
this consists of a statement about the person or body authorised to audit, review and generally oversee the use of the data, including:
- data maintenance activities
- release of data to external bodies
- review, routine deletion and weeding of data
- all aspects of compliance, including: the Human Rights Act, Data Protection Acts, Freedom Of Information Actsm etc, etc
Recourse
this documents:
- the person or body authorised to handle disputes as to the existence or not of a data record
- any due processes required to do so equitably
Partition
this documents:
- how the data is partitioned (eg by local authority)
- what activities should be taken to manage data across partition boundaries
The third mechanism, and on the least under-appreciated and misunderstood one is via the requirement to report.
If a decision is taken that all primary schools must report on the music education they offer their children, then, as if by magic, a host of computer systems are called into life, like dragon’s teeth. Some local authorities will lets schools collate their own, some will set up central systems to do, but systems will spring to life like Spartoi.
There are problems with an over-arching formulation like this. Some of the computer systems simply don’t matter — the hypothetical leisure centre booking system discussed earlier. Legislation should also confer on state bodies the powers of specific competence to specify computer systems for their own needs.
This needs to be caveated though.
The civil service is, and ought to be, a creature of statute, civil servants are simply civilians empowered by law to do very specific and restricted things.
Tech entrepreneurs have a general license to do what the hell they like, within the constraints of the law (and the case of companies like Uber a self-regarded right to operate on the other side of the law).
This can not be the case for public servants — companies have to sell, and require consent to prosper — agents of the state do not.
The proper commissioning agent of state computer systems remains the legislature and the cabinet — every piece of administration legislation, every tweak to the administrative state brings as night follows day a new computer system or changes to an existing one.
But with the collapse of the price of computing, it is the civil servant at the far end, down the management tree, dealing with agricultural payments, or air quality monitoring or traffic measurement who has access to power and resources to do that exceptionally well.
This specification problem also informs the market for computer systems in the public sector. When you look at systems for social work, or other local authority functions the markets for them are typically very illiquid — one or two suppliers, mostly with roots in local authority IT departments. Part of this is from the opacity of specification:
This translation of computer specification from an opaque but public world to a clear but private one distorts the market for specialist systems and dampens the rate of new entrants to the sector, with consequences for speed of innovation. Better, faster and easier tendering must be built — and we know from the Big Bang and the European Single Market what an effect they can have.
Some of this confusion can be simply removed — if a department or sub-organisation is given some specific powers and the freedom to specify its own systems then all appears to be good. There is a problem however, that the reports that a body is expected to make to the government specify its computer systems — freedom to specify implies freedom from fixed reporting requirements.
The language and the mechanisms to do this cannot be that hard to find. Current commercial companies cannot create generalised computing systems, their data management and usage must conform to general law — this regime already exists and can be reused.
Lessons From The Private Sector
There are a some lessons from the private sector that can be usefully learnt at this point.
The most important is the notion of testable hypothesis.
It can be said with 100% certainty that the meta-narrative listed previously is true: the price of computing has fallen, people low down in the hierarchy have free access to immense computing power, the structure of legislation has changed since computerisation.
How ever it can also be said with 100% certainty that the detail listed previously is wrong, at least in part. This paper is based on a series of hypotheses about how computerisation and the state works. These hypotheses might be based on experience in the public and private sectors, they may be informed hypotheses, but they will nonetheless prove to be incorrect, in some way.
The very successful internet start-up community in Edinburgh grew by adopting the formulating and systematic testing of hypothesis about products and markets as core competencies. This testing was embedded in a culture of sharing of experience, of learning from other people in different companies, but the same situation, of steadily and slowing increasing the competence of the whole community.
It is also clear based on observation, that the performance and capability or organisations and communities is not a linear thing. Organisation, software development teams, companies tend to experience phase-like behaviour.
If you heat cold ice up nothing much happens, it gets a bit warmer, and then a bit warmer, it expands very slightly, but not much. Then suddenly it melts.
This pattern of phase change behaviour pervades the physical world: heated liquids suddenly become gases, ketchup changes from a solid to a liquid under stress, crystals precipitate suddenly from solutions etc, etc.
Continuous improvement processes and communication and inter-organisational learning operated the same way in the Edinburgh start-up community. A commitment to incremental improvement on all fronts (lets be 10% better at X next year, for a range of X’s) was rewarded by sudden step changes in the community.
It is said that a unicorn can only be seen by a pure and pious virgin out of the corner of her eye. It turns that software unicorns are the same. Ply your trade, improve your capabilities, learn, learn and learn some more, teach others and suddenly the unicorns will find you.
The goal of making Scotland the best digital state in the world must likewise be approach obliquely, It is the continuous improvement of the things we do that will make it happen, not some Arthurian quest.
The Closing Of An Era
The retreat from a statist economy was very successful in the 1980s, the lifting of capital controls, the deregulation of the city and the European Single Market were all generally successes (the financial bubbles consequent to over-deregulation not withstanding).
All of these were driven by expertise from within business. People who knew the problem domain worked through the issues and were brought in to help inform the political debate, and get politicians out of problems of overreach of their own making.
However, the post-Thatcher era has seen a cargo-cult of supposed private sector competence build up. The theory has been that loser choose public service, the public sector is intrinsically flabby and inefficient and that rigorous disciplines of the private sector and some sort of ‘magically transferable’ managerial competence needed to be transferred in.
That model has not aged well, domain specific knowledge remains required, squeezing state operations into private-company corsets caused a lot of pain to little good.
That era is clearly over. This age of public sector reform needs to be led from within, using the native talents of civil servants, based on actual experience of government, without the cant that public sector workers only respond to the knout and the lash.
The Civil Service And Its Discontents
One way of thinking of this, is that ubiquitous computerisation will involve the biggest change to the operation of the civil service since Northcote-Trevalyan. What is interesting on reading that report is that it makes clear that the modern civil service was not created de novo.
The pub quiz view is that pre-Northcote-Trevalyan the civil service was a mass of idiots-fled-their-village, jobbing placemen, salary-seeking desperadoes and in-laws. Then the competitive civil service exam is introduced, and lo a miracle occurs.
Reform is a caesura: a sudden cleansing of the Augean stables and the emergence of a new ‘Rolls-Royce’ civil service.
The actual history is slightly different.
The old civil service was pregnant with the modern one — an archipelago of excellence become a continent.
That, I think, is where the Scottish civil service stands now — an archipelago of excellence.
Any proposal to transform the state has to start with this raw material, to establish a General Staff of the new world from people with practical experience of all corners of the public sector and to let them guide the development of the new world. Again Northcote and Trevalyan recognised this:
This approach is consonant with the notion that we have to make some hypotheses and then rigorously test them.
Gall’s Law tells us that sufficiently complex working systems emerge from simpler working systems. This suggests an approach to implementing some of the ideas in this paper.
This is the solution to the problem of a political class keen, but scared. Small iterative changes building capability up and up. The failures, which will come, are contained in size, the lessons are learned at the small stages as capabilities build.
First up is to map the archipelago of excellence, what resources exist that we know to be excellent.
From those islands staff up embryonic organisations that we think the new state will need. Here are some examples of organisations and the small beginnings we should grown them from.
National Bureau Of Computer System Statistics — starting from a spread sheet build after a call-around
National Usability Office — starting from a usability tester recruited from an advanced department
National Post-Mortem Department — starting from a collection of historical studies and a project manager capable of running post-mortems on some notable recent failures (CAP payments system?)
National API Register — starting from a spread sheet or github account
Staff Training College with a course on the management and commissioning of state computing systems — starting from some presentations — a 2 day on-boarding course for new team members
The future building blocks must be paradoxically small and puny — to enable confident failure and restructuring — and wildly ambitious with a clear vision of the end goal
These embryo organisations would work as a team to flesh out and test the hypotheses in a thoroughly practical way.
Here are some suggested initial projects, but these are merely suggestions, they could be either/or or both/and or neither/but…
- find a major system that has a poor reputation or usability, or high levels of required remedial work (people put the wrong data in, or not enough). Do a post-mortem review of its implementation and then work up a remedial programme to fix it — a programme with no limits on its scope — that can go back and ask for changes to underlying legislation if need be. Complete a remediation project and report back.
- the blockchain is an emerging technology for managing registers (can be banking registers for money, could be land registers, almost anything that needs to be kept in a list). Have the team work up a complete process for a trial system with small impact in the Scottish public sector using blockchain. This process starts by writing the legislation and introducing it to the floor of the Parly via the Standards, Procedures and Public Appointments Committee. This legislation will have an experimental structure and an experimental path through parliament. Complete a full implementation and report back.
- pick an admired programme from the Estonian government and clone it — from soup to nuts
The key thing to understand about this proposal is the the first step is substantially about building the team who are capable of achieving the transformation of the Scottish public sector. That team needs to grow in capability and confidence first, and then, and only then, in size and scope.
Partners And Buy In
This needs to a project for the whole public sector — building from an archipelago of excellent to a continent. The final output would be a mixture of a Staff College for public sector managers, new oversight bodies, a changed legislative process and, most critically, a methodology of computerising the administrative state — based on a theory of state that is not the warmed-over private-sector-good-public-sector-bad that has passed for analysis for the past quarter century.
This cannot be an ‘elite’ project, and most especially not an ‘outside’ project — the public sector is not in need of rescue, it is merely in need of empowerment and improvement.
If you like this please circulate it on social media and follow me on Twitter
