The Problem of Incommensurability

Post-Occupancy Evaluation in Retrofit

Introduction

In this ruthless era of optimisation and efficiency, feedback has permeated even the most archaic industries. All but the construction industry, which remains steadfast in its Luddism. Despite a growing urgency to utilise feedback to tackle the impending climate crisis, the only ‘loop’ occurring is the circular offloading of responsibility to perform it. ‘Stage M — Feedback’ initially introduced in the 1963 RIBA plan of work, amounted only to a spectre that still haunts the architectural profession. While issues endemic to the construction industry present barriers to the widespread implementation of feedback, its current incarnation in the form of ‘post-occupancy evaluation’ presents more fundamental problems. The incommensurable values of building design question whether the very notion of feedback is itself an engine of a technocratic regime, which assumes that like everything, buildings are a system. Using domestic retrofit as a vehicle to explore its nuanced flaws, this text advocates for more socially oriented and future-facing methods of POE.

1. Historical Overview of POE

There is no clear consensus on the exact scope and definition of POE. This lack of clarity is a major hurdle if its adoption is to become widespread. Nascent in his 1998 text, Preiser defines it as, ‘process of systematic data collection, analysis, and comparison with explicitly stated performance criteria pertaining to occupied built environments’[1]. While Bordass gives a simpler explanation, ‘learning from what you are doing or from what you and others have done to understand where you are and to inform and improve what you are about to do.’[2] POE covers a broad spectrum of feedback, and there appears to be a conflict between the hard, empirical data gathering of aspects like energy performance, and the attempt to skew subjective, qualitative feedback into quantitative data.

RIBA included ‘Stage M — Feedback’ in their 1963 plan of work but removed it only 9 years later. While Stage M was initially ‘far sighted and optimistic’, Frank Duffy believes its failure was due to its vague definition and economic ambiguity. ‘Feedback’ failed to distinguish evaluating building procurement and contracts, from occupancy and client satisfaction. This original inclusion of feedback also failed to establish an inclusion of feedback costs in architects fees, leaving the costly procedure entirely in the hands of the client.[3] . Clients appeared hesitant to pay for feedback services, and RIBA removed it so as not to imply architects provide it for free.[4]

It is contested that POE has roots in much earlier practices. Schoenefeldt details in depth the lengthy processes of feedback in 19th Century buildings, including the Houses of Parliament, “managed by a superintending engineer, also referred to as the ‘Officer in charge of the warming and ventilating arrangements’ and separate attendants were employed by each House to supervise the operation of the complex systems in the two debating chambers.”[5] Similarly, informal practices that resemble POE have been carried out for centuries — apprenticeship of crafts and knowledge sharing through guilds and anecdotal feedback. In recent years, calls to systematize these activities empirically, allied with a new focus on energy performance have been repackaged as post-occupancy evaluation.

1. Why Stage M Failed — Barriers to POE

We can clearly see a long history of intent towards POE — at first implemented rigorously only on buildings of great cultural and functional value, and then endorsed, albeit briefly, by the RIBA plan of work in 1963. But today the uptake of POE, particularly by architects, remains incredibly low — around 3%. Deep rooted issues in the construction industry culture, an inertia from both RIBA and the government, and lack of a shared understanding of POE are all constricting factors.

The largest barrier to POE is cost, it is an inherently expensive procedure to carry out and seen by many as optional. Naturally, this leads to questions of who pays for POE, and who then owns the knowledge or data that it bears? The Egan report of 1998 openly called for client led POE, but much of the construction industry’s client base relies on maximising short-term profits, to which POE is only a burden. So, we remain in the stalemate of the 1960s with neither clients or architects willing to fund POE. Although, in their interviews with architects, Hay et al. discovered that some large practices see a long-term benefit in performing their own POE and are able to absorb the costs, while many smaller practices seek to partner with academic institutions to pool funding.[6] It seems that in the eyes of the consumer, POE remains a blue-sky research project, rather than a practical tool of immediate benefit.

In a similar dilemma, there are fears that negative findings in POE will open doors to litigation and accountability. This is reflected in insurers’ uneasy relationship with POE and its effect on Professional Indemnity Insurance. RIBA recommends that practices undertaking POE inform the RIBA Insurance Agency of their intention to do so.[7] The construction industry is already notoriously litigious, if POE brings up failures beyond completion, it opens more potential litigation. However, POE has the potential to reduce risk in the long term.[8]

As Frank Duffy notes, architecture is organised horizontally — architects are supposed to have a great breadth of knowledge, but little depth. In RIBA’s document ‘Pathways to POE’, they state that “POE research should be focused on the practice specialism.”[9] POE favours architects who work on similar typologies. Pouring resources into evaluating a housing scheme is not directly beneficial if a practice’s next project is a school. Similarly, the vast majority of client-led POE appears to be undertaken by clients who have a long-term stake in a specific building type — such as supermarket chains. Currently, there is no clear incentive for clients who work on diverse or one-off projects, to undertake POE.

1. The Siren Call of Retrofit — Exposing POE’s Flaws

Retrofit seems like a perfect testing bed for POE, its primary concerns are energy performance and user comfort. As homeowners tend to be both the client and occupant for retrofit, they are invested in the long-term success of the project in both occupancy and exchange value. The UK government’s ambition to retrofit 70% of existing homes is a vast but typologically specialised undertaking, allowing feedback to be easily reincorporated on similar projects. While retrofit may seem a natural environment for POE to thrive, it continues to struggle.

Retrofit involves traditional building work as well as the installation of complex standalone technologies. Naturally, these product-like technologies tend to progress much faster than building techniques,[10] leading to a disjunct. The management of these technologies within a larger building system, entangled with occupant social relations is a challenging task. Equally, there is risk of ‘data decay’ in POE, as the latency period of architectural feedback is so long, due to the building timescale. This is amplified if there is a gap between projects, where the project may now be concerned with entirely different materials, systems, methods, rendering the previous POE largely redundant before it can be incorporated into a new design.

This complexity naturally draws POE towards clear metrics, leading to an over reliance on easily measurable conditions. Isolated metrics are inherently reductive and can be misleading. Thermal comfort has become synonymous with air temperature. Only around 2% of body heat loss is by convection to the air, while around 65% is lost through radiation to surfaces.[11] Air temperature is far easier to measure than the body’s radiation into surfaces, leading to an incorrect conflation between air temperature and thermal comfort. Pender and Lemeux argue that the historic ‘interior clothing’ of buildings with soft materials, was fairly successful in mitigating radiation heat loss, our obsession with air temperature is misguided.[12]

In ‘A Socio-Technical Approach to POE’, the authors note that even in very similar retrofits, useful POE is hard to achieve due to ‘quirks’ in seemingly similar buildings, and more so in behaviour of inhabitants.[13] Questioning of current views on comfort and how it is measured are echoed by Cole et al. who call for a redefinition of ‘comfort’ — previously based on passive physiological models, to a dynamic, adaptive definition, concerning social relations, communication and dialogue.[14] ‘Comfort is seen as something people achieve through performance of a web of other household practices (‘doing’ and ‘saying’) such as cooking, washing, cleaning, that differ culturally, spatially and temporally’[15] Not only are buildings diverse, but people are also. Unlike public, commercial and office settings, domestic occupants have complete control over their home and its environment, possibly feeling more comfortable to behave in ways that counteract ‘rational’ expectations or energy saving principles. Occupants can override mechanical ventilation by opening windows, sacrificing performance for personal preference.

Gupta and Chandwala suggest that failing to analyse the behaviour of occupants in their existing homes (pre-retrofit) can lead to incorrect suggestions of a design-built performance gap. Pursuing the possibility that occupant behaviour is the driving factor in energy consumption, collecting data on how occupants behave in their existing home can help create more realistic projections of energy use in the retrofit, or even change design decisions.[16] This approach suggests that buildings do not underperform their design, but assumptions in model predictions are wrong. Although ideal, there is seldom much time available to complete ‘pre-refurbishment’ evaluation, and additional expense is unattractive, particularly when post-occupancy evaluation is already neglected due to its cost. Similarly, Santin has discovered that the ‘rebound effect’ is particularly pronounced in retrofit. As retrofit and building technologies increase the energy efficiency of home, occupants tend to increase their consumption as a result.[17] Both ‘pre-furbishment’ evaluation and the rebound effect, show how current methodologies of POE are failing to understand social forces behind energy performance. Removing quantitative data of all context can distort results.

POE lends itself to universality, but buildings do not exist in vacuums. The myriad attempts to create universal standards, tools, metrics for POE have largely failed. Even on similar buildings, the standardised PROBE methodology used on all 80 ‘Retrofit for the Future’ buildings produced vastly different results based on which contractors carried out the POE.[18] Building remains as one of the few human outputs that has not been comprehensively standardized. Almost all retrofits are conducted on a project-by-project basis. The heterogeneity of the built landscape makes POE difficult, as its success largely relies on benchmarking and the comparison of shared metrics.

Even when collecting data on straightforward metrics, raw data alone is not helpful. Data needs to be managed, translated into knowledge, and implemented effectively. A criticism of POE is that revisiting a building once during its lifetime will not give an accurate picture of its performance, or how it changes over time. Attempts to automate aspects of evaluation are being developed, but most clients and practices will lack the resources of the Boston Dynamics-Foster and Partners collaboration that employs a $70,000 robot dog to constantly monitor an office building, through and after construction. The data gathered is fed into a ‘4D digital twin’ of the building, that tracks changes and performance over time. The future automation of feedback could prove to be vital in scaling the collection of quantitative performance data, but robot dogs will not resolve the lack of social context and may even exacerbate this severance.

1. Existence or Success?

The cultural issues of the construction industry can likely be overcome to foster widespread adoption of POE, through institutional or governmental regulation, inclusion within contracts, or profit sharing-incentives. However, for feedback to be successful rather than just exist, it needs fundamental changes. Many have lauded the ability of POE to highlight the design-built performance gap but failed to identify how POE can close the gap. Similarly, it may be the exclusion of occupant behaviour from current POE methods that is artificially inflating incorrect correlations. Clearly, POE can highlight issues, but not so obvious is how these issues can be solved.

In an ouroboric dilemma, POE is most useful before the project it evaluates has been built. POE must become more future facing; its current retrospective nature does not help assuage the rapidly changing requirements for buildings. Even the most rigorous POE could not have foreseen a global pandemic that has mercilessly transformed the usage of home and office space. Perhaps POE can not only focus on the success of presently completed buildings, but on the failures of those they replace. A suggestion to orient POE forwards is to not only conduct it after a building’s completion, but to offer it early in the design process.[19] Evaluating existing buildings to inform a current design gives a clear incentive for a client to fund the process, as it offers immediate value to an ongoing project. This immediacy also prevents knowledge learned from POE from becoming dated.

Buildings are complex, and unlike simple systems, often juggle incommensurable values. The desire for natural light is in conflict with the need to reduce solar gain. The human desire to feel sunlight on one’s face cannot be expressed as a G-value, making the designer’s decision to balance these concerns that have no common measurement, largely subjective and not conducive to empirical feedback. The requirement for flexibility conflicts with a need for specificity. In the PROBE studies by Bordass and Leaman, POE was able to highlight that flexibility in buildings is often a trade off with function or usability.[20] Again, reinforcing the notion that feedback often highlights conflicts in buildings, that impede the efficacy of feedback itself.

For years, advocates of POE have argued that, ‘buildings should no longer be seen as practically complete when they were physically complete.’[21] The world is changing ever more rapidly, obsolescence sets in faster, the needs of building occupants evolve far quicker. On the surface, it appears logical that the design team should remain monitoring and tweaking. Historically seen as idealistic, today these extended involvements have become normalised in other areas of life — we expect constant software updates, hardware upgrades, and problematically our online activity is monitored to ‘improve’ user experience. While the ‘mereological’ nature of our increasingly digitised existence has led to a focus on systems, the reading of buildings as systems is not a truism. While operations such as lighting and heating can be isolated to systems, buildings as a whole and the incommensurable conditions of their use are not so clear in their ontology. While feedback can easily be applied to a closed system such as heating in isolation, its application within an entire building is not so straightforward. Most POE is conducted as objectively as possible by a party outside of the occupancy. Possibly, this outside-in approach isolates and decontextualises the results. While ‘walkthrough’ surveys help somewhat to break this, perhaps POE can go one step further — domestic retrofit provides a perfect setting for occupants to conduct their own POE alongside an outside contractor or architect. This dialectical approach could help to marry a reliable empirical method with one that better reveals and understands occupant behaviour. A platform such as RetroForm that guides occupants to complete their own POE could be a starting point.

Conclusion

Despite increasing urgency for feedback to aid in tackling the climate crisis, POE still faces myriad barriers to smooth implementation. The legacy of Stage M’s failure is still felt — POE remains vague, its definition and methodology fragmented. While central powers have called for POE to be client-led, its cost is seen as counteractive to the short-term profit seeking and blame culture that plagues the construction industry. There remains conflict over how POE can become fully embedded, with architects calling on RIBA, and RIBA calling on the government, while government funded Egan and Latham Reports advocate fully consumer-led POE.

Analysing the use of POE in domestic retrofit has shown it neglects the social forces that lie behind energy performance, and grapples with conflicts arising from an intrinsic incommensurability between design values. With the adoption of POE so low, much focus has been on encouraging wider adoption of feedback, rather than interrogating how it can be successful. Difference in both occupant behaviours and buildings negate attempts to standardise feedback and the underlying universality of POE. The mere performance of POE will leave raw data and highlight issues, but the narrow methods and backwards-looking nature can lead to incorrect correlations. Equally, is feedback as we know it, simply a symptom of a paradigmatic isomorphism, that sees us equate everything in terms of the dominant technology of our time? Is viewing building-occupant relationships as a system fundamentally flawed? A more holistic approach to feedback that spans over a wider timescale can help POE contextualise energy performance in the social relations that drive human behaviour, both must be evaluated as intertwined. If the inherent incommensurability in building use and design is to be overcome, a dialectical method can be applied to POE that slowly works to a conclusion from the starting point of two opposing views — occupant and designer, empirical and subjective.

[1] Wolfgang F. E. Preiser, Harvey Z. Rabinowitz, and Edward T. White, Post-Occupancy Evaluation (Van Nostrand Reinhold, 1988).

[2] Bill Bordass and Adrian Leaman, ‘Occupancy — Post-Occupancy Evaluation’, Assessing Building Performance. Elsevier Butterworth-Heinemann, 2005, 72–79.

[3] Francis Duffy and Les Hutton, Architectural Knowledge: The Idea of a Profession (Taylor & Francis, 2004).

[4] Bill Bordass and Adrian Leaman, ‘Making Feedback and Post-Occupancy Evaluation Routine 1: A Portfolio of Feedback Techniques’, Building Research & Information, 33 (2005), 347–52

[5] Henrik Schoenefeldt, ‘The House of Commons: A Precedent for Post-Occupancy Evaluation’, Building Research & Information, 47.6 (2019), 635–65

[6] Rowena Hay and others, ‘Post-Occupancy Evaluation in Architecture: Experiences and Perspectives from UK Practice’, Building Research & Information, 46 (2017), 1–13

[7] ‘RIBA Post Occupancy Evaluation and Building Performance Evaluation Primer’ <https://www.architecture.com/knowledge-and-resources/resources-landing-page/post-occupancy-evaluation> [accessed 8 January 2021].

[8] Hay and others (2017)

[9] RIBA and Hay, R., S. Bradbury, D. Dixon, K. Martindale, F. Samuel, A.Tait (2016), Pathways to POE, Value of Architects, University of Reading, RIBA

[10] Lai Fong Chiu and others, ‘A Socio-Technical Approach to Post-Occupancy Evaluation: Interactive Adaptability in Domestic Retrofit’, Building Research & Information, 42.5 (2014), 574–90

[11] Robyn Pender and Daniel Lemieux, ‘The Road Not Taken: Building Physics, and Returning to First Principles in Sustainable Design’, Atmosphere, 11 (2020), 620

[12] Pender and Lemieux, (2020)

[13] Lai Fong Chiu and others (2014)

[14] Raymond J. Cole and others, ‘Re-Contextualizing the Notion of Comfort’, Building Research & Information, 36.4 (2008), 323–36

[15] Lai Fong Chiu and others (2014)

[16] Rajat Gupta and Smita Chandiwala, ‘Understanding Occupants: Feedback Techniques for Large-Scale Low-Carbon Domestic Refurbishments’, Building Research & Information, 38.5 (2010), 530–48

[17] O. Guerra Santin, ‘Occupant Behaviour in Energy Efficient Dwellings: Evidence of a Rebound Effect’, Journal of Housing and the Built Environment, 28.2 (2013), 311–27

[18] Lai Fong Chiu and others, ‘A Socio-Technical Approach to Post-Occupancy Evaluation: Interactive Adaptability in Domestic Retrofit’, Building Research & Information, 42.5 (2014), 574–90

[19] Bill Bordass and Adrian Leaman, ‘Making Feedback and Post-Occupancy Evaluation Routine 1: A Portfolio of Feedback Techniques’, Building Research & Information, 33 (2005), 347–52

[20] Adrian Leaman and Bill Bordass, ‘Flexibility and Adaptability’ (Taylor & Francis, 2004).

[21] Bill Bordass and Adrian Leaman, ‘Making Feedback and Post-Occupancy Evaluation Routine 1: A Portfolio of Feedback Techniques’, Building Research & Information, 33 (2005), 347–52