Is the leadership hype really worth it?

The evolution of a new world of Management
By Dr. Reg. Butterfield
20 minutes read and questions for discussion.

It has been said that leadership is one of the most observed and least understood phenomena on earth. The significant amount of scrutiny about the subject of leadership is not surprising as it is a universal activity identified in both humans and animal species.

From whatever perspective one views leadership, Western and Eastern history is full of examples, stories, and writings about individuals who take charge of a group and lead it, often against the odds, to safety, victory, or prosperity. There seems to be a common societal belief that leadership is vital for effective organisational and societal functioning. Anthropological evidence suggests that there are no known human societies without some form of leadership.

It seems that most research about leadership, and in particular the development and use of leadership in formal organisations, comes from psychological, anthropological, sociological, biological, and zoological perspectives. It is seldom that the “evolutionist” perspective is put forward, yet we argue that this view is of particular importance today. It is also fair to say that there has been little research or discussion about the phenomena known as “followship”, yet it seems highly pertinent in the discussions around leadership and leaders.

Today’s newsletter aims at opening up a debate about what leadership means in traditional, typically hierarchical, organisations today and what may be needed for the emerging New Era Organisations.

In doing so, we will argue that leadership and followership are adaptive reactions for dealing with group challenges. We also posit that leadership and followership are complimentary ways for dealing with such challenges, whilst acknowledging that the interests of followers do not always correspond to those of the leaders; our earlier white paper on the problems associated with strategic human resource management is such an example. Lastly, we argue that the increasing alienation of people working in organisations today is a legacy of the inability of those organisations to introduce and operate with the evolving leadership psychology and methodology needed to resolve the impact of “modern” society on organisations.

Putting things into context

In previous newsletters we have discussed some of the challenges facing organisations today in an environment of changing needs, interests, concerns, and expectations of their employees. Such as:

‘The big quit’ 8 May 2022;

‘To build for the future, first understand the alphabet’ 15 May 2002;

‘Management Taboos — dealing with symptoms vs managing the problem’ 22 May 2022;

‘Are you innovating or stuck in the mud?’ 29 May 2022;

‘Knowledge workers’ 12 June 2022;

‘The complexity of work’ 19 June 2022; ‘Understanding leadership and management’ 7 August 2022; and

‘The quiet quitters’ 14 August 2022.

Whilst not a rigorous piece of research, a recent survey of 1,300 ‘team leader’ respondents by US-based company “Resume Builder” (a resource provider for job finders) proffered that 74% believe that Generation Z is more difficult to work with than any other generation; citing that they lack motivation, and they are easily offended. Currently, Gen Z represents just 5% of the workforce and this is anticipated to increase to 27% by 2025. We should not be surprised at this observation about Generation Z because organisations are already changing what work means through their experience of having Millennials in the workplace. Millennials value interesting work and a good work–life balance. They do not believe that excessive work demands are worth sacrifices in their personal lives. Millennials want flexibility in their working hours and are willing to give up pay increases and promotions for a flexible working schedule. The impact of Covid on workplace behaviour has only served to highlight and reinforce that meeting the “demands” of recent generations is a challenging process for organisations. It is not a new phenomenon, history clearly shows us that as society changes, so does the notion of what leadership is and means.

Each generation has different expectations and work ethics when compared to previous generations, nothing new here. So, what makes the impact of the millennials and emerging generation Z something to be concerned about? To help make sense of this, we first need to understand the reason for leadership.

Leadership through the lens of Evolution

Any student of leadership knows that whilst the term leader was conceptualised before biblical times and noted in the 1300s, it wasn’t until the 1700s that the concept and term of leadership emerged explicitly.

The work of Eben Mumford on the evolution and origins of leadership was published as a series in various formats as it emerged sometime in the early 1900s and has been discussed and reviewed by countless researchers, and psychologists in particular. His original work is as detailed as it is fascinating for those of us who are trying to make sense of the difficulties associated with leadership in the current century. Whilst the style of reporting can be a challenge, the content cannot be ignored.

An important conclusion in one of Mumford’s essays, which were apparently in a book that seems to be illusive today, he is quoted as writing, “All social changes, whether a progressive or regressive character, originate in stimuli, creating tensions in the social process and demanding adaptive activities. In these adjustive processes the leader finds his chief function. In the adaptive processes there are various degrees of failure and success, but if the group is to survive, the successful activities must predominate.”

We suggest that this is important because as societies and organisations evolved, we argue that so did the development of leadership as a subject and associated behaviour that tried to cope with the changes. We explore this statement with a brief history of leadership.

It can be no surprise that during the 1840s the “Great Man” heroic theory of leadership emerged as famine and revolution were happening around the world and society sought to identify the rare creature(s) who could bring about change and peace. As such mortals were rarely found the idea of “leadership traits” took over during the 1930s to 1940s in order to find the relevant traits and train leaders in the light of what was needed. Unfortunately, the perfect traits were never found, and the search was abandoned. With the background of world war, the problem of a distinct lack of leaders emerged as attrition was greater than sufficient people for assignment and as such the 1940s — 1950s saw the traits morph into “behavioural theory”; asserting that leaders are largely made, rather than born and that particular behaviours can be learnt to ensure effective leadership. Unfortunately, whilst this approach largely ignores the situation and environment of the leader, this view of leadership is still trained to this day using patterns of behaviour grouped together and called leadership styles.

During the 1960s it was realised that the environment has a direct influence and impact on the dynamic of leader and follower. The theories of “contingent and situational leadership” took pride of place as people endeavoured to adopt the best style to fit the circumstances. This was an important view given the plethora of differing world events at the time: the first spacewalk, the cultural revolution in China, Kennedy’s assassination, the civil rights act, the erection of the Berlin Wall, and more. The social scene was one of experimenting with fashion, music, and the beginning of sexual liberation for all forms of behaviour and gender. It was clear that one approach to leadership was not possible if success was to be achieved across different fronts.

The 1990s through to today have introduced the clear recognition that focusing on just one aspect or dimension of leadership cannot address all the complexities of the phenomenon called leadership. As the world became more complex and challenging, new theories emerged. In a world of rapid change, disruptive technological innovation, and stronger forms of globalisation it was necessary to redefine leadership and break away from being unidirectional top-down power systems that distinguish between leaders and followers.

What has emerged is a complex interaction among the leader, the followers, the situation, and the system as a whole; the latent leadership qualities and capacity of the followers were being recognised, if not truly used. This approach has become known as “transactional, transformational theories”. In recent times a wide range of other theories have emerged under the umbrella of shared, collective, and collaborative practices. More about this later.

All the attempts to define and practice leadership have something in common, they were all developed to try and bring order to the role of leadership at a given moment of history. Not one single theory or form of leadership alone has proven suitable as the complexity of organisations and society has changed over time.

We suggest that the development of leadership theory and practice can be seen as a result of the evolution of society.

When organisations are looking for leaders today, it seems that they still focus to a large extent on the early idea of collective leadership traits. This preference remains strong even though research and operational evidence indicate there is no such thing and that other models have tried to address its inherent defects, which we discussed briefly above.

For evidence of this stubbornness, think about how the media and the public attribute traits to people whom they believe are or were great leaders. You read or hear them describe the personality traits of business and other leaders like Bill Gates, Mary Barra, Steve Jobs, Elon Musk, Gioria Meloni, Rosa Parks, and others: they make tough decisions, they are innovative, they are motivators, they demonstrate presence, they stand firm during tough times, and other distinguishable personality traits. To this day, research continues to find the optimal traits of a leader, arguably through selection bias that ignores the situation and support systems that enabled their “leadership success”.

There has always been a plethora of diverse expectations as to what leadership means within organisations. As the new era of work and what that means unwraps almost a quarter of the way into this century there seems to be even greater confusion and fragmentation of opinions also outside of the organisations. As a result, the role of leadership is becoming more complex and political, which makes it even more important to understand what it means and where it lies in organisations today and in the future.

Do we really need role models of traits or other attributes of famous leaders, or do we need something else to ensure that organisations today and in the future are successful? Where does the all but forgotten role of managers fit into the equation?

The relationship of leadership and management

Management is not new. Its concept has been around for thousands of years. For example, Socrates stated that management was a competency distinctly separate from possessing technical skills and knowledge. The Romans are legendary for their use of hierarchy, authority, and accountability whilst the Roman Catholic Church was organised along the lines of specific territories, chain of command, and job descriptions. Guilds were prevalent during the Middle Ages and followed similar approaches to management as those used by the Catholic Church.

If we fast forward to the industrial revolution (1700s — 1800s) and the subsequent influence of people like Henry Ford on the nature of work and its control, a need for optimisation of work emerged. In comes the infamous Frederick Taylor with his approaches to the nature of work and how it should be undertaken. Beyond Taylor, other management theorists came onto the scene and people such as Frank and Lilian Gilbreth (a genius in the art of living), Harrington Emerson, and others expanded the concept of management reasoning with the goal of efficiency and consistency, all in the name of optimising output.

As organisations and the work became more complex it was necessary to move beyond just worker output to that of ensuring the whole organisation was linked towards a common objective. History of management credits the Frenchman Henri Fayol as being the ‘father’ of the management concepts of planning, organising, coordination, command, and control. These have led to the four basic management principles used today: planning, organising, leading, and controlling.

Around the end of the 19th Century “welfare officers” were introduced to protect women and children from the harshness of the existing working methods and eventually led to the function known as personnel management, the forerunner of the human resource function of today. It was seen purposely as a distinct form of management and separate from the daily roles of managing the workforce. This is an important distinction because we argue it sowed the seeds for many of the difficulties encountered by workers today. The separation of the ‘care of people’ from the ‘use of people’ to achieve and meet the need for business outcomes. People became ‘human resources’ used to create profits and shareholder value as if part of a machine with replaceable parts.

To reinforce this need to ensure the whole organisation was linked towards a common objective, subsequent performance management systems have been designed (to oil the parts) on this premise and have resulted in worker compliance as opposed to motivated people.

Two basic forms of management have emerged over time to ensure the complexity of work and organisation needs are met, these are leaders and managers. Historically, leaders were the visionary, inspirational, direction setting, and in many cases the entrepreneurs. The leaders set the direction of the organisation and the rest of the organisation worked in that direction and delivered its anticipated outcomes.

If we look at the original English meaning of the words, to manage means to handle, which implies a high degree of direct involvement, whereas to lead means to go before and prepare the way. Leaders did not have daily contact with the workers, their managers did.

Obsession with leadership

It is hard to pinpoint the day when somebody said, “we need more leaders in organisations”. Yet, since that time, right up to today, an unhealthy obsession with the notion of leadership has become de rigueur. The demolition of hierarchies within traditional “command and control” organisations has seen the rapid demise of managers and an increase in new titles for executives. The managing director is now the chief executive with a host of other executives for finance, strategy, HR, and others, each being seen as a ‘territorial’ leader in their own right. Their emphasis being on vision, strategy, and communication, without the challenge of getting things done; managers used to do that. The few “managers” that are left tend to be the experts of that work domain and as such have two roles ‘manager’ and ‘expert’, with the expert taking up the time and energy in a world of insufficient knowledge workers and rampant skills shortages.

So, what has happened to getting things done that the managers used to do? You do not have to go far to find the answer.

The chaos at airports, health services, armed forces, government offices, mobile phone service providers, logistics companies, and more show that if nobody is managing situations the leader’s vision and fanciful ideas amount to little or nothing.

Covid is an excellent example of how much management skills and experience were needed to ensure the survival of the organisation. In our earlier newsletters we discussed the Big Quit and Quiet Quitters, which are excellent examples of no matter how good the leaders are with their visions, etc., it is the need for managers to create work environments and other support that people need to feel valued and motivated to work. The number and types of managerial experience are just not there anymore. Interestingly, the task of coping with Covid seemed to rely on the HR people as the ‘leaders’ had little idea of how to cope with such circumstances and the ‘technical expert’ hybrid managers were too busy trying to cope with their other day job.

In this organisational management vacuum, the purveyors of leadership work hard to sell their wares and demonstrate new forms of leadership based on social skills and awareness as a means to fill the void of not enough managers. Surely this is missing the point as the situation requires more than soft-centred visionaries with little or no experience of what matters to their people. After all, shareholders want their leaders to capitalise on generating income, profits, and shareholder value, not sit with an unhappy worker who feels stressed out.

The changing nature of what work means

Work today, and increasingly the case for the future, is much more than the traditional skills and methods of yesteryear. It is about people working in teams with “technical co-workers” such as automation, robotics, and supporting technology such as that associated with AI; it is a world way beyond the ideals of leaders with soft skills. It is about understanding the need to create and support an appropriate environment that encourages skilled people to join the organisation and remain because they are valued and enjoy what they do. If you can find one, you will see that this is what good managers have done for decades.

Before getting too excited about this world of techno-human fit and the roles of management and leadership, it is important to remember that evolutionary history demonstrates that the solutions today are only as good as the stability of the environment in which the solution was developed. If the external environment changes, then it is likely that the approach to management and leadership will also be expected to change unless the whole notion of what work means is reconsidered.

The ‘power’ of the organisational design

The one thing that has remained pretty constant throughout this journey has been the nature of the organisational design within which the leadership operates. With few exceptions, it has taken the shape of a hierarchical bureaucracy with power and control as its engine driving both society and different forms of organisational operations; even though the levels of hierarchy may have flattened in recent years.

It now seems appropriate to return to the work of Eben Mumford who wrote, “The leadership function must precede the leadership structure”. If we follow this with another of his quotes, “One of the most important generalizations to be gained from the examination of the data … is that every individual that attains a position of leadership in the group must do so by the performance of some function which the group considers is of importance.”, we can only identify the roles of leadership and management once the needs of the workforce are identified and understood.

However, we argue that combining the above needs, to identify what the ‘group’ considers important, together with what the leader’s function is, has a missing element, that of the operating environment. It is not surprising that this is missing from the plethora of work around the concept of leadership because until now the role of leadership has been to reinforce the operational power structures that have been taken for granted by generations of leaders. Equally, the leadership within society reflects or drives the structural model of that society. Doubters only have to look at the geopolitics of the US, Russia, and China today to make sense of this statement.

The paradox for us is that the leadership models have changed in response to difficulties of operation in emerging environments and yet the structure of organisations and their ways of working have been largely ignored, even though the problems of bureaucracy have been well known for centuries.

Moving away from Control

In organisational terms, what happens if we re-focus the effort from power and control to that of ensuring that the work itself is supported and brought to fruition? We will return to this concept later as we suggest that it offers a key to opening up a whole new world of what leader and follower can mean for the future.

Form follows Function.

The architect Louis Sullivan coined this maxim, which resumes the famous French architect Viollet-le-Duc’s theories (1814–1879): a rationally designed structure may not necessarily be beautiful, but no building can be beautiful that does not have a rationally designed structure.

We hesitated in using this architectural example because many readers will have experienced the opposite when trying to live in premises that look beautiful and yet are not very functional; the same often goes for hotels, we suggest. Yet, this negative perspective serves to reinforce that form must follow function if organisations are to be successful in harnessing the immense capabilities that the techno-human world can provide.

So often, attempts to ‘solve’ the so-called ‘leadership dilemma’ have led to re-organising the processes of leadership, together with introducing new leadership traits, as a means of dealing with what we recognise as symptoms created by existing structural problems. These structural problems lead to human response behaviours, which in themselves create other issues to be resolved. The multitude of issues that organisations suffer from as a result of traditional structures are well documented and we will not repeat them here. However, just changing the structure does not solve the problem as it is just one part of the jigsaw of organisational behaviour. This is often experienced as a whack-a-mole game as organisations move through a never-ending sequence of reorganisations.

The jigsaw of organisational life

It is easy to criticise the efforts of enthusiastic people trying to solve the problems of organisational life. My colleagues and I have been guilty of such criticism, albeit we have endeavoured to do so in a constructive way to learn from those approaches.

Whilst the traditional approach to running organisations has many drawbacks, it has been pretty successful to date with some tinkering around the engines. To really bring about change to such machines means taking on the challenge of going back to first principles of organisational life and work, which is a daunting process. Figure 1 is an example of the areas of impact that need to be considered in such an endeavour.

Figure 1: Impact analysis framework.

However, changes in the societal environment and associated technologies are arguably changing at an unrelenting pace not experienced within living memory. The evidence is piling up and the messages are clear. The stress and associated human casualties seem to be increasing in so many major ways that it is now time to take on the challenge of going back to the drawing board. In doing so, ask the questions that help meet the criteria set by understanding the work of Eben Mumford discussed earlier.

These questions form the foundation of the debate about ‘form follows function’ and management follows the identified needs of the people undertaking the work.

It is the view of my colleagues Dr Ross Wirth, Silvia Calleja, and me that it is time for a totally different view of what work means and how that work will be undertaken and managed.

In summary

In this discussion we have examined both leadership and management from a historical perspective. In doing so, we have demonstrated that the different approaches to leadership have been in response to the changes and expectations of society over time, with limited degrees of success. This limited success is visible in the lack of finding any one single successful approach to leadership, albeit there has been no lack of searching for a solution.

This need for a universal form of leadership has spawned a global industry of consultant, trainers, academics, commentators and more who serve up their own variation of common leadership methods; virtually nothing is new and much of which borders on snake oil when truly scrutinised.

We commented that during this search for the holy grail of leadership, little or no account was taken of the organisation’s operating environment. Whilst changing the operating environment itself may not ‘solve’ the problems around leadership, it does help to make sense of the reason some approaches fail.

The one consistent factor during this search was almost all organisations shared a common structure and philosophy, that of a hierarchical often scalar form with top-down authority and power, which may account for some of the difficulties encountered.

During the last two or three decades many organisations opted for a flattened structure, which typically meant removing levels and numbers of managers. The vacuum that these managers left was either never filled or those who were left were encouraged to take on the mantel of expert, manager, and leader. This has led to poor performance at best and chaos at worst. We provided evidence of the impact of this lack of people managing the organisation and that Covid served to reinforce this impact.

It seems ironic that focusing on leadership traits has demonstrably failed in the past and yet it is probably the most common approach used to develop and select leaders today.

We argue that this unholy focus on leadership is creating a situation that is inevitably going to cause organisations to fail. What is needed is just one leader to decide and promulgate the purpose, direction, strategy, and goals of an organisation. To achieve these goals and align people with the purpose is what managers have done throughout history, yet there are few left to do this today.

An important debate about what leadership and management means

Over the last year we have published a range of organisation-related subjects in our weekly newsletter. The majority relate to our interpretation of the future of organisations, which we call Futocracy. It is the result of years of research. It is a work-based approach as opposed to the traditional power and control systems that predominate organisations currently.

Work is being undertaken by an important relationship between technology, people, AI, and robotisation. This newsletter argues the case for a different understanding of what leadership means today and the form of management needed for success in meeting the changing needs within a diverse working environment.

Our questions to readers are:

1. Do you agree that organisations only need one leader and other forms of organising the people who focus on getting the work done?

2. What form of leadership and management do you suggest organisations should instal to be successful in the emerging new era of work, whether they are traditional in design or otherwise?

Academic references for this newsletter are available on request.