Agile: Transformational and Charismatic Leadership

11 min readSep 1, 2019

In my previous post, I picked up on the finding from the 2017 State of DevOps report on the importance of Transformational Leadership and focused on the contrast between transactional and transformational leadership and similar ideas that run through organisation theory. However, a second line of thought was in my mind too: transformational leadership is closely related to charismatic leadership. That’s the focus for this post, starting with looking at what transformational leadership is and how it ties to charisma and then digging deeper into charisma itself.

Transformational leadership is characterised by four dimensions:

Idealised Influence — acting as a role model for followers
Inspirational Motivation — presenting a vision that inspires followers
Individualised Consideration — genuine concern for the needs and feelings of followers
Intellectual Stimulation — challenging followers to be innovative and creative

The first two — Idealised influence and inspirational motivation — build upon the leader’s personality: charisma. Involving Charisma triggers a number of thoughts around the nature and role of charisma, and asking if Agile transformation requires charismatic leadership. To answer those questions I’ll start by digging in to the idea of charisma.

Charisma

Charisma is originally a religious term for divinely conferred grace on an individual that inspired religious followers; the term evolved over time and became popular in Germany in the late 19th century where the sociologist Max Weber adapted the term to its modern, secular definition to mean a form of authority associated with the behaviours and characteristics of the leader.

But to fully understand what Weber meant we need to understand the context in which he was using it. Weber identified three “ideal types” of authority, “charismatic”, “traditional”, and “rational-legal”.

Weber’s Types of Authority (adapted from Wikipedia)

Ideal types are analytical constructs: they are unlikely to exist in reality but are a useful lens for viewing and analysing authority and organisations. Authority in any real organisation is likely to have aspects of all three types. If we start with scrum as an example of Agile, the proponents of scrum may well argue that it rests on rational-legal authority (it’s the logical, rational way to operate); in practice, scrum is often adopted as traditional authority (this is what the scrum guide says); the State of DevOps report found that charismatic authority, in the form of transformational leadership, was present in high-performing IT operations. One way to use ideal types is to analyse how that type differs from the others and how it affects the system: which is how we are going to use it here.

Charisma and Leadership

Charismatic authority — and the leadership that stems from it — differs in two ways from Weber’s other forms of authority: it rests on characteristics of the individual, not the organisation and charisma is uniquely tied to change. These two differences are both important for understanding the importance of the role of charisma in Agile transformations. But this does not explain how charisma is instrumental in achieving Agile transformation. For that, we need to look at more closely charisma in the context of the organisation.

Charisma in Context

Charisma is frequently portrayed as a “great man” model of leadership, particularly in the more popular descriptions of leadership. That is, the leader’s personality is central and instrumental to the flow of events. This leads to a line of research tying specific personality characteristics or behaviours to to the effectiveness of leaders. This line of thinking sits behind the innumerable books, articles, presentations, and so on that list the characteristics of leaders. They refer to a mix of personal characteristics (e.g. integrity), behaviours (e.g. earning respect), and activities (e.g. developing a vision). While this approach is interesting, it tends to produce a complex and confusing array of advice to aspiring leaders or organisations trying to identify and cultivate leadership. On one hand, the common themes running through this advice tends to be fairly similar; on the other, it tends to be so generic that it provides limited practical guidance as to what constitutes leadership and what I should do to encourage it in my business. But there is a deeper problem with this view of charisma — and of leadership in general: leadership does not occur in a vacuum it includes the led as well as the leader. To properly understand charisma we need to understand how charisma is connected to the group.

Charisma and the Group

One way out of this dilemma is to look to the psychological and sociological research on charisma. This research emphasises that charisma is a social relationship between the leader and led. That is charisma is not just about the leader but about the relationships around the leader. When I first encountered this field of research, my instinctive reaction was to reject the evidence in favour of my firmly held belief in leadership. That’s a common reaction: this idea rarely appears to gain much traction outside of reasonably specialised research. But it’s an idea well worth detailed exploration.

This isn’t a new idea. A range of research in the 1970s showed the impact of leadership was much less than we commonly assumed. One significant summary from Jeffrey Pfeffer and George Salancik stated that leadership accounted for around 10% of the variance in corporate performance. The measurable impact of leadership is much less than the impact of the environment, such as the state of the economy or of the firm’s industry. As much as we want there to be leadership, the data tells us it’s not the all-powerful force we often think it to be.

Findings such as this led James Meindl to coin the phrase, “the romance of leadership". There is a natural tendency to see leadership: to attribute outcomes to leaders, irrespective of the facts of reality. This tendency comes from the human desire to see agency, particularly individual agency, in our world. Consequently, we over-attribute events and outcomes to leadership and under-attribute them to social context — a finding supported by experimental research in leadership. This line of thinking flips the normally assumed causality: leadership is caused by the social group as much as leadership causing particular behaviours and outcomes in a social group. The question then becomes, what aspects of the social group determine leadership. Meindl later found that organisations with a collectivist culture and organic structures were more likely to see the emergence of charismatic leadership. Conversely, perceptions of crisis were negatively related to the emergence of charisma. These three findings warrant further exploration.

According to Geert Hofsteder, who researched international cultural comparisons for over 40 years, collectivist culture — at a societal level — promotes selflessness, working as a group, doing what is best for the society, and giving families and communities a central role. Not coincidentally, Japan — the birthplace of Lean — is relatively more collectivist than the US and Europe. At the organisational level, the generative culture found in the State of DevOps to be associated with high-performing technology businesses has much in common with the collectivist culture. The generative culture focuses on the organisation’s goal: it values high co-operation, sharing risks, and bridging between people and teams. While these two descriptions of culture are not the same, they share a lot in common: the shared responsibility for solving problems and the focus on the organisation/society rather than the self. It’s a small leap to say that the generative culture is a ripe ground for the emergence of charismatic leadership or that both generative and collective cultural values align closely with the Agile values.

In the previous post, I briefly introduced the idea of an organic organisational structure, coined by Tom Burns and G.M. Stalker in the late 1950s. An organic organisation can be described structurally: it has:

has a flat structure with few layers of management — formally defined as low complexity;
is made up of multi-talented individuals with limited job specialisation — defined as low formalisation;
devolves decision-making and responsibility within the organisation — low centralisation.

Externally, organic organisations adapt to rapidly changing external environments. Their structure allows for quick, efficient communication; rapid adaptation to changing environments. While Burns and Stalker were working in the late 1950s, their description of organic organisations and their benefits wouldn’t look out of place in any Agile textbook today.

The third finding is that charismatic leadership is less likely when their is a crisis. This finding is a little surprising, as one of the more common views of the leadership process is that leaders emerge (or construct) a crisis (e.g. John Kotter); that surprise warrants closer examination. When individuals perceive a crisis, they look for charisma in leaders. But charisma is less likely to emerge from a crisis situation. Another study (Chen, et al) found that the emergence of charisma depended on the nature of the transformation: change for growth was more likely to produce charisma than change for retrenchment. This all speaks to the importance of context in charisma.

This discussion has returned us to the idea that transformational leadership is more likely in a receptive environment. We have looked at three dimensions of that environment associated with charismatic leadership: culture, structure, and context. Charisma is more likely to be found when the culture and structure of the organisation is essentially Agile. Moreover, charisma is more likely to be found when there is a positive change for growth is more likely to produce charisma than retrenchment in response to crisis. But what happens at the end of charisma.

The End of Charisma

Charismatic leadership is uniquely tied to change: the charismatic leader bases their power on themselves and challenges the organisational and societal structures of traditional and bureaucratic authority. This idea begs the question of what happens when the charisma ends: when the organisation or society is transformed into the leader’s image or when they leave? There are three common answers to this question — two of which represent a danger to the organisational transformation.

Firstly, another charismatic individual can step in. The previous section argued that charismatic leadership is more likely in organisations that have the hallmarks of Agile or Lean: a generative or collectivist culture and an organic structure. If these characteristics are genuinely embedded in the organisation, then the emergence of another charismatic individual is likely. Conversely, if this does not happen, then it suggests that the culture and structure may not be as Agile as we believed. The new individual will undoubtedly be associated with further change in the organisation: this is essential for continued agility. This is to be expected, but it should be a progressive change: continuing and building upon the culture and structure already in place, rather than rejecting and replacing it.

Secondly, charismatic authority and the transformation it generates can be followed by “routinisation” as the inspiration of the leader is either embedded into tradition or leads to a new rational settlement (a new agreement on the rules by which the organisation operates). This process of disruption of the status quo, change to a new reality, and cementing change is almost identical to the three-stage model of change that I described in an earlier post. It sees the charisma mutate into new forms of authority. However, if we take seriously the notion that Lean and Agile are built on continuous improvement, routinisation presents a real danger to the organisation. Agile practices can easily become routinised and locked-in, irrespective of their continued effectiveness. Routinisation can signal the lack of a generative or collective culture and a progressive movement away from an organic structure to the increasing formalisation, complexity, and centralisation of a mechanistic organisation.

Thirdly, the organisation can revert back to the previous structure and culture. When this happens, it is a signal that the transformation engendered by the charismatic leader was superficial and did not reach into the more elemental aspects of the culture. For example: decentralisation of the organisation requires a high level of trust in the individuals taking decisions. Conversely, centralisation is a signal of a lack of trust in those individuals. While it is not unusual for this to occur, it highlights the problems of associating the culture too highly with an individual.

What does this all mean for Agile Transformation?

In many ways, this post has spectacularly failed at the exam question. We are exploring how to achieve Agile transformation and the importance of transformational leadership. The answer so far is, if you have an Agile organisation, you’ll have ground ripe for cultivating charismatic and leadership that leads to transformation. A circular argument which suggests that we’re asking the wrong question.

The more pertinent questions are how do we create the fertile soil for Agile transformation, what sort of individuals do we want in senior roles for that transformation, and how do we ensure that the transformation lasts?

The answer to the first question comes back to culture, structure, and context. For a successful Agile transformation, the organisation needs to be Agile. It needs to have values and beliefs aligned to Agile. These are not captured in formal statements of values but manifest in the structure of the organisation. Specific actions will help to achieve this:

Providing clarity on organisational goals
Collapsing structures and using a direct focus on goals and outcomes to provide control
Genuine empowerment of teams coupled with trusting their decisions, even when things go wrong
Expectations that the teams will be multi-functional and both hiring and rewarding people on this basis

The answer to the second question is to reward, promote, and hire individuals and behaviours that are aligned to the essential beliefs and values that enable Lean and Agile. This more clearly than anything signals what is important to the organisation. Conversely, showing intolerance to behaviours that do not align with those essential values is equally important. Rewarding — even tacitly — people who do not behave in accordance with the values you want to promote undermines the culture necessary to build an Agile organisation. These guidelines apply across all levels of the organisation, but particularly to more senior roles and provides the best chance for charismatic leaders to emerge.

Finally, the third question is answered by largely answered by the first two. The commitment of the organisation to Agile and the culture that it is built upon needs to persist beyond an initial change. This means the above steps and guidelines need to be applied permanently not just as an initial transformation programme followed by a retrenchment into more traditional organisational structures and values. The only missing piece is that these decisions need to be accepted and actively implemented by those ultimately responsible for the organisation: the owners, the board, and the executive. This applies regardless of whether they are the charismatic or transformational leaders of the organisation.

This post has focused on charisma as an important aspect of transformational leadership. I delved into the origins of charisma and then moved to the research, particularly focusing on research that shows charisma is as much a social attribution caused by the automation as it is about defining the organisation. Drawing on this, I looked at what happens when charisma ends and how to sustain transformation beyond an individual transformational leader. These themes were then drawn together into three essential recommendations for how to instil Agile through structure and culture and how to sustain Agile in the longer term. This post has focused on the organisation — which could mean a team, a group, a division, or a whole organisation. It has planted the seeds for my next posts: Agile governance.

References

I’ve referred to a few different pieces of research. I’ve highlighted some of the key sources that provide the evidence base for this post.