The Colossus computers were developed for British codebreakers during World War II to help decipher teleprinter messages between the German High Command (OKW) and their army commands throughout occupied Europe, messages which had been enciphered using the mechanical Lorenz SZ 40/42 cipher machine; part of the operation of Colossus was to emulate the Lorenz machine electronically.
Purpose and Origins
Long before the war began, the airwaves were full of coded messages as Hitler prepared for battle. Cracking the German ciphers became the priority of a special British Intelligence unit.
In May 1938 Admiral Sir Hugh Sinclair, head of the British Secret Intelligence Service (SIS), bought Bletchley Park, an ornate mansion 50 miles north of London, for use by Government Code and Cypher School (GC&CS, forerunner of GCHQ) and SIS in the event of war. Initially, a wireless tiny room was set up below the water tower on the mansion roof which was chosen because the presence of extensive lead piping significantly amplified radio signals. This enabled MI6 to maintain radio contact with their agents from as far away as Norway to North Africa and with listening stations all over Britain that were intercepting German messages.
By mid 1940, the German Army had conquered all of western Europe. Hitler was tightening the noose around Britain. In the Atlantic, German U-boats were decimating Allied convoys, threatening to cut off Britain’s only lifeline. In the spring of 1941, the naval war was building up in the Mediterranean. Hitler had joined forces with the Italian Fascist Mussolini.
By the summer of 1941, Bletchley Park was able to crack the naval Enigma in less than two days, due partly to U-boat documents and partly because they had learned to exploit a crucial weakness of the Enigma machine.
Hitler demanded a more sophisticated encoding system that the most senior Nazi military leaders could use with confidence, one far more complicated than the Enigma machine, which was introduced in 1923 by Chiffriermaschinen Aktien-Gesellschaft (Cipher Machines Stock Corp).
The German Army High Command commissioned C. Lorenz AG, a major supplier for the German military, to produce for them a high security teleprinter cipher machine to enable Hitler himself and his generals in charge of huge forces to communicate by radio in complete secrecy.
Lorenz developed a rotor stream cipher machine based on the additive method for enciphering messages invented in 1918 by Gilbert Vernam in America.
In answer to Hitler’s demand for a device that produced unbreakable code, Lorenz had created an astonishingly powerful encryption machine. Teleprinters operated on a simple, universal binary code that was widely known. Lorenz attached to a standard teleprinter a machine that cunningly exploited the teleprinter language itself to produce a highly complex code.
Enigma was capable of sending out a code in 15 million million different ways. The Lorenz SZ40 took encoding to a totally different level. It was capable of sending out a secret message in 1.6 million billion ways.
Lorenz also only needed one operator to use it, whereas Enigma needed three at the sending end and another three at the receiving end.
An experimental link using SZ40 machines was started in June 1941. The enhanced SZ42 machines were brought into substantial use from mid-1942. The more advanced SZ42A came into routine use in February 1943 and the SZ42B in June 1944.
The teleprinter signals being transmitted by the Germans, and encyphered using the Lorenz 12 wheels cipher machines, were first heard in early 1940 by a group of policemen on the South Coast who were listening out for possible German spy transmissions from inside the UK. They heard these weird signals. And they sent them to Bletchley Park.
The Testery and The Newmanry
As the number of intercepts increased, now picked up by Britain’s Y-stations at Knockholt in Kent and Denmark Hill, a section was formed in Bletchley Park headed by Major Ralph Tester and known as the Testery.
Very secure and easy to use, Hitler and his frontline generals were so confident that the Lorenz code could not be broken that the machine spread very fast through Nazi-occupied Europe to transmit highly secret military messages. What they could not plan for was human error.
John Tiltman, one of the top codebreakers in Bletchley Park, took a particular interest in these enciphered teleprinter messages. They were given the code name “Fish”. The messages which (as was later found out) were enciphered using the Lorenz machine, were known as “Tunny”.
A number of Depths were intercepted in but not much headway had been made into breaking the cipher until thanks to a German operator error on August 30, 1941, a cipher breakthrough was accomplished.
In a brilliant achievement, Tiltman broke the code in just ten days. But as memorable as this achievement was, it did not tell British Intelligence how Tunny / Lorenz worked. This task was given to William ‘Bill’ Tutte.
Tutte examined the broken code and found patterns and repetitions in it. Using his own intuition and mathematical formulas, Tutte and his team found out how Tunny actually worked — a feat a recent BBC programme about Tutte called “the greatest intellectual achievement of World War II”.
It soon became clear just how important Tutte’s breakthrough had been. Bletchley Park decoded messages sent by the most senior of the Wehrmacht’s officers — Keitel — and even messages sent by Hitler himself.
Max Newman was a brilliant mathematician who had closely followed the reverse engineering work of William Tutte on the German Lorenz SZ40. However, while Tutte’s achievement had been vital, the decoding process around Lorenz was slow. Newman believed that it was possible to mechanise the process that Tutte and his team had to do by hand. Newman suggested using high-speed electronic counters to mechanise the process.
Electronic counters had been developed in Cambridge before the war. Used for counting emissions of sub-atomic particles, these had been designed by C. E. Wynn-Williams, a Cambridge don. Newman knew of Wynn-Williams’ work, and in a moment of inspiration he saw that the same idea could be applied to the Tunny problem. Within a month of Tutte’s inventing his statistical method Newman began developing the necessary machine. He worked out the cryptanalytical requirements for the machine and called in Wynn-Williams to design the electronic counters. Construction of Newman’s machine started in January 1943 and a prototype began operating in June of that year, in the newly formed Tunny-breaking section called the Newmanry. The prototype machine was soon dubbed ‘Heath Robinson’, after the famous cartoonist who drew overly-ingenious mechanical contrivances.
Heath Robinson worked, proving in a single stroke that Newman’s idea of attacking Tunny by machine was worth its salt and that Tutte’s method succeeded in practice. However, Heath Robinson suffered from ‘intolerable handicaps’. Despite the high speed of the electronic counters, Heath Robinson was not really fast enough for the codebreakers’ requirements, taking several hours to elucidate a single message. Moreover, the counters were not fully reliable — Heath Robinson was prone to deliver different results if set the same problem twice. Mistakes made in hand-punching the two tapes were another fertile source of error, the long chi-tape being especially difficult to prepare. At first, undetected tape errors prevented Heath Robinson from obtaining any results at all. And paramount among the difficulties was that the two tapes would get out of synchronisation with each other as they span, throwing the calculations out completely. The loss of synchronisation was caused by the tapes stretching, and also by uneven wear around the sprocket holes.
The question was how to build a better machine — a question for an engineer. In a stroke of genius, the electronics expert Thomas Flowers solved all these problems.
Colossus Mark 1
The original Colossus was designed by engineer Thomas “Tommy” Harold Flowers with input from Allen Coombs, Sid Broadhurst and Bill Chandler to solve a problem posed by mathematician Max Newman at the Government Code and Cypher School (GC&CS) at Bletchley Park.
Flowers was brought in to fix ‘Heath Robinson’ but swiftly realised that the problems could never be solved, and he made the radical proposal that the mechanical switching units, which made up the bulk of the Robinson machines, should be replaced by state-of-the-art vacuum tubes (thermionic valves), thyratrons and photomultipliers. The wheel patterns would be generated electronically in ring circuits and the computer would read only one paper tape, eliminating the Robinsons’ synchronisation problems.
Flowers’ proposal was met with disbelief on the part of the codebreakers who were convinced that valves were unreliable and would keep breaking down. Flowers was confident he could make it work. He had, before the war, wired together an experimental installation containing three to four thousand valves (by contrast, Wynn-Williams’ electronic counters of 1931 contained only three or four valves) and explored the idea of using valves as high-speed switches. He knew that if they were never moved or switched on and off valves would run and run.
By the time of the outbreak of war with Germany, only a small number of electrical engineers were familiar with the use of valves as high-speed digital switches. Thanks to his pre-war research, Flowers was possibly the only person in Britain who realized that valves could be used reliably on a large scale for high-speed digital computing. When Flowers was summoned to Bletchley Park — ironically, because of his knowledge of relays — he turned out to be the right man in the right place at the right time.
Flowers started in March 1943 with a blank sheet of paper, never been done before. He was thinking of a machine with 1,500 valves in it. The biggest machine ever at that time had 150 valves in it.
He succeeded in persuading the codebreakers that valves would work, but then ran into a second problem. They asked him how long it would take to produce his machine. When he told them it could be done in a year, they replied that this was no good, since by then Hitler would have won.
Newman pressed ahead with the two-tape machine. He offered Flowers some encouragement but effectively left him to do as he wished with his proposal for an all-electronic machine. Once Heath Robinson was a going concern, Newman placed an order with the Post Office for a dozen more relay-based two-tape machines it being clear, given the quantity and very high importance of Tunny traffic, that one or two machines would not be anywhere near enough. But Flowers was by now so convinced that his valvebuilt machine would work that he decided to build it anyway.
Such was the scepticism of the managers of Bletchley Park, that they refused to fund the project. Flowers had to fund a great deal of the project out of his own money.
Flowers and his team at Dollis Hill, in northwest London, constructed the first prototype in 10 months, working around the clock. Colossus, as it was to become known, was demonstrated at Bletchley Park on December 8 1943.
Colossus reduced the time to break Lorenz messages from weeks to hours. But it was its accuracy in comparison with Heath Robinson that astounded the codebreakers. They set out to test it by setting up a problem to which they already knew the answer. Each run took about half an hour, and they let Colossus run for four hours. It solved the problem eight times, on each occasion coming up with exactly the same answer.
The prototype Colossus was dismantled, brought to Bletchley Park in lorries on 18 January 1944, and reassembled by Flowers’ engineers. Tutte’s statistical method could now be used at electronic speed. The computer attacked its first message on Saturday 5 February. Flowers was present. He noted laconically in his diary, ‘Colossus did its first job. Car broke down on way home.’
Colossus was faster, more reliable and more capable than the Robinsons, so speeding up the process of finding the Lorenz chi pin wheel settings. It did so with an optical reader which, at 5,000 characters per second, was driven much faster than the Robinsons’ 2,000 cps and meant that the tape travelled at almost 30 miles per hour (48 km/h). This, and the clocking of the electronics from the optically read paper tape sprocket holes.
Colossus was not a general-purpose machine, being designed for a specific cryptanalytic task involving counting and Boolean operations. Like the later ENIAC of 1946, Colossus did not have a stored program. To set it up for a new task, the operator had to set up plugs and switches to alter the wiring.
Colossus compared two data streams, counting each match based on a programmable Boolean function. The encrypted message was read at high speed from a paper tape. The other stream was generated internally, and was an electronic simulation of the Lorenz machine at various trial settings. If the match count for a setting was above a certain threshold, it would be sent as output to an electric typewriter.
The output of Colossus was a series of counts indicating the correct wheel settings. Not even the de-chi was produced by Colossus itself, let alone the plaintext — and there was certainly no facility for the automatic translation of German into English.
Colossus immediately doubled the codebreakers’ output.
Indeed, a crisis had developed, making the work of Newman’s section even more important than before. In December 1943 the Germans started to make widespread use of an additional device in the Tunny machine, whose effect was to make depth-reading impossible. The hand breakers had been prone to scoff at the weird contraptions in the Newmanry, but suddenly Newman’s machines were essential to all Tunny work.
It was clear to the Bletchley Park authorities — whose scepticism was now completely cured — that more Colossi were required urgently.
Colossus Mark 2
Following the success of Colossus Mark 1, the Bletchley Park codebreakers asked Flowers to build a bigger version, boosting the speed to 25,000 cps.
By means of repluggable cables and panels of switches, Flowers deliberately built more flexibility than was strictly necessary into the logic units of the prototype Colossus. As a result, new methods could be implemented on Colossus as they were discovered. In February 1944 two members of the Newmanry, Donald Michie and Jack Good, had quickly found a way of using Colossus to discover the Tunny wheel patterns. Flowers was told to incorporate a special panel for breaking wheel patterns in Colossus II.
In March 1944 the authorities demanded four more Colossi. By April they were demanding twelve. Great pressure was put on Flowers to deliver the new machines quickly. The instructions he received came from the highest level, the War Cabinet, and he caused consternation when he said that it was impossible to produce more than one new machine by 1 June 1944.
The new Colossus had to be ready by June 1944 or it would not be of any use. Although the reasons for the deadline were never disclosed, Flowers immediately realised its significance. An improved Colossus Mark 2 with nearly 2,400 valves and using shift registers, processing five streams of dot-and-cross in parallel, first worked on 1 June 1944, five days before D-Day. It was just in time for the deciphering of messages which gave vital information to Eisenhower and Montgomery prior Normandy Landings.
Just before D-Day, Marshal Rommel, the Desert Fox, was appointed inspector general of the western defenses, and he sent this enormously long message, very detailed description of the western defenses, where each unit was located and what equipment they had. It was a — it was 70,000 letters, which was read. When Bletchley Park decoded Rommel’s message, it contained alarming news. German tanks were massing at the exact spot where American troops were about to parachute into Normandy.
They were redeployed. That the codebreakers were able accurately to identify the location of 60 of the Germans 62 divisions greatly facilitated a successful landing.
Allied planners knew the whole German defensive structure along the northern French coast. They knew how many tanks and aircraft the Germans had in France and where they were based. The even knew how many aircraft were actually unable to fly because they were being serviced.
The deciphered Lorenz messages showed that Hitler had swallowed the deception campaigns, the phantom army in the South of England, the phantom convoys moving east along the channel; that Hitler was convinced that the attacks were coming across the Pas de Calais and that he was keeping Panzer divisions in Belgium.
Hitler refused to move more troops to Normandy. No senior Nazi military officer was willing to argue with Hitler regarding this. Therefore, while a massive invasion of Normandy was bound to result in casualties, they were not as bad as might have been expected for such a large-scale invasion. The only sizeable casualties occurred at Omaha Beach — and these had nothing to do with the work done by Flowers.
Those at Bletchley Park even used Tunny to predict what Hitler would do next. They knew that he was not willing to withdraw German troops from Italy. Therefore the campaign there continued in the full knowledge that Hitler would use resources there defending Italy that were much needed in France after the success of D-Day.
After D-Day the French resistance and the British and American Air Forces bombed and strafed all the telephone and teleprinter land lines in Northern France, forced the Germans to use radio communications and suddenly the volume of intercepted messages went up enormously.
When the final signal came through from Donitz surrendering, it was in clear. When messages did begin to come through in clear, then all the secrets of the war really were beginning to fade into history already.
By the end of hostilities, 63 million characters of high grade German messages had been decrypted — an absolutely staggering output from just 550 people at Bletchley Park, plus of course the considerable number of interceptors at Knockholt, with backups at Shaftesbury and Cupar in Scotland.
The end of Colossus
Colossus was constantly updated: by the end of the war there were 10 in operation, manned 24 hours a day by Wrens working to the programmes laid down by the codebreakers. Each of them occupied a large room in F Block or H Block in Bletchley Park.
After VJ Day, suddenly it was all over.
Eight of the ten Colossi were dismantled in Bletchley Park.
In April 1946, the remaining two were moved to British secret service buildings in Eastcote in suburban London. The old name of the organisation, ‘Government Code and Cypher School’, was formally changed to ‘Government Communications Headquarters’ (GCHQ). Six years later another move commenced, and during 1952–54 GCHQ shifted its personnel and equipment, including its codebreaking machinery, away from the London area to a large site in Cheltenham.
One of the Colossi, known as ‘Colossus Blue’ at GCHQ, was dismantled in 1959 after fourteen years of postwar service. The remaining Colossus is believed to have stopped running in 1960.
Details of what they were used for remain classified.
There is a hint of the importance of one new role for these Newmanry survivors in a letter written by Jack Good:
I heard that Churchill requested that all Colossi be destroyed after the war, but GCHQ decided to keep at least one of them. I know of that one because I used it myself. That was the first time it was used after the war. I used it for a purpose for which NSA [National Security Agency] were planning to build a new special-purpose machine. When I showed that the job could be carried out on Colossus, NSA decided not to go ahead with their plan. That presumably is one reason I am still held in high regard in NSA. Golde told me that one of his friends who visits NSA told Golde that I am ‘regarded as God’ there.
Colossus documentation and hardware were classified from the moment of their creation and remained so after the War, when Winston Churchill specifically ordered the destruction of most of the Colossus machines into “pieces no bigger than a man’s hand”; Tommy Flowers was ordered to destroy all documentation and burnt them in a furnace at Dollis Hill. He later said of that order:
That was a terrible mistake. I was instructed to destroy all the records, which I did. I took all the drawings and the plans and all the information about Colossus on paper and put it in the boiler fire. And saw it burn.Some parts, sanitised as to their original use, were taken to Newman’s Royal Society Computing Machine Laboratory at Manchester University.
Throughout this period the Colossus remained secret, long after any of its technical details were of any importance. This was due to the UK’s intelligence agencies use of Enigma-like machines which they promoted and sold to other governments, and then broke the codes using a variety of methods. Had the knowledge of the codebreaking machines been widely known, no one would have accepted these machines; rather, they would have developed their own methods for encryption, methods that the UK services might not have been able to break. The need for such secrecy ebbed away as communications moved to digital transmission and all-digital encryption systems became common in the 1960s.
The destruction of most of the Colossus hardware and blueprints deprived most of those involved with Colossus of credit for their pioneering advancements in electronic digital computing during their lifetimes.
When World War Two ended Flowers was given a reward of £1000 for his invention — but this sum did not cover the amount of money he had personally invested in the project. This was a considerable sum of money in a country where rationing was still a day-to-day occurrence and where extra money was always a bonus. It is a mark of Tommy Flowers that he divided the £1000 up among the team that had helped him and at the end of it, he gave himself £350 — a good sum of money for 1945 but perhaps not a huge amount for the man credited with inventing the modern computer. If Flowers could have patented the inventions that he contributed to the assault on Tunny, he would probably have become a very rich man.
Newman was offered an OBE for his contribution to the defeat of Germany, but he turned it down, remarking to ex-colleagues from Bletchley Park that he considered the offer derisory. Tutte received no public recognition for his vital work. Turing accepted an OBE, which he kept in his toolbox.
The Newmanry’s Colossi might have passed into the public domain at the end of the fighting, to become, like ENIAC, the electronic muscle of a scientific research facility. The Newmanry’s engineers would quickly have adapted the equipment for peacetime applications. The story of computing might have unfolded rather differently with such a momentous push right at the beginning. Churchill’s order to destroy the Colossi was an almighty blow in the face for science — and for British industry.
Colossus Rebuild
Recommended videos
Alan Turing at Bletchley Park
IEEE Computer’s editor Charles Severance visits Bletchley Park to commemorate the 100th anniversary of Alan Turing’s birth. Turing’s ground-breaking work in the 1940s continues to have an impact on computer science as we know it. The Turing test, Turing machine, Turing completeness, and Church-Turing computability bear his name in acknowledgment of his early breakthroughs and influence. In the video, we see the German Enigma machine used to encrypt messages, and the BOMBE mechanical computing system that was designed by Alan Turing to crack the Enigma code. We also see the first electronic tube/valve-based computer called the Colossus that was built to break the more sophisticated Lorenz SZ42 encryption used for Hitler’s strategic messages during World War II. We see and hear both the BOMBE and Colossus running as if they were in production doing code-breaking during the war.
Uncovering Colossus
Colossus, the world’s first electronic computer, was built during World War II, but kept secret for more than 30 years. Professor Brian Randell tells the story about how he stumbled across a reference to its existence and eventually led to the UK government lifting the veil of secrecy surrounding this pioneering computer in 1975. Prof Brian Randell’s presentation was given in the new Colossus Gallery in The National Museum of Computing on 7 February 2013.
This short documentary made by Lord Charles Brocket for Cannes Film Festival 2010. It told British code-breakers in the Testery at Bletchley Park cracked Hitler’s H.Q most top-secret Tunny (Lorenz cipher). Tunny messages provided vital information that changed the course of the War in Europe and saved tens of millions of lives at critical junctures — such as the Battle of Kursk and D-Day.
Gen. Eisenhower (later U.S President) said after the War “Bletchley decrypts shortened the War by at least 2 years”. Much of this down to Bill Tutte’s success broke Tunny-system and the Testery team breaking messages.
Tunny only declassified in 2002, compared with Enigma’s 1970's. Please spread the word about 3T’s (Tunny, Bill Tutte and the Testery) if we can!
Captain Roberts, a founder member, leading codebreaker and linguist in the Testery at Bletchley; he was the last survivor of 9 cryptanalysts, who worked on Tunny daily, he died on 25th March 2014.
For his final 6 years, Jerry has been campaigning for proper recognition for his fellow codebreakers in the Testery who broke all those Lorenz messages, and for its 3 “Heroes” — Alan Turing, Bill Tutte and Tommy Flowers.
Recommended reading
Further reading
- Lards Ulfving, Frode Weierud, The Geheimschreiber Secret — Arne Beurling and the Success of Swedish Signals Intelligence http://www.usna.edu/Users/math/wdj/_files/documents/papers/cryptoday/ulfving_weierud_secret.pdf
- Stephen Budiansky, Battle of Wits (Free Press, New York, 2000)
- Paul Gannon “Colossus: Bletchley Park’s Greatest Secret” (Atlantic Books, 2006)
- F. H. Hinsley, Alan Stripp, Codebreakers: The Inside Story of Bletchley Park (Oxford University, 1993)
- Michael Smith, Station X: Decoding Nazi Secrets (TV Books, New York, 2001)
- Harvey G. Cragon, From Fish to Colossus: How the German Lorenz Cipher was Broken at Bletchley Park (Cragon Books, Dallas, 2003; ISBN 0–9743045–0–6)
- Ted Enever, Britain’s Best Kept Secret: Ultra’s Base at Bletchley Park (Sutton Publishing, Gloucestershire, 1999; ISBN 0–7509–2355–5)
- Tony Sale, The Colossus Computer 1943–1996: How It Helped to Break the German Lorenz Cipher in WWII (M.&M. Baldwin, Kidderminster, 2004; ISBN 0–947712–36–4)
- Michael Smith, Station X, 1998. ISBN 0–330–41929–3.
- Paul Gannon, “Colossus Bletchley Park’s Greatest Secret” 2006 Atlantic Books; ISBN 1–84354–330–3.
- Jack Copeland: Colossus. The Secrets of Bletchley Park’s Codebreaking Computers. Oxford University Press 2006. ISBN 0–19–284055-X
