VW did not cheat — How building a smart car was a stupid decision

The world is shocked to learn that Volkswagen (VW) supposedly “cheated” tests and manipulated the results to make it seem their Diesel cars are better than what they really are. I argue they did not tamper with results and that the problem is not VWs methods. Instead with Diesel itself, the industry and the scrutiny we hold them at!

The problems has been known for a while and for some reason it suddenly turned dramatic. Media has gone for the low hanging fruit in how reporting this story is done and it is time to clear up a few misconceptions. VW have done nothing wrong given the rule-set of the industry. The tests measured the engine emissions during optimal conditions. The EPA, who is in charge of the law suit against VW declares correctly that all auxiliary emission control devices (AECD) in the car must be declared for (full paper here, great read… it really is!). It must have a justification of why its there and what parameters they control. This was assumed to be enough in terms of stating the tests intent. Assumed to be enough!

“Assumption is the mother of all fuck-ups!”

Despicable as it might be, the problem is in our assumption that Diesel is more environmentally friendly, that manufacturers intent is good and that tests results are accurate and there are but one truth to the numbers. Every car manufacturer wants to build a great performing car and that means tweak the engine to maximize performance. The Notice of Violation from EPA to VW states clearly this oversight in the sentence “VW knew or should have known that its road calibration… bypass, defeat, or render inoperable element of the vehicle design related to compliance”.

Basically they are saying “you did something to produce test results that were accurate, we assume that you did this intentionally”.

VW did not change, tamper nor manipulate any test results.
Instead they engineered the vehicle to be smart about how it used the engine, and shouldn’t they?

Building efficient AECDs are expected of them. They merely followed protocol in a highly competitive market. The fault is not with VW but with the test protocols used and our idea that Diesel somehow is more friendly to the environment than alternatives.

How can a car show 35x less emissions in tests?

“An ECM from a 1996 Chevrolet Beretta- 2013–10–24 23–13” by User:Mgiardina09 — Own work. Licensed under CC BY-SA 3.0 via Commons

In today's car climate the average consumption of a car is a very important factor for the potential buyer.

At the same time the buyer want a fast, powerful and sporty car. This seemingly impossible equation has been solved with the introduction of the ECU-chip in every car since 2002. The ECU is a little computer in your car. Initially it didn’t do to much except regulate air, fuel and compression of the engine. Digitally it could alter levels based on temperature of the engine, throttle position and a few other factors. Over time the responsibility has grown to the point where we today remotely can take control of cars from the living room sofa and override the actual driver (here’s how).

This allow the car manufacturers to build a engine capable of several different running modes and let software dictate how it should operate. Be sporty or be mellow, stiff or soft, strong or fast.

If you have a fairly new Diesel car you might have experienced how it sometimes on the highway gives a feeling of dropping all power and just cruises. This is when the car changes algorithm to be ultra efficient. This technology is very sophisticated and how our cars today are so much more efficient than in the past. This is no secret, it is part of every car ad and known by everybody in the industry. Today cars dont compete so much with engines, as with software. The miles per gallon is not a fixed value per car, nor is it very tightly coupled with the engine in the car but the software decisions the car makes.

What the company VW a few weeks ago was well respected for, is the same reasons everybody now is so upset. This software even has a name, “DynaMode”. It really is two cars in one.

The illusive art of testing

There is a very important lesson to be learned here. Coming from the software industry where testing is critical to offering a good product I have learned a lot about testing. Primarily the necessity to specify which criteria to test as well as under what condition the test should take place. Many people focus only on one part of it; “what to test” and then develop methods to achieve this which become the “how do we test”. They completely forget about the equally important “what are the conditions we are testing for”, which can be answered by asking oneself “why are we testing this”.

In the case of VW the “what” to test was easy. They wanted to test emission levels and then they figured out which machinery and methods was needed for a accurate reading. The “why are we testing emissions” was only assumed and never declared. The assumption was “We are testing emissions to ensure that the car can be used in urban locations and wont affect our climate severely over time”. In reality the tests became a game of having better numbers than the competitors and ensure lower numbers than regulations specify as ceiling-values. Without the why to state the context, the tests become useless. In the software industry this is called a “moot test”.

Outsmart the smart

Whatever you test, the tester needs to make sure that the test results are clinically correct, comparable and reliable. To achieve this you need to provide a “controlled environment”. This is also the inherit problem.

A controlled environment means no external factors will influence the results. One typical external factor is reality. A controlled environment is perfect for measurements, but also perfect for optimizing the performance of the test subject.

So to set the stage we have a predictable isolated environment, a car which can change running mode depending on the situation and clear numbers that needs to be matched.

What the media says VW did

Generally speaking most media outlets report that VW altered the test results for pollution from a large fleet of Diesel cars to pass emission tests. They did this by developing a intelligent piece of software that could detect when the car was being tested. During tests it would ensure that the values showed 35 times lower than what was actually the case. The detection mechanism was triggered by checking if the steering wheel was moving in combination with the engine keeping a steady speed together with some other factors. Media states that this was a deliberate intent from VW to trick tests according to most reports and has led to a record drop of share value (31% at time of writing). A record recall of 11 million cars has been issued and the CEO Martin Winterkorn has been forced to resign.

What VW actually did

VW never changed a single value reported from the engine. This does not mean what they did was fair, but right should be right. What they did was abuse the fact that the test conditions were poorly specified. The same software being criticized for fooling the tests is critical for the performance of the car when driving on the roads. Were the car not to use these smart utilization the car would be useless. However when the software is factoring in the environment to improve engine performance and the car is in a clinical environment you get unrealistically good values.

VW has optimizations to ensure best fuel consumption according to terrain and the drivers intent. These optimizations are partly based on predictive algorithms. What this means is that the car does samplings many times per second and then adjust engine operation parameters based on the assumption that nothing will change. It adapts to the current conditions to maximize performance. This is fair game and to be expected!

The engine perform exceptionally well if the conditions are exceptional. Any place outside of roads are exceptional for a car!

Are they alone in this?

Hardly, as software powers all cars today with the purpose of making sure the car is performing perfectly based on the situation it is presented with. Are the lab environment normal conditions? Hell no! But should the engine intentionally perform worse even though it can prevent massive pollution?

They knew that given the isolated test conditions the output is nowhere near what you find when actually driving in traffic. But should VW be faulted for optimizing their software or should we not instead give this scrutiny to who is in charge of these tests, what are the tests supposed to prove and who designs the conditions in which tests should be done?

What is the lesson to be learned?

The problem is not the software that intentionally tried to lower the emissions. The problem is that we have been duped to believe that Diesel is in any shape or form more environmentally friendly than normal petrol which is not the case. The industry manipulated politicians to create incentives which lowered the price of Diesel. Even after it was clear that the emissions from Diesel were dangerous, car manufacturers did not change their line-up, politicians did not remove the incentives and consumers are trapped believing they are doing the best they can for the longevity of our planet.

I learned at great cost in my years as software engineer, technical project manager and Chief Technical Officer that you must consider the environment when designing tests.

Isolation simplifies understanding the results and makes test data comparable, but at the expense of realism.

If you are testing for reality you must accept that the very definition of reality is that it is ever changing, unpredictable and anything but isolated from external factors. The only thing you can predict is unpredictability and this must be the core of your test if you wish to learn anything at all!

Finally, we can improve the test protocols ad infinitum, but never should it take away our responsibility as human beings. It can not be the responsibility of the tests to prove something wrong, but to the manufacturers to prove something right!

If you found this article worth your while, press the heart below to recommend it to others. It really does make a difference for me. Everything I write and research I do for free, if you like it you can pay with attention, recommendations and comments. Engaging with our readers is why we all do this!