Blind Spots

Jeff Davis
Jul 20, 2017 · 5 min read

When you build a trading strategy one of things that will occur are the inevitable blind spots. I’m sure many of the more technically inclined are thinking to themselves “what is this guy talking about now, that’s why I back test”. Back testing is something that many of you are very proficient at. I will admit I am not. One reason for this is because I am one of those people that doesn't have confidence in them. This probably stems from the fact that I believe it very easy to find that slippery slope called curve fitting or over optimization. In many cases this can happen in ways that you don’t even realize. This post is not meant to blast back testing, but try and help in the use or even non use of it. I don’t have the skills of a Dave Bergstrom or Mike Harris and I freely admit to this and just try to do what makes sense to me. If you want to follow guys that know this stuff cold and can help in this regard follow them and read everything they write.

Talk to a fund of funds or a sophisticated institutional investor and you will find out quickly they don’t put very much trust in a back test. They want to see actual live performance. They will tell you they have never seen a bad back test presented to them, yet they see many great back tests generate dismal performance when live. That fact alone should have you going deeper and figuring why this is. This gets me to the topic of blind spots.

The overwhelming majority of systems/strategies all have blind spots when you first build and use them. A blind spot is an area that your system will under perform or perform outside the parameters you expected. It is a weakness that you don’t see coming. Most everyone understands that there will be some sort of draw down period in their system, but a blind spot isn’t this. A blind spot isn’t anticipated and how you handle them can determine success or failure. The curious thing is that what you do in response to the now discovered blind spot can sometimes be worse than the blind spot itself.

Semi trucks use 2 types of mirrors to give different views of the traffic taking place behind and alongside them. This doesn't eliminate a blind spot but reduces the size of it. In one of the systems I helped build a blind spot emerged. Now back testing had been done, but the samples used were over too big a time frame and this blind spot went unnoticed. The system was funded and deployed with no idea a blind spot lurked. The first 2 months the system performed within the parameters expected. Then the blind spot started to emerge. The institution which invested had just inquired about the recent performance. Confident that everything would right itself and performance would quickly return to within the expected parameters no changes were made. Long story short, the investor yanked funds 4 months later. The market had changed. The conditions now present in the market had not been present for any length of time in either our in sample or out of sample data. A huge learning lesson. The system did not lose money over this 6 month period, but it didn’t make any either. All the back testing showed very shallow draw down coupled with very quick recovery times. The draw downs were shallow, but the recovery was shallow also. Not what was expected. It was a day trading system built to make consistent small gains with negligible draw down in both time and returns. Blindsided by a blind spot.

The choice to not immediately fiddle with the system rules was made and now the work had to begin again. Should we have fiddled quickly? Did we make the right choice? Was this blind spot hiding in plain sight and we were the blind ones? If we fiddled what would we have done? Turn it off? Change rules? Guess what, we weren’t sure of any of these answers. Hard work of going back through process and accountability review needed to take place. Already deflated by the failure we now had to persist and try and figure it out.

In the end a bunch of lessons were learned and some things were answered with very uncomfortable answers. Some parts of the blind spot could have been discovered. Some parts we had no chance to know it was coming. Markets have a way of always finding ways to keep everyone off balance. Blind spots are there because the data won’t show a similar period because the market will rhyme not repeat and by the time this rhyme starts to play out the participants won’t be the same thus a rhyme and never a repeat. The hardest impacting lesson was to always take time and try and shatter your system. Throw lots of scenarios at it and see where it fails. Work your ass off trying to break all your hard work. Exactly, mentally not easy to want to destroy what you just spent countless hours trying to create. Spend as much time trying to expose where any possible weakness could come from as you do trying to prove it can work. Yes this is twice the work you thought you were in for going into this. Resist all temptation to actually think you have something that can work and truly try to break it and crush it. Then and only then can you start to build confidence in what you have worked to build. The market is always building a new roller coaster that nobody has seen the exact design, yet you need to be confident you can either ride it or get off at the proper time. Do your work and then try and disprove your work before you place bets upon your work. Enjoy the ride because you are going to be riding whether you want to or not.

)
Welcome to a place where words matter. On Medium, smart voices and original ideas take center stage - with no ads in sight. Watch
Follow all the topics you care about, and we’ll deliver the best stories for you to your homepage and inbox. Explore
Get unlimited access to the best stories on Medium — and support writers while you’re at it. Just $5/month. Upgrade