|
|
|
|
|
|
|
curve fitting is taking historical data and analyzing it to produce a system that looks great historically but has so many variables that it is untradeable in real time. The mistake is adding another rule (variable) to handle an exception so the historical trading looks fantastic. |
|
|
|
|
|
|
|
|
Neal: How do you avoid curve fitting? |
|
|
|
|
|
|
|
|
Committee: From my own studies, I have come to agree with the literature. It states that five or fewer variables will likely avoid curve fitting. I am very comfortable with three variables. Having four variables makes me slightly uneasy. And a system with five variables is to be closely watched. Using six or more variables is dangerous. One reservation I have about neural nets arises over the issue of curve fitting. A neural net can examine hundreds of variables and put them together in untold combinations. This leads to another question I was often asked during my seminars. How many trades should I have on a historical test before I consider the results statistically reliable? According to sampling theory, the answer is 30 or more (closed) trades. I developed a long-term system that had only seven trades, and six were winners. Someone else asked me if I would trade a system that looked good but had fewer than 30 trades in its historical testing. This is a matter of personal preference. I would trade the system, but I would not risk much of my capital on it. |
|
|
|
|
|
|
|
|
Neal: Do you optimize your systems? |
|
|
|
|
|
|
|
|
Committee: I used to optimize everything I could. I found the historic results were great and real-time trading results were not. I am not against optimization, but now I use it very sparingly. During one of my past studies, I wanted to optimize variables every day and then see if I could make money by trading the next day. It didn't work. I believe the reason it didn't was the variables I used back then. This is important because it showed me there might be a world of indicators out there, but very few |
|
|
|
|
|