How useful is the genetic algorithm for financial market forecasting?
There is a large body of literature on the "success" of the application of evolutionary algorithms in general, and the genetic algorithm in particular, to the financial markets.
However, I feel uncomfortable whenever reading this literature. Genetic algorithms can over-fit the existing data. With so many combinations, it is easy to come up with a few rules that work. It may not be robust and it doesn't have a consistent explanation of why this rule works and those rules don't beyond the mere (circular) argument that "it works because the testing shows it works".
What is the current consensus on the application of the genetic algorithm in finance?
I've worked at a hedge fund that allowed GA-derived strategies. For safety, it required that all models be submitted long before production to make sure that they still worked in the backtests. So there could be a delay of up to several months before a model would be allowed to run.
It's also helpful to separate the sample universe; use a random half of the possible stocks for GA analysis and the other half for confirmation backtests.
Is that a different process than you would use before trusting any other trading strategy? (If so, it is not clear to me what you gain from making a GA model using data to time t, then testing until t+N before trusting it, versus using data to time t-N, testing from t-N to t, and using it immediately.)
@DarrenCook one issue I see is that if you test from t-N to t and find it doesn't work well, then you're going to create another model that gets tested on that same time period t-N to t (ad infinitum). That introduces the likelihood of "meta"-overfitting during the model creation process.