html menu by Css3Menu.com
The Portfolio Optimization Myth and other misleading claims !!
There is nothing worse than a competitor trying to advance their own cause at the expense of spreading misleading and false claims about your own products. Recently it was brought to our attention that a said competitor (well sort of a competitor) was sprouting mistruths about TradeSim on their own website and forum hoping that this would sway people away from purchasing or using TradeSim in favour of just sticking with their own product. And fair enough to, if only the claims were true !!!
The first erroneous and misleading claim made on their forum goes like this:-
“Spending so many $$$ on TradeSim only to run Monte Carlo is not wise thing, when you can have the same for free……..”
No sorry, it’s not the same and here’s why !!
Not all Monte Carlo analyzers provide useful results.
The TradeSim Monte Carlo analysis is unique in that it provides a true portfolio Monte Carlo analysis rather than a Monte Carlo analysis based on time series shuffling or synthesis of trades using random generators and price distributions. When thousands of simulations are run in a TradeSim Monte Carlo analysis each simulation represents a real world portfolio-trading scenario, which uses real trades that occurred from the historical trade database. Unlike other software TradeSim does not synthesize trades by time series shuffling or price synthesis using questionable price distributions.
Looked at it another way when TradeSim runs a 10,000 run Monte Carlo simulation on the top 200 stocks for example it is like having 10,000 traders trading from the same portfolio or universe of securities, and using the same trading rules. If there are a large number of trades with the same entry dates then you would expect each trader to come up with a different result as each trader would not choose exactly the same trades even though they are obeying the same set of trading rules. Because of this capability we believe our Monte Carlo analysis is unique and stands alone compared to any other offering from any other vendor. To further undertsand this concept please read The Search for the Holy Grail Trading System : Part 2
Have you ever wondered why prioritization of trades using a PositionScore variable is used in AB ?? Quite simply without it they could not possible optimize their systems and produce a flashy 3D color chart. If you don't believe this then ask them to optimize a trading system where PositionScore = Random. Apart from being excrutiatingly slow it forces you to trade a certain way in order to take advantage of the gimmicks in the software and why should you do this ??
The second erroneous claim also made on the same forum goes like this:-
“I wonder why you insist on using TradeSim while AB has way superior portfolio backtester with features that Tradesim users can only dream about.
"Tradesim is for Metastock that does not have portfolio optimizer.”
Apart from the fact that TradeSim can be used with other Vendors charting Packages including there own, lets address the last point because the claim made about the "features that “TradeSim users can only dream about” is based on conjecture and opinion. The problem with making this claim is that there are many people still using their product with TradeSim to do backtesting even though as they claim there own product has "features that TradeSim users can only dream about".
The Portfolio Optimization Myth !!
Some vendors of charting and trading software offer trading system optimizers. Essentially an optimizer steps through a set of trading system values and runs a simulation for each set of values and plots a performance metric in a table or on a chart. From this it is possible to see which set of trading parameters produce the most optimum results. Apart from the issues of curve fitting and unrealistic expectations of trading a real world system there are some potentially hidden traps especially when it comes to trying to optimize a portfolio trading system.
We often get emails asking whether or not TradeSim offers optimization of a portfolio trading system. My response to this is that to do it properly you would have to run a full TradeSim Monte Carlo analysis every time you stepped a trading parameter. The reason for this stems from the fact that it is not possible to characterise a portfolio trading system with one simulation and set of results. This is because there are many different valid trading possibilities when trading a system from a universe or portfolio of securities. For example the two equity charts shown below obtained from consecutive simulations of the same trading system, same portfolio hence same trade database show that just by selecting different trades can have a significant impact on the final result !!! Which one do you use to determine the performance of the trading system ??
Depending on the complexity of the trading system and the size of the trade database you may have to run a 10,000 run Monte Carlo analysis each time you stepped a trading parameter. This may mean running 100 Monte Carlo runs or 1,000,000 simulations !! Even then, it is not a simple matter of inspecting a simple performance metric such as average net profit as each Monte Carlo run will be characterized by a complete set of statistics, data histograms and distributions. Again compounding the problem even further, is the fact that not all statistical distributions produced by TradeSim exhibit a regular distribution. Some distributions have double humps whilst others are completely irregular and trying to characterize this using a single metric is totally misleading and pointless !!! Even if you could run an optimization sweep how could you optimize a system ??
But wait one minute. They will tell you that you have to rank the trades and force a fixed sequence of trades so your portfolio analysis really behaves like a fixed single security backtest. But why not randomly select trades from a group of trades with same entry date just as is done in TradeSim Professional and Enterprise Editions ?? The reason why they don't do this is because as explained above they can't optimize their systems with random trade selection and the second reason is that the thing would take months to complete :( So instead of highlighting the limitations of their own software they proceed to discredit ours. Very clever marketing trick !! Hopefully people reading this will wake up to the scam ;)
In conclusion any vendor of a charting package who claims that there software performs portfolio optimization without proper statistical analysis obviously only runs one portfolio simulation for each set of trading parameters. Trying to characterize the performance of a portfolio trading system from the results of one portfolio simulation is totally misleading and pointless !!
Vendors who claim their software offers portfolio optimization
as some sort of fancy 3D color-coded surface chart either does not understand
the problem or are purposely misleading the public with their hyped up