8. Framework Monoculture and Methodological Homogeneity

Off-the-shelf backtesting frameworks are not neutral tools. Each framework embodies a set of assumptions about how strategies should be structured, what data they consume, how signals are generated, and how execution is modelled. These assumptions are absorbed by users through documentation and example code, and they produce a characteristic sameness in the way that problems are framed and solutions developed.

Most popular frameworks, for example, encourage a pattern in which a strategy is expressed as a series of indicator calculations followed by threshold-based entry and exit rules, evaluated bar-by-bar on a fixed-frequency time series. This pattern is natural for a certain class of strategies but is poorly suited to others: event-driven strategies, strategies that operate across multiple timeframes simultaneously, strategies that incorporate non-price data, or strategies whose execution logic is itself a source of edge. When the framework’s implicit recipe does not accommodate these approaches, developers tend to adapt their ideas to fit the framework rather than the reverse, leading to a narrowing of the strategy space that is explored.

The more experienced users of a given framework develop conventions and “best practices” that are passed on to newcomers through tutorials and forum discussions. These conventions often reflect the limitations of the framework as much as any principled methodological choice. The community that forms around a given framework ends up approaching strategy development in remarkably similar ways: using the same indicators, the same entry patterns, the same default parameters, and the same evaluation metrics. Strategies that emerge from such an environment are correlated with one another. This is a problem for both individual traders (whose strategies may be crowded) and for the field as a whole (whose collective output represents a narrower exploration of the strategy space than its volume would suggest). And, since crowded strategies tend to unwind at the same time, the correlation creates systemic risk that no individual backtest can detect.

A concrete illustration of this phenomenon can be found in platforms such as TradeStation, which has for decades been one of the most widely used retail algorithmic trading environments. TradeStation’s EasyLanguage scripting system and its built-in strategy development workflow are, by design, optimised for a specific class of strategies: indicator-driven, single-instrument, bar-by-bar systems evaluated on fixed-frequency data. The platform makes this class of strategy extraordinarily accessible: a user with modest programming experience can have a moving-average crossover system running within hours of first contact with the software. But the platform’s strengths are also its constraints. The ease of building indicator-based systems within TradeStation’s framework implicitly discourages approaches that do not fit its paradigm: multi-instrument portfolio strategies, strategies with complex position-sizing logic, strategies that incorporate non-price data, or strategies whose execution model departs from the platform’s built-in assumptions. The tool shapes the work, and when thousands of users adopt the same tool, the collective output converges toward the same narrow set of approaches, evaluated under the same set of assumptions, producing strategies that are correlated not because the underlying market relationships demand it but because the platform’s architecture channels all users toward the same destination. This is not a criticism unique to TradeStation (it applies, in varying degrees, to every framework that makes certain approaches easy and others difficult) but TradeStation’s longevity and large user base make it a particularly visible example of how platform constraints become community conventions.

It should be noted that many people over the years, including myself, have spent a considerable amount of time trying to add band-aid solutions (resorting to DLL callouts, implementing a client-server architecture to access TradeStation from outside its walled garden, mis-using the TradeStation Optimizer API to improve the user experience, building cut-and-paste code fragments to export data for processing outside TradeStation, and much more). The degree to which users of the TradeStation platform have used their ingenuity to try to work around its many shortfalls is a testament to their tenacity, but also an indicator of a dangerous blind spot: traders who become proficient on a particular platform cannot overcome the inertia of moving to a new platform, even if it is obviously better. In the brave new world of AI, it is even more imperative for traders to critically assess the relevance and competency of their platform to ensure that they are not left behind. My working assumption is that whoever takes the opposite side of any trade I place is probably smarter than I am, so I don’t want to make it any harder than it already is by bringing the trading equivalent of the knife to a gunfight. To be fair to TradeStation, every platform has its limitations. The software development experience on most (all?) of the widely-available platforms is extremely poor. Some (including me in this paper!) will argue that it’s not all about the platform. That’s true, but I see no reason to operate on inferior tools. In my case, I have built my own backtesting platform. (For almost all traders, I would strongly recommend against going down that path!)

Other platforms have recognised specific instances of these problems and introduced partial mitigations. TradingView’s “Bar Magnifier” mode exists because the OHLC sequence ambiguity described in Section 4.2 is a real and widely encountered failure: the feature uses finer-resolution data to resolve intra-bar order fills during backtesting, an implicit acknowledgement that standard bar-by-bar evaluation is insufficient. TradeStation and MultiCharts offer an “Intra-Bar Order Generation” (IOG) mode that evaluates orders within each bar rather than only at bar close, addressing the same class of problems through a different mechanism. QuantConnect’s LEAN engine takes a more explicit architectural stance, pushing users toward finer-resolution data for realistic fills and exposing custom slippage and margin models as first-class configuration options. That these features exist at all confirms the severity of the underlying problems; that they are options to be configured rather than default behaviour confirms that the retail backtesting ecosystem continues to prioritise ease-of-use over simulation fidelity.

The constraints in TradeStation’s case go beyond conceptual preferences encoded in documentation and examples; many are hard limitations enforced by the platform’s parser and runtime environment. EasyLanguage does not permit orders to be placed inside loops, eliminating the possibility of dynamic pyramiding or algorithmic scaling into positions. Named orders must be string literals rather than dynamically constructed expressions, preventing programmatic management of multiple concurrent entries. The platform cannot query how many data series have been provided to the strategy, making it impossible to write strategies that adapt to an arbitrary number of instruments. Variable back-references are subject to hard-coded limits that constrain lookback calculations. Complex data structures (maps, trees, graphs, and the arbitrary nesting of collections that modern languages provide as primitives) are effectively non-existent. Portfolio-level support is extremely limited, meaning that strategies are evaluated in isolation from one another with no mechanism for cross-strategy risk management or capital allocation, a limitation that directly reinforces the signal isolation problem discussed in Section 9 . And there is no facility for custom optimisation metrics, so the researcher is constrained to the platform’s built-in fitness measures when searching the parameter space.

Each of these limitations individually forces a design compromise; collectively, they make entire classes of strategy design impossible within the platform. A trader who has spent years developing within these constraints may not recognise what they have never been able to try. The strategies they build are shaped, to a degree that is difficult to appreciate from inside the ecosystem, by what the tool permits as much as by their ideas about markets. This is the mechanism by which platform constraints propagate into methodological homogeneity: not through explicit instruction, but through the gradual, invisible narrowing of the design space to the subset that the tool can express.