Why market data is harder than you think

Overview

There are two kinds of traders: those who know data is hard, and those who are yet to find out.

At face value, getting data looks easy: download some prices and find an edge.

A quant shop is more likely to budget for rigorous data management and make someone responsible for it. Retail traders tend to choose a provider, get comfortable with its conventions, and build years of research around them, and reconsidering that choice gets harder every year.

The examples here come from Marketfeed, the data archive behind our trading platform, and include vendor conventions and mistakes in our own software.

A standard nomenclature

If you trade E-mini S&P 500 futures, ES probably seems like a sufficient name. Your charting package understands it, and so does everyone you discuss it with. But once you start moving data and orders between systems, you need to be a lot more specific. A ticker belongs to a namespace, and each provider gets to decide what its names mean.

We use class:venue:ticker as our standard nomenclature, in which fut:XCME:ES identifies a futures root on CME, using the Market Identifier Code (MIC) XCME for the venue. IBKR uses CME as its exchange label, and even that small difference is a translation you have to get right.¹

Including the asset class and venue lets us distinguish products that happen to share a ticker. An index and a future might have colliding names; the same ticker on two exchanges might identify different securities. You also need a way of specifying a particular futures expiry. It’s easy to postpone these decisions when your universe consists of a handful of instruments, but changing identifiers after they’ve spread through strategy configurations and historical position records is an unpleasant job.

Then there are the vendors: Databento, FirstRate, and TradeStation each have their own naming conventions. IBKR has several within the one integration: its underlying search symbol, trading class, and contract ID identify different things. The search might return a family of contracts from which you still have to select the product and expiry you want.

Micro Silver is a useful example:

For reliable support of multiple vendors, you need a common internal identity and an explicit mapping to each vendor’s namespace, which I consider a basic design requirement. How you represent that identity (a string, a registry key, whatever) matters less than the rule: the system must resolve an instrument using the vendor’s namespace and the required contract details, or report the ambiguity. Adopting a supplier’s ticker throughout your strategy code just makes that supplier’s assumptions harder to remove later.

The venue and the supplier need separate meanings too, because buying CME history from another supplier shouldn’t change the instrument’s identity.

Even class:venue:ticker needs a historical interpretation, because a security can change its ticker and an unrelated security can later inherit the old one. Including the venue doesn’t help when the reuse is on the same exchange. You need a persistent instrument identity behind the name, with mappings recording when each name was valid; a lookup against today’s symbols won’t do for a backtest of earlier years.

Order generation and order management

A common nomenclature also lets you decouple order generation from the backend order management system (OMS). The strategy expresses what it wants to trade using the canonical identity. The OMS keeps its orders and positions under that identity, while a broker adapter translates the outgoing instruction into the contract representation the broker accepts.

That gives you this round trip:

Order: strategy → order for a real canonical contract → OMS → broker adapter → broker.

Fill: broker execution → broker adapter → canonical contract → OMS position and strategy state.

A new broker’s naming conventions stay confined to its adapter. There will still be work on contract sizes and supported order types, and every broker has its peculiarities, but none of that needs to touch the strategy’s trading logic.

The return path deserves as much attention as the outgoing order. A fill has to update the correct position, including the correct expiry. Keeping the broker’s identifiers in execution records is useful for reconciliation, but you also need to translate the execution back into the identity the rest of your system understands. Otherwise, two integrations can leave you with two apparent positions in what is actually the same instrument.

Our Micro Silver mapping once used SI as its alias, the same as standard Silver, and a search on that underlying can return both products. The micro needs the SIL trading class to distinguish it, and failing to make that distinction left both contract selection and reverse lookup vulnerable to error. Standard Silver is 5,000 troy ounces; Micro Silver is 1,000. Routing a one-contract Micro Silver order into standard Silver would give you five times the intended exposure.^{2 3}

So an ambiguous mapping must fail, and so must a missing broker route or an expiry that isn’t listed, while an execution that comes back with an unrecognised identity goes to a human for review.

One more thing about fut:XCME:ES: it names the root, and a root can’t be traded. The backtester can run a strategy on a synthetic continuous series and generate signals from it. The strategy’s orders must still name a real contract, such as fut:XCME:ES:U26 for September 2026, and selecting that contract requires a roll rule. Marketfeed, our archive, supplies the dated contracts and never submits an order; the trading engine determines the rolls, and the separate OMS manages orders.

Timestamps

Suppose your one-minute bars are labelled with their opening time. The row marked 10:00 contains the high, low, and close from the minute that follows. If your backtester makes that completed row available at 10:00, it has given the strategy a minute of future information.

Databento labels its OHLCV bars with their opening time and our legacy archive used the closing time, so moving between them meant shifting each timestamp by the interval.^{4 5}

Vendor-local timestamps for a European instrument can turn out to be in New York time, which sounds implausible until you find that FirstRate Data does exactly this. A UTC label doesn’t guarantee much either, since a fixed offset may agree for months and then fail at daylight saving. The US and Europe change their clocks on different dates, and the weeks in between are the best time to compare imports from different providers.

We made the same class of mistake ourselves. Our futures session registry declares each venue’s timezone once, in a header, and lists session times beneath it. The NYMEX and COMEX headers say America/Chicago, because that’s where Globex lives, but the times underneath were written in Eastern, which put 17 energy and metals roots an hour out. Nothing complained, because 1800-1700 looks like a perfectly good session in either zone.⁶

Some sessions are anchored to a clock that isn’t the venue’s at all. ICE’s New York softs (sugar, coffee, cocoa) open and settle on London time. For the three or four weeks a year when the US and UK are out of step on daylight saving, ICE issues a notice moving the New York opening times by an hour so that they still match London.⁷ A single HHMM-HHMM in America/New_York is right for about 48 weeks a year and wrong for the rest. The registry has to carry the anchor timezone separately from the one used for display.

For slower data, the delay between observation and availability can be much larger. Commitments of Traders reports generally describe positions on Tuesday, with publication on Friday at 3:30 pm Eastern. If you put those figures against Tuesday in a backtest, then the strategy gets several days’ notice of information it couldn’t have had.

Even a rule that always moves the figures to Friday gets some weeks wrong. Over the last 16 years, 90 reports missed the usual Friday release. Holiday Mondays pushed some back; the government shutdowns and the ION cyber incident in 2023 held others for weeks,⁸ the worst by 47 days. A Friday rule stamps those 90 weeks, 11% of the rows, before they were published.⁹ The CFTC’s release schedule includes the exceptions.¹⁰

FRED dates each value by the period it describes, so a March figure sits on 1 March even though it came out weeks later. If you join on that date, then quarterly figures reach your strategy a median of 119 days early, monthly ones 34 to 68 days early, weekly ones 5 to 12 days early, and daily ones a business day early.⁹

Macroeconomic observations can be revised repeatedly after publication, and today’s download may contain a later estimate for an old quarter. Even delaying that figure until the original release date gives the strategy a version that didn’t exist then.

FRED’s default history gives you what is known today about the past. Its historical real-time periods and ALFRED’s vintages allow you to retrieve earlier versions. For intraday trading, you may need to go further and establish when the release reached the service you actually used; FRED distinguishes the original release date from availability on its own services.¹¹

Look-ahead bias introduced here can survive a very elaborate validation process. Walk-forward testing will repeatedly give the strategy data before it was available, just as the initial backtest did.

What do you mean by a daily bar?

If you ask two providers for daily data, then you can get different bars even when they agree about the underlying trades. An overnight futures session crosses midnight, so grouping by calendar date and grouping by exchange trade date produce different results. Databento’s own daily OHLCV uses UTC dates; if you want exchange-session bars, you need to aggregate a finer interval over the appropriate window.¹²

Providers handle holiday half-sessions differently: some report them separately, while others fold the activity into an adjacent trade date or drop the date altogether. That changes both the rows present and their open and range. Your previous-day-high signal inherits those choices; a session filter can’t regroup trades already compressed into daily bars.

And what does “close” mean: the last electronic trade, an auction price, or the exchange settlement? Settlement is what valuation and clearing use, and it needn’t equal the final trade in your chosen session. Any of them is a useful price, provided you know which one you’ve got.

The gap between the two is set by exchange rule, and it differs by product:¹³

If you take the “close” from the last bar of the session, then you’ll disagree with the price your broker marks you at. For the E-mini, the difference depends on an hour of trading, and on 45 minutes before September 2020.

If your roll rule uses volume to select a contract, the choice of series can change which contract wins. The publication time matters as well: a decision based on yesterday’s official volume has to wait until the figure is available.

Session hours change

Exchanges change their hours more often than most traders remember. The changes are made per venue and product family, by dated notice, rather than per contract. Live cattle is the clearest case in our archive.¹⁴

Our registry held one row per root, with today’s hours. Applied to 2013, that row keeps under a quarter of the tape and throws away the rest, including every overnight print. It went unnoticed because the session filter simply returned fewer bars.

CBOT corn, soybeans, and wheat were nearly as busy, changing session four times between May 2012 and July 2015 (twice within a single month), and the day session’s open moved from 9:30 am to 8:30 am along the way.

The Globex close for equity index futures moved from 3:15 pm to 4:15 pm in November 2012, and the 3:15 to 3:30 halt that came with it survived until June 2021. Eurex moved the open for its most liquid index and bond futures to 1:00 am CET on 10 December 2018; before that, every Eurex product opened at 8:00 am.¹⁵ A backtest of the DAX future over 2015 to 2020 under today’s session claims seven hours a day of trading that didn’t exist for the first four years.

In the registry’s RTH column, 36 of our 43 CME roots carried 0830-1600, which is the daytime portion of Globex and nothing more. The equity index day session that traders actually mean is the old pit session, 8:30 am to 3:15 pm, which ended at the settlement until CME moved settlement to 3:00 pm in September 2020. FX and rates haven’t had a day session worth the name since the pits closed. Any code that treated RTH as “the liquid part of the day” was using a placeholder.⁶

The NYSE traded on Saturday mornings too until 1952, so a year had about 300 sessions: Ken French’s daily market returns, built from CRSP, have 301 in 1935, and 51 of them are Saturdays. The exchange closed on Saturdays each summer from 1945, held its last Saturday session on 24 May 1952, and moved to a five-day week from 29 September 1952. To make up the time, it added half an hour to the weekday session, which then ran to 3:30 pm. It also closed for holidays it no longer observes: Lincoln’s Birthday, Columbus Day, and Armistice Day until 1953, and Election Day until 1968. From June to December 1968 it shut every Wednesday as well, to clear a paperwork backlog.¹⁶

Few traders who quote a backtest starting in the 1920s or 1930s could say how their data and tools handled those Saturdays, and the tools don’t agree. Yahoo Finance’s ^GSPC history for 1935 has 250 rows and no Saturdays, so each Saturday’s move is folded into Monday’s close-to-close return. The pandas_market_calendars NYSE calendar has all 301 sessions, with Saturdays running from 10:00 am to noon. exchange_calendars 4.13.2, the calendar library behind zipline-reloaded, gives a plain Monday-to-Friday year of 261 sessions for 1935, including New Year’s Day, Lincoln’s Birthday, and Election Day, when the exchange was shut. MATLAB’s nyseclosures assumes a six-day week before 29 September 1952 by default, while its holidays function assumes five. The business-day functions in pandas and numpy count 261, and performance libraries such as quantstats and empyrical annualise daily returns with 252 unless you tell them otherwise.¹⁷

Each of those choices changes a backtest without producing a single error. If you annualise 1930s daily returns with 252, then you understate the decade’s return by about a sixth and its volatility and Sharpe ratio by about 8%; we made exactly that mistake in the first draft of a study of a century of returns. A 200-bar moving average covers about eight months of 1935 on the six-day calendar and nearly ten on weekday-only bars. Saturday sessions were short, and their moves were about two-thirds the size of a weekday’s, so an average true range that mixes the two is pulled down. Day-of-week results change most: from July 1926 to May 1952, Monday’s average return was −19.6 basis points (t = −5.1) against +6.3 for Tuesday to Friday and +14.3 for Saturday. On weekday-only bars, where Monday absorbs Saturday, the same Mondays average −8.2 basis points (t = −1.7), and a strongly significant effect turns into noise.

So a session is a dated fact, and so is the set of days on which there is one. The registry needs an effective-from date, a timezone of its own rather than one inherited from the venue, the exchange notice that established it, and a separate settlement window, none of which fits in one row per root.

The most recent change is already in the data. CME’s Bitcoin and Ether futures have traded around the clock since Friday 29 May 2026, with a two-minute maintenance pause on weekday afternoons and a longer one early on Saturday. Trades from Friday evening to Sunday evening carry Monday’s trade date and clear on Monday.¹⁸ A row that says Days: Mon-Fri is now wrong for those products, as it was for the NYSE before 1952, and a daily bar labelled Monday contains Saturday’s trades.

For equities, Nasdaq has announced 23-hour trading, five days a week, from 6 December 2026, and NYSE Arca plans the same start, subject to some conditions. Nasdaq adds a 9:00 pm–4:00 am Eastern overnight session, with a daily pause from 8:00 pm to 9:00 pm. The regular 9:30 am–4:00 pm session and closing auction remain.¹⁹

The Sunday launch starts the trading day labelled Monday, 7 December. Nasdaq assigns trades from 9:00 pm to midnight to the next trading date; those from midnight to 8:00 pm carry the current date.²⁰ Midnight in New York doesn’t start a new trading day, as these timestamps from the launch week show:

Monday’s session spans two New York calendar dates and two UTC dates. In Melbourne it runs from 1:00 pm Monday to noon Tuesday AEDT, and because that conversion changes with daylight saving, a fixed offset won’t work all year.

Grouping by New York calendar date would split sessions and combine parts of different trading days. Grouping by UTC date would put Monday’s last hour into Tuesday’s bucket. Keep the execution timestamp for sequencing events and the exchange trading date for session membership.

Vendors can supply core-session bars, full-session bars, or calendar-day aggregates, and an opening-gap signal inherits whichever you use. A full-session open comes from the first eligible overnight trade; the official closing auction remains at 4:00 pm. And a completed full-session bar labelled Monday isn’t available until Monday’s 8:00 pm finish. If you give it to a strategy at the regular close, then you’ve supplied four hours of future information.

Session boundaries need separate treatment too. Nasdaq cancels outstanding overnight orders at 4:00 am even though the trading date continues, and a backtest must account for that cancellation when deciding whether a later trade could fill an order.²¹

Nasdaq’s proposed handling of certain corporate actions would halt affected securities before 9:00 pm on the evening preceding their effective date, then reopen them at 8:00 am. A split adjustment keyed blindly to your computer’s midnight can put observations on the wrong price basis.²²

Use a dated exchange calendar: for a Monday exchange holiday, Nasdaq’s FAQ puts the reopening on Monday evening. Keep core-session and full-session definitions explicit, and build those bars from minute data if necessary. Version the rule change so December’s hours aren’t imposed on earlier history. Verify each provider’s definition and coverage before the cutover; a field that’s still called date says nothing about what’s now in it.

Whatever the notices say, check them against your own bars. For each root and trade date, take the first and last minute with volume, in venue-local time. Track the modal boundary over a rolling 20-session window and record a change point when it shifts by more than a bar and stays shifted. The change points should land on the exchange’s effective dates. Where they don’t, trust the tape and go looking for the notice you missed.

A first pass over our registry found three roots whose extended session ended before their regular session did, which is impossible by definition. Extended hours contain regular hours; no traded bar falls outside the session for its trade date; each session leg is longer than zero and shorter than a day. Those are cheap to assert and they were all violated.⁶

Continuous futures prices

A continuous futures chart makes it easy to think of ES as one instrument with decades of history. Somebody, at some point, selected a succession of expiring contracts and joined them with a roll rule. They may then have adjusted the prices to remove the gaps between contracts.

For the same trade date, 10 July 2014, we had these three opening prices:

The actual session opened at 1967.50. FirstRate’s historical level had been shifted by 384.75 points; the Panama reconstruction carried a 626.00-point shift. The adjusted numbers include the effects of later contract transitions, and neither was the price at which that session opened.

With additive, or Panama, adjustment, earlier history is shifted to remove a gap at each roll. The cumulative shift depends on the roll rules and where you anchor the series. Our reconstruction used the end of the query window as its anchor, so a request ending in 2014 and one extending through 2026 could return different prices for the old session.

Adding a constant preserves point differences inside a uniformly shifted segment, but percentage returns change because the price in the denominator changes. Absolute price thresholds and price-based sizing can change as well. Ratio adjustment preserves proportional relationships within a uniformly scaled segment, while changing point distances.

A strategy can use synthetic prices to generate signals, provided its calculations make sense under that adjustment. Any resulting limit or stop level must be translated onto the real contract’s price scale. Simulated fills must use that contract’s market data too; the synthetic series never traded.

There were actual defects in the sample too, including inconsistencies in FirstRate’s adjustment ladder and differences between its minute and daily roll construction. Those findings refer to the files we examined, and the figures above shouldn’t be read as an endorsement of each method.

I prefer to keep the dated contracts and make the roll and adjustment rules explicit, so there’s enough information to rebuild the series and explain a discrepancy. A pre-stitched download is convenient, but it commits you to the supplier’s decisions (and errors!).

Adding up positioning

A Commitments of Traders report is organised by contract market: the E-mini, the Micro E-mini, the full-size contract while it lasted, the sector and mid-cap E-minis, and dividend futures, each with its own code. If you want one positioning figure for the index, somebody has to add those markets up. Our archive did, and got it wrong in three different ways.

The first was double counting: for some groups of interchangeable contracts, including the S&P 500, the Dow, and the Nasdaq-100, the CFTC publishes its own consolidated figure alongside the members. The consolidation is filed under the same commodity code as its members, so a sum over every market with that code counts the members twice. In our S&P 500 total, the consolidation made up a median 19% of open interest over 2010–2026, and 42% after May 2023. In the Dow total it was a third.²³

The second was units. A contract count treats a Micro E-mini, a tenth of the size, as though it were an E-mini, which is the Micro Silver mistake again, this time in your research. The CFTC’s consolidation does weight its members, but you need to know which unit it uses, and that changed. Until the report of 2 May 2023, the S&P 500 consolidation was stated in full-size contracts: an E-mini counted a fifth, and the micro wasn’t included at all. From that report on, it has been stated in E-minis, with a micro counting a tenth.

Open interest didn’t really grow by 2.6 million contracts that week. The Nasdaq-100 consolidation jumped fivefold on the same report, and the Dow’s doubled. An indicator normalised by open interest, or one built on week-on-week changes, sees a break the market never had.

The third was classification, since a contract’s CFTC commodity code isn’t permanent. Between 2010 and 2026, 25 contract codes moved from one commodity code to another. One rates contract moved in 2012 and one energy contract in 2014. In August 2025 the CFTC moved 21 crypto contracts to a new code, and moved most of them back in January 2026. And from June 2026, two dividend-index futures have been filed under the S&P 500 code.

A total that follows the CFTC’s code week by week jumps whenever the code does:

Our S&P 500 total rose 19% in the week of 2 June 2026. Of the 892,000 extra contracts, 719,000 were dividend futures the CFTC had re-filed under the S&P 500 code.

Every figure is the CFTC’s own, and the aggregate is wrong only because of how it was put together. Even the choice of report matters. The CFTC publishes futures-only figures and figures that include options, and they needn’t agree on direction: in 86 of 845 weeks, leveraged funds’ net position in Euro FX futures has the opposite sign depending on which one you read.²³

We’ve removed the double count, and we’re moving our strategies onto single, well-defined contract markets: the E-mini for the S&P 500, and the main contract for each commodity. If you need an aggregate, build it yourself: decide which contracts belong to it and keep that decision fixed, state the unit, and weight the smaller contracts by size. Then plot it and look for steps that the market can’t explain.

Equities are complicated too

If a stock splits four for one, the holder ends up with four times as many shares and a price basis divided by four. Adjusting the chart makes comparisons across the event easier, but an execution simulation must also keep track of quantities and pending orders.

Dividends are another way to get apparently good results with remarkably little effort. A simulation using raw prices can credit the dividend as cash. A total-return series already incorporates distributions into its return measure. If you feed an already dividend-adjusted return stream into a process that adds the dividends again, then you’ve counted the same benefit twice. It’s an easy mistake if the adjustment policy is hidden somewhere inside the platform.

Tickers change, and get reused

Meta Platforms changed its Nasdaq ticker from FB to META on 9 June 2022, keeping the same CUSIP for its Class A shares, and the stock’s history still extends back through its time as FB. Earlier that year, however, META identified Roundhill’s metaverse ETF on NYSE Arca, which changed to METV on 31 January. The bare ticker has belonged to two quite different investments.²⁴

Treating the Meta rename as the birth of a new security throws away valid history. If you join everything called META, then you can splice an ETF’s prices into a technology company’s record. Returns across that join are meaningless, and corporate actions can end up applied to the wrong security.

Reuse is rarer in futures, but CME did it with the TN product code. The 2010–2012 TN contracts, such as TNK1, carry security group TN, a February/May/August/November cycle, and expiries around the middle of the month. The Ultra 10-year contracts from 2016, such as TNU9, carry security group Z1. Both have asset TN, venue XCBT, a 1/64 tick, and $100,000 notional, so they’re easy to mistake for each other, although they’re different instruments. If your roll logic keys on the root alone, then it can quietly stitch the old contracts onto the front of the Ultra 10-year history.

This is why the reference data needs to remember which security used each name, on which venue, and when. A routine rename should preserve the connection to existing positions and orders; a new security needs its own identity. Our archive work required separate event ledgers and rename operations that kept bars, corporate actions, and instrument records connected. Changing the catalogue label alone would have stranded the old files.²⁵

Today’s surviving tickers may have excellent histories and still be the wrong universe for a historical backtest. Ask how your provider handles delisted securities before spending much time on an equity model.

Missing data and data cleaning

A missing bar tells you very little by itself. The exchange might have been closed, the instrument might not have traded, or there might have been a failure somewhere between the exchange feed and your local file. How do you distinguish them?

Databento omits an OHLCV record when no trade occurs, and we encountered a genuine one-minute aluminium series whose median spacing was two minutes because so many minutes had no prints. A classifier that inferred the bar interval from the median gap got it wrong.^{26 27}

Elsewhere in the migration, the missing periods were entire years: BFX had no rows for 2024, COMEX aluminium ALI lacked 2018, and Henry Hub HH lacked 2011. If you summarise coverage using only the first and last timestamp, those holes disappear from the summary. We had to record the separately covered periods.²⁸

If a feed stops in the afternoon, the morning’s observations can still produce a daily bar that passes routine OHLC checks, although its volume may be understated and its final price far from the session close.

Some defects are worse than a gap, because nothing is missing that you could notice. In August 2026 Databento published an issue affecting its CME Globex history before 21 May 2017, the period sourced from the exchange’s older MDP 2.0 feed. On a list of days between 2010 and 2015, the OHLCV schemas return far fewer bars than the number of intervals that traded, because most of the day’s activity was folded into a single bar. Databento tracks it internally as D-8175; it was still marked “Fix in progress” in mid-September 2026 and closed by 26 September, about 16 years after the first affected date.²⁹

From the archive’s side, each folded bar is well formed: the high is above the low, the open and close lie between them, and the volume is plausible for a whole day, because it is a whole day. A short day isn’t a warning either, because Databento legitimately omits intervals with no trades and a quiet market produces the same shortfall in the bar count. Most of the listed dates are the 15th of a month, which is a sensible thing to look for once you’ve been told. I doubt a validation rule would ever have found it. A one-minute bar carrying most of a day’s range invites a simulated fill at a price the market only reached hours later, and gives a volume-based session or roll rule a day it can’t interpret. The only reason we could exclude those dates was that the vendor said so.

Compare the exchange calendar and the vendor’s feed-condition reports with your own import logs. A holiday calendar can’t explain why your downloader stopped halfway through a Tuesday; a gap in your file doesn’t establish that the vendor lost anything.

Outlier filters need care too: in one Brent validation case, the front contract fell about 22% and a back month about 12%. Using the front month as the reference made the back-month move look suspicious, but nearby maturities corroborated it. Different parts of the futures curve can behave differently, especially during the sort of market you most need to understand before trading a strategy.³⁰

Removing those observations because they make the model harder to fit would be a mistake, and it’s exactly what a badly chosen cleaning rule does. Forward-filling a gap is a modelling decision too: the repeated value is something you’ve supplied. Both need a reason, and a record you can go back to later.

Combining vendors and resolutions

Long histories often come from several sources, so the oldest period might contain minute bars from one vendor, with second bars from another covering the later years. Trade-level records might exist for only part of the range. A single start date and a description saying “20 years of data” don’t tell you much about those differences.

An overlap gives you the opportunity to compare suppliers, but you still need a rule for deciding which observations your reader serves. Concatenating the files can double-count the same period. Filling gaps can be more subtle: if the source changes inside a bar you’re constructing, you need to know whether the contributions can legitimately be combined. You can otherwise create a bar that neither vendor delivered.

Compare the same dated contract, session, and interval, with matching timestamps, and check source boundaries and daylight-saving weeks. Agreement in the overlap gives some confidence in the older source, but can’t verify history for which no comparison exists.

Older futures data carries two warnings of its own. Databento’s CME history begins in mid-2010, so our earlier bars come from Tick Data and FirstRate, and those vendors applied their own session filters before we ever saw the files. Session-filtered bars tell you what the vendor thought the exchange’s session was, which is the very thing you’re trying to audit.

And before July 2015, when CME closed its futures pits, the electronic tape was a partial view of the market. Around 40% of live cattle volume was still going through the pit in the months before it closed.³¹ A session boundary derived from Globex bars in 2013 is the boundary of the electronic market: the right one for backtesting an electronically executed strategy, and the wrong one for a claim about what the market did. A volume-based roll rule on those bars is choosing on a minority of the trade. Data Quality: The Contaminated Foundation covers the pit, transitional, and electronic eras in more detail.

Be honest about the resolution, too: if an early period contains only one-minute OHLCV, a request for five-second bars should fail and identify the limiting period. Repeating or subdividing those observations creates prices for which you have no evidence. You can choose a coarser experiment or acquire finer data, but that should be a conscious decision by the researcher.

Keep track of sources and native resolutions even when you serve a common interval. Investigate a performance change at a vendor boundary before attributing it to a market regime. I’ve discussed why those resolution limits matter in The lie hidden inside daily bars.

Keeping the archive usable

You also have to keep the data you already acquired. During implementation review, we found a bug that could replace a yearly partition with only the newly fetched rows, silently discarding the rest. The fix merged old and new contents, kept recovery copies, and recorded replacement steps so an interruption could be recovered.³²

Checking the delivered file, decoding it, mapping its instruments, and storing it safely all have to succeed. Coverage records must follow the durable writes; otherwise a subsequent download can skip the lost period because your own records claim you already have it.

Because a completed job can leave individual sources stale, report which series advanced and which failed, and count newly acquired observations separately from rows already in the archive.

Our IBKR margin history depends on prospective captures. If we miss a day, today’s figure can’t fill the gap, and we’d need another source of historical evidence.³³

Keep a backtest you can replay

A backtest should be deterministic and replayable: given the same preserved inputs, software, configuration, and initial state, it should generate the same orders, simulated fills, positions, and P&L. If it uses randomness, fix and record the seed. Control event ordering and the runtime too: a seed won’t cure inconsistent ordering in parallel processing.

Give each run an identifier and retain the data snapshot or recoverable version, including reference data. Record the strategy and engine versions, parameters, initial positions and cash, session and adjustment policies, and execution assumptions. Preserve the order and fill log, with timestamps and the observations behind each decision.

The first check is to replay the saved run against its frozen inputs. If the result changes, investigate your own software or environment. Next, download the same historical range again, keep the old copy, and compare the records using identical query and normalisation rules. A hash can flag a difference, but a record-level comparison tells you which bars or reference values changed; different compression or file ordering needn’t mean different market data.

Run the frozen strategy against the new data as a separate experiment. Now you can trace a changed trade back to the changed observation that caused it. Corrections to genuine errors are welcome, but you still need to know which previous research results they affect. An unchanged backtest doesn’t prove the history is unchanged either; the strategy may simply be insensitive to the revised rows.

Automate this audit after every data update, so you can tell provider revisions from your own changes, although at best it proves only that the data hasn’t moved.

When the past gets corrected

Two issues updated on Databento’s tracker in the last fortnight concern CME data that is between 11 and 16 years old. Both were posted on 10 August 2026 and have been marked closed since mid-September.³⁴

One is the folded-bar problem described earlier (D-8175); the other, D-8174, covers 21 of the same dates, from 15 July 2010 to 15 September 2015: a request for part of a UTC date could come back truncated or empty. Databento’s workaround was to start and end every request on a UTC date boundary.

Most CME Globex products trade from 5:00 pm to 4:00 pm Chicago time, so the usual request for one session starts partway through a UTC date and ends partway through the next, which is exactly the request the issue describes. An empty response looks like a holiday and a truncated one like a quiet afternoon, and neither raises an error.

I’m glad these were fixed, and gladder still that they were published. But a fix rewrites history: anyone who downloaded those dates before the correction still holds the old version, and if you download the same range today, then some of your bars won’t match the ones your research used. The replay audit above is how you find out which.

The harder question is what you decided on the old data. Every backtest ends in a decision to keep the idea or drop it, and if you kept it, you probably tuned its parameters on the same bars, so faulty data can push the decision either way.

Some strategies should have been ruled out and weren’t. A folded bar puts a whole day’s range at a single timestamp, so a limit order resting anywhere inside that range gets filled in the simulation, at a price the market may only have reached hours later. A breakout rule gets an entry it could never have had. Trades like that tend to be winners, because the bar already contains the outcome.

Others should have survived and didn’t. The same bar can trigger every stop at once, handing a mean-reversion system a loss it wouldn’t have taken on the real tape. An empty or truncated day can drop a signal, or hold a position through a move it would otherwise have exited. The D-8175 list has 25 dates out of roughly 1,350 trading days in that span, which sounds harmless. But drawdown limits and worst-day filters live in the tails, and a couple of bad days is enough to fail an idea on maximum drawdown alone.

The two mistakes don’t cost the same. A strategy that was wrongly kept will eventually show itself when you trade it, which is expensive but visible, whereas one that was wrongly rejected never comes back. When did you last rerun an idea you binned in 2022?

So keep a research log with the decisions alongside the results: what you kept, what you rejected and why, and which data version each test used. When a vendor publishes a correction, check its dates against the log and rerun the affected tests, rejected ones included. Databento’s dataset conditions and modification dates make that check possible,³⁵ but most vendors give you nothing to check against.

What happens when you report an error?

Databento is a good example of a vendor that’s open about its data. It publishes data-integrity issues and exposes dataset conditions and modification dates through its API. You can identify dates with known degradation, follow an investigation, and decide whether affected research needs to be rerun.³⁵ The incomplete-bars issue above is the kind of notice that matters: a list of dates, a cause, a status, and an internal reference you can quote when you follow up. You still have to read it; nothing in the delivered files will tell you.

TradeStation, on the other hand, is a good example of how not to manage data issues. In 2020, I raised concerns about the integrity of its data, and I stopped using it. When I revisited the issue in 2024, four years on, nothing had been fixed. When pushed, TradeStation admitted the problem existed and offered a defence: “This problem is not an easy fix.”³⁶ I wouldn’t recommend that any trader rely on TradeStation’s data.

Other providers have acknowledged and corrected basic errors when challenged. That’s a better response, although such elementary mistakes raise questions about their quality assurance. The customer shouldn’t have to perform the vendor’s data integrity checks.

The cost of changing providers

Changing data providers is enough work that many traders never do it. That limits the strategies they can build and the markets they’ll consider. Often the provider was never really chosen, because it came with the platform or it was what everyone else seemed to be using.

Taking data seriously

A suitable provider and platform can do much of this work for you, provided you understand their conventions well enough to judge the results. Ask them:

• Identity: What exact instrument does this observation or order represent? How are ticker changes and reuse handled, alongside vendor translations and returning fills?
• Construction: Which session produced these values, in which timezone and effective from when? Which close, volume definition, roll rule, and adjustment policy? Does the tape agree with the stated session? Which contracts make up an aggregate, and in what units?
• Availability: Could the strategy have received this version of the observation at the simulated decision time?
• Coverage: Which periods and resolutions are actually present, and how are closures, inactivity, and failures distinguished?
• Reproducibility: Can you replay the saved run exactly, and identify which observations changed after a fresh download?

Data Quality: The Contaminated Foundation goes into the broader catalogue of errors. The problem for a strategy developer is deciding which of them the chosen tools deal with, and which remain the developer’s responsibility.

Retail traders can adopt this discipline on a smaller scale. Keeping recoverable data and testing a second source early gives you a chance to find problems before the cost of moving becomes an excuse to tolerate them.