METHODS AND CHART DATA
Companion to “Looking beyond price for signals”
Post dated 3 November 2026

These are descriptive checks on an archive, not tests of trading profitability.
The downloads below reproduce selected counts, percentages, and the illustrative
supplemental TSA threshold example. They do not independently authenticate the
source pages, establish the accuracy of the news classifier, or reconstruct
every chart dot.

DOWNLOADS

Download chart-data.zip for this note and all the evidence files in one archive.
Individual files are beside this note at https://ohlcv.io/posts/alternative-data/:

  availability-summary.csv   20 series: counts, lag summaries, and evidence types
  tsa-example.csv            Both stored vintages for travel date 1 June 2020
  tsa-source-captures.txt    Dated TSA page extracts with public capture URLs
  tsa-2020-comparison.csv    294 eligible 2020 dates: earliest and latest values
  tsa-yearly-summary.csv     Yearly revision summaries for the eligible TSA dates
  news-grades.csv            All 61 subjects, including pending and grader counts
  silver-pageviews.csv       Daily pageviews, December 2020–February 2021
  news-keywords.txt          Deployed subject keyword entries
  news-screening-rules.txt   Deployed exclusions, context rules, and subject scopes
  news-relevance-prompt.txt  Classifier prompt, before subject scopes are appended
  verify-evidence.py         Python 3 arithmetic checks; no third-party packages
  evidence-sha256.txt        SHA-256 hashes for these evidence files

Download those files into one directory and run:

  python3 verify-evidence.py

The hashes identify this evidence package; they do not authenticate the original
publishers. The TSA source extracts link to independently inspectable captures
for one travel date; they do not verify the entire reconstruction.
The underlying ingestion code and larger source archive are not part
of this public package. COMEX settlement data is not redistributed here.

SNAPSHOT AND FIELDS

The research notes record that vintage files and the CoT release calendar were
copied from Marketfeed on 3 November 2026. The Wikipedia CSV records a fetch on
that date. The news count CSV and silver settlement CSV do not record their
original extraction times; no precise time is inferred from file modification
dates. Public extracts are derived from those saved inputs, not a fresh query.
“Latest” means the last vintage in this snapshot, not the live publisher today.

Most source vintage rows contain:

  ref             observation date or period reference
  realtime_start  vintage date used by the store
  value           numeric observation
  available_at    stored availability instant in UTC
  basis           evidence used to assign that instant
  late            flag for an initial sighting too late to use as a first release

ALFRED rows contain only ref, realtime_start, and value. A stored available_at is
not necessarily an observed publication instant. The basis must accompany it:
source timestamps, inferred dates/times, and capture bounds have different force.
In particular, a first archive capture proves that a value existed by then; it
does not prove when the publisher changed it or that no earlier vintage existed.

AVAILABILITY CHART

For each reference period, take its earliest stored vintage BEFORE filtering.
Exclude the date if that first vintage has basis=capture and late=1; do not let a
later revision stand in for a missing first release. This removes 440 dates in
the plotted window, all from the TSA series. The other inputs include estimates
and should not be read as a census of independently verified first releases.

Reference dates range from 1 January 2016 to 31 October 2026, or the available
series start. The availability cutoff is 3 November 2026 at 00:00 UTC. Monthly
and quarterly references are mapped to the last day of the period. The baseline
is 00:00 America/New_York on that day, including daylight-saving changes. Lag is
(available_at - baseline) in elapsed seconds divided by 86,400. It is NOT delay
from the close of trading or from the end of the observation day. A same-day
afternoon publication therefore has a positive lag of several hours.

ALFRED's date-only vintages are assigned 08:30 New York; this is an assumption,
not evidence of the actual hour of every historical revision. CoT calendar dates
are assigned the usual 15:30 New York release time. Rules and calendar-derived
stamps, including atlas-derived dates, are grouped as estimates in the chart.
Rows with basis=source are shown as publisher timestamps; basis=capture as
capture upper bounds. The summary CSV gives counts in each group. The mix varies
substantially by series, so precise-looking medians do not imply equal evidence.
Fourteen of the 20 series use estimates throughout. Across 23,791 observations,
19,091 (80.2446%) use estimated timestamps, 4,677 use publisher timestamps, and
23 use capture bounds. This is largely a release-calendar reconstruction, not a
measurement of actual historical feed delivery. Away from daylight-saving
changes, a 33-hour lag means 09:00 on the following day: nine hours after the
observation day ended.

TSA REVISION CHART AND SUPPLEMENTAL WORKED EXAMPLE

For each travel date, sort vintages by available_at and compare the earliest
value with the latest value. Keep dates whose earliest vintage has late=0:
2,419 dates in this snapshot, from March 2020 to October 2026. This is a coverage
filter, not proof that every retained row was published at its assigned time.
The earliest eligible vintage can still be a capture or a scheduled estimate.

For 2020, 294 dates pass the filter. The change is 100*(latest/earliest - 1).
Its median is 5.864593% and its maximum 31.232573%. The upper chart shows centred
seven-observation means for April–December 2020; this smoothing is retrospective
and is not a tradable feature. The lower chart plots every eligible daily change,
including negative changes. No causal explanation for the revisions is asserted.

For 1 June 2020 the store contains:

  353,261 at 2020-06-02T13:00:00Z, basis=schedule
  379,146 at 2024-01-06T02:48:53Z, basis=capture

The first instant is the standing 09:00 New York release rule. The second is an
archived observation of the later value, not its exact revision time. In the
store the latter row also has realtime_start=2024-01-05; the example orders by
the explicit UTC available_at, not that date-only field.

For this revision, two public TSA source pages were independently retrieved and
their table rows checked. They are transcribed in tsa-source-captures.txt:

  353,261 in the 2 June 2020 capture at 12:54:49 UTC:
  https://web.archive.org/web/20200602125449/https://www.tsa.gov/coronavirus/passenger-throughput

  379,146 in the 6 January 2024 capture at 02:48:53 UTC:
  https://web.archive.org/web/20240106024853/https://www.tsa.gov/travel/passenger-volumes/2020

The earlier capture is 08:54:49 New York, already showing the value before the
09:00 schedule estimate stored in tsa-example.csv. That row retains the original
estimate used in the chart calculations; the source check does not turn it into
an observed release time. Both captures bound availability, without establishing
the exact release or revision instant or recovering all intervening versions.

In this supplemental worked example, the threshold “daily count >370,000” is
chosen solely to illustrate the two errors. Applying the latest value on its
reference date triggers on 1 June. Moving that same value to the scheduled
release date triggers on 2 June.
Using the earliest archived value at the scheduled release does not trigger.
This demonstrates a changed decision, not a profitable strategy, an executable
trade, or an estimate of the overall size of backtest bias. The threshold was not
pre-registered, and there is no out-of-sample test or return series in this example.

The historical TSA reconstruction uses Wayback captures; the seed's notes say
2,457 captures through 1 November 2026. Sparse captures bound what can be known.
A source-page hash can detect a mismatched download but cannot fill gaps between
captures or establish that the first surviving value was the original release.

NEWS RELEVANCE CHART

Window: 1 February–30 September 2026, 242 UTC days for each of 61 subjects.
For each subject and day, select the newest saved retrieval with mode=articles
and source=gdelt-bigquery-v2, then count its article entries by relevance grade.
This is a retrospective selection of the latest saved retrieval. It does not
show precisely what the news feed contained on that historical day.

The unit is an article–subject match within a daily retrieval, not a globally
unique story. Subjects overlap, including micro and standard versions of the
same market. The same URL can count for several subjects; distinct syndicated
URLs can also represent the same story. The overall percentage weights subjects
by their number of matches, rather than taking an equally weighted market mean.
The graphic shows 20 selected subjects; its overall total uses all 61.

The query searches GDELT's English-language Web NGrams records for configured
keywords, then joins article metadata. Screening rules assign grade 0 when no
keyword hit survives exclusions/context requirements, or the domain is blocked.
The remaining matches are assessed using the configured model alias
claude-haiku-5-5 and prompt relevance-v1. The input is headline, site, GDELT topic
tags, hit counts, and keyword-context snippets, not a reading of the full article.
Subject-specific scope text is appended to the downloaded prompt. The provided
keyword, screening, and prompt files are the deployed configuration inspected on
3 November 2026; the count CSV does not retain per-row configuration versions.

Grades: 0 unrelated; 1 material market context; 2 about the subject's market.
An owner label can override a rule or model grade, but this extract has none:

  Grade 0                 8,287,148
  Grade 1                 3,061,300
  Grade 2                   919,457
  Graded total           12,267,905
  Pending, excluded           4,057
  Graded by rules         5,305,341
  Graded by model         6,962,564
  Graded by owner                 0

The reported unrelated share is 8,287,148 / 12,267,905 = 67.551452%, rounded to
68%. Pending matches are excluded from both numerator and denominator. This is
the pipeline's assessment, not a human-validated estimate of relevance. No
labelled evaluation set, precision/recall estimate, or inter-rater check is
provided. Counts also depend on keyword breadth, the coverage of GDELT, and the
choice of subject scopes; they cannot be generalized to all financial news.

The prompt asks the model to assess topical relevance without later market
knowledge. This is a restriction on the task, not a demonstrated cure for
historical leakage. The saved model alias does not establish its training cutoff.

SILVER PAGEVIEWS CHART

Series: English Wikipedia, Silver_as_an_investment, all-access, agent=user,
daily granularity, fetched 3 November 2026. Display: December 2020–February 2021.
“User” excludes traffic identified as bots; undetected automated traffic remains.
The prior baseline for 28 January is the median of the 28 measurement days from
31 December 2020 to 27 January 2021: 235.5 views. The count is 635 on 28 January,
1,810 on 31 January, and 3,088 on 1 February. The ratios are 2.6964 and 13.1125
times baseline on 28 January and 1 February respectively.

These dates label UTC measurement days. Midnight closes the measurement window;
it is not the API publication instant. Wikimedia says loading usually takes a
few hours and can take 24 hours or longer. No archived API availability evidence
for this 2021 episode is supplied. Wikimedia has also corrected historical data
after processing errors, so current pageview history is not guaranteed immutable.
References:
https://doc.wikimedia.org/generated-data-platform/aqs/analytics-api/documentation/troubleshooting.html
https://lists.wikimedia.org/hyperkitty/list/analytics@lists.wikimedia.org/thread/AXABVANPKBPKJTZVKYVJW6ZA2VMWA2WV/

The price panel uses unadjusted front-contract COMEX silver settlements by trade
date, exported with mf-view --daily-close settle --adjust none. A trade date is
not an observation timestamp. The episode was selected after the event and is a
timing illustration, not evidence of predictive lead or a strategy test.

PROPOSED GAS-STORAGE EXPERIMENT

The article summarises this experiment, which has not been run. The full
specification, which would be fixed before any evaluation, is:

  Target          Lower 48 weekly net change in working gas, billion cubic
                  feet, as initially published in the EIA report. The original
                  release is retained alongside later revisions.
  Forecast time   09:00 America/New_York on the scheduled release date. The
                  forecast and the input versions are recorded at that cutoff.
  Baseline        Mean initially reported change for the same ISO week
                  number of the reporting week's ending date in the preceding
                  five years. Week 53 is left out of this first experiment.
  Added inputs    Heating and cooling degree days and total gas-fired
                  generation for the reporting week, in the versions
                  available by the cutoff. Regions, weather weights, and
                  generation coverage are chosen before evaluation.
  Augmented       Baseline plus a linear regression estimate of (initial
  forecast        announcement - baseline) from the added inputs, fitted only
                  on earlier releases within a fixed five-year rolling window.
  Comparison      Both forecasts recorded for the next 52 eligible releases
                  and evaluated on the same weeks. The specification is held
                  fixed and those outcomes are kept out of model tuning.
  Measure         Mean absolute error in billion cubic feet for each forecast,
                  their difference, and the weekly errors, so the effect of
                  unusual releases on the average is visible.

History. Each training week's baseline uses its own preceding five years, so
fitting five years of baseline errors needs about 10 years of target history.
The weather and generation inputs are the versions available at each week's
cutoff. If less history can be recovered, a shorter window, chosen before
evaluation, may suit better. Where old input versions cannot be recovered, a
new collection can support tests on future releases instead.

Exclusions and changes. Incomplete weeks are excluded from both forecasts, and
the omissions are recorded with the results. Any change to the inputs or the
model starts a fresh evaluation period.

Trading. A trading test would come after the forecast comparison as a
separate step, with an expectations measure, an executable entry time, and
transaction costs.

WHAT REMAINS UNTESTED

There is no claim here that the assembled data adds returns after costs. A useful
next test needs a pre-specified signal, contemporary information sets, a baseline,
out-of-sample evaluation, conservative availability assumptions, and plausible
execution costs. Preserving vintages makes that test possible; it does not supply
its result. The research hypotheses in the article remain hypotheses.
