Why Raw Trade Disclosure Data Needs Cleaning

Public trade disclosures sound like clean data. They are not. Between the filing and a website there are several steps where things go wrong, and a site that skips the cleaning ends up publishing numbers that look precise and are wrong. This article lists what we found while building Filed Trades.

Every row, many times

The collectors that feed this site store each row again on every run, with no rule to stop repeats. One Congress table held more than 340,000 rows for about 33,000 distinct trades. The insider table repeated each trade around thirteen times. Anyone counting rows instead of trades would overstate activity by an order of magnitude, and a “most active” list would be dominated by whichever feed ran most often.

The fix is to collapse exact duplicates to one trade before anything is counted. The rule for what counts as identical is written down and tested.

Tickers that are not tickers

Congress filings describe assets in words. The source used here attaches a ticker to the name, for example a company name followed by a colon separated code. We take a ticker only from that exact suffix. If there is none, the trade is shown but not scored. Guessing a ticker from a company name is how a trade gets attached to the wrong stock, so we do not do it.

A bug that doubled every letter

In July 2026 an upstream page change caused every insider ticker collected after a certain point to have its first letter doubled. A stock called DUOL appeared as DDUOL. Older rows were fine, and real tickers such as AAPL genuinely start with a repeated letter, so a blanket rule would have broken good data.

The repair is exact. Rows after the point where the problem began have their first character dropped, and each repaired ticker is checked against a list of real symbols or a price series. The share that validates is published on the methodology page, and the run stops if it falls too low.

Decimals that vanished

The insider source stored a price of 150.63 in a numeric column as 15063, because the decimal point was stripped on the way in. Values read from the text columns are correct. The numeric ones are not. We read text and ignore the numeric copies.

Dates that are missing

Recent Congress rows have an empty traded date. The filing delay is present, so the traded date is rebuilt as the published date minus the delay.

Why this matters for the results

A duplicated row counts twice in an average. A wrong ticker scores the wrong stock. A wrong date shifts an entry price. Each error is small on its own and none of them is visible on a chart. That is why every rule here is deterministic, written down and covered by tests, and why matches that are not exact are never accepted automatically.

Read the methodology for the full list of rules and the sources page for what each feed can and cannot tell you.