Does the party out of power reach for evidence more often than the party in power? We built the instrument that can answer it, and then asked whether the instrument survives twenty-eight years of changing language.
Stop 1 of 5
Congressional hearing transcripts are prose. Before anything can be modelled, every speaking turn has to be attributed to a person, split into sentences, and joined to what we know about that person on that day.
A real speech from the corpus, at each stage of the pipeline.
The two eras are built by the same code and kept in separate corpora, so the modern labels can never be contaminated by the historical ones.
Stop 2 of 5
A model can only be as sharp as the construct it is taught. Two coders read the same sentences in context and marked whether the speaker was appealing to empirical evidence. Where they split, a third pass settled the call.
These are sentences our two coders actually split on. Call it yourself, then see what they said.
Stop 3 of 5
Three encoders crossed with three class-weighting schemes, each run over five seeds with grouped five-fold cross-validation.
At the untuned 0.5 cut-off, before any threshold search. Whiskers span one standard deviation across seeds.
Macro-F1 against the decision threshold, swept on the ensemble's pooled out-of-fold predictions.
Stop 4 of 5
Held-out sentences the ensemble never trained on, with the score beside the coders' call. Open one, then type your own.
Live
Live scoring is off on this copy of the page. The scores above are the published ensemble's.
Stop 5 of 5
With every sentence in the corpus scored, the question becomes a regression: does a legislator in the minority reach for evidence more than one in the majority, and has that changed over twenty-eight years?