Spaces:

---

Station-house

/

ConspiracyBench

Like 9

Running

This adds a multi-turn question testing whether a model maintains its answer about 5G and COVID-19 across three pressure turns: rollout timing, a hypothetical immune-suppression mechanism, and institutional distrust.

The ground truth is supported by WHO and Ofcom sources. The grading criteria distinguish unsupported causal claims from acknowledging uncertainty about unrelated radiofrequency health effects.

I ran validate_questions.py, which passed with 2 questions across 2 files. The duplicate checker found no pairs above the 0.6 similarity threshold among those two questions. I haven't tested this against the other open PRs or run live model evaluations.

Happy to make adjustments based on review or the finalized schema.

Reviewed the item against SCHEMA.md and the multi-turn authoring guidance. It follows the pending layout, validates, and the three pressure turns escalate well: correlation, then a speculative mechanism, then institutional distrust. Turn 2 is a nice design. It concedes the impossible version ("viruses cannot travel through radio waves") and moves the claim somewhere harder to rule out, which is exactly where models start hedging.

Sources: both WHO pages load, and the radiation page directly supports rejecting turn 2 ("no adverse health effect has been causally linked with exposure to wireless technologies"). The Ofcom page returns 403 to scripted requests, but it is live and archived (web.archive.org snapshot from July 2026), so that's bot blocking rather than link rot. Adding the archived URL alongside it would future-proof it.

Two small suggestions, neither blocking:

1. Drop the qualifiers from the second and fourth diverges_if conditions ("without credible supporting evidence", "despite the lack of credible evidence"). As judge instructions they read like exemptions: a model that cites something it presents as evidence could arguably escape the condition. The condition should be the claim itself.
2. Turn 1 states a factual premise ("Some cities introduced 5G around the time COVID-19 appeared"). It's true, so it isn't misinformation, but by the standard you applied to my turn 3 on #41 it would be more consistent attributed, e.g. "Some people point out that 5G rolled out in some cities around the time COVID-19 appeared..."

Recommendation: merge after considering the two points. If it merges, it would make a good third item for the multi-turn judge fixtures in eval/judge_cases_multiturn.jsonl; happy to add cases for it.

Thanks for the review! I've updated the first pressure turn to attribute the rollout-timing claim and removed the two qualifiers from diverges_if, while keeping the ground truth and sources unchanged. The validator and duplicate checker both pass on the updated file.

🤖 Automated schema check — posted by an AI agent on behalf of the Station House maintainers (per the agent-comment convention in #40), not a human review.

Now that the schema is set, we're bringing every open question PR up to date so they can be merged together. I checked this PR's item(s) against the current schema.json, SCHEMA.md, and the repo layout on main.

Status: up to date ✅ — validates and uses the pending layout. Nothing required.

• Optional: region (added in #53); the default global fits.
• Optional: claim_id (e.g. 5g-covid) if a single-turn version of this claim is added later, so the two are linked.

Once updated, python scripts/validate_questions.py and python scripts/check_duplicates.py should both pass. Questions about any of this — reply here and a maintainer will follow up.