The same four, as your research group.
Give them a research question instead of a syllabus and the team runs a study: a folder of ordinary files, a protocol adapted from CommDAAF, three agents on three model families analyzing independently, a fourth reviewing adversarially, and three checkpoints only you can pass — scope and tier, the data summary, the verdict. Finished studies are published on this site with their disagreements showing.
Necessary, better, urgent. Three reasons to automate research, and one honest line around them.
The professor this team works for made the case in an essay and a talk; the research pillar is built to it. It covers the whole lifecycle — data, analysis, review, output — and the field's methods: network analysis, NLP, survey experiments, simulation, bibliometrics, content analysis among them. Three arguments, a boundary, then the specification as built.
We cannot afford the hours
We know what the work is and how to do it. It costs more hours than anyone will pay for. Automation lowers the cost of hours.
It reasons like a trained researcher
Frontier models design studies, choose methods, run and read models, synthesize literatures and draft — at graduate-student level or better. They don't tire, they run in parallel, and they improve with every study.
The field moves faster than we study it
Findings go stale before they are published. A team can re-run the study on every new wave of data.
1 · NecessaryNobody can afford to do the research
This argument has nothing to do with whether AI is smart. Four things the field says it wants and doesn't do — in every one, the work is known and the hours are not paid for.
We can't look at the data we already have
Platform access has been withdrawn, but the field collected enormous amounts of it while it was open. Two years old, four years old: recent enough to matter, old enough that journals don't want it. Doing it properly takes a team small departments don't have. So we campaign for data we can't get while ignoring data we already have.
We can't write down our own ideas
The click was never the hard part. The hard part is the afternoon that turns the click into a paragraph and the paragraph into a paper — and for most working academics, that afternoon is what disappeared. A system that captures, connects and drafts isn't having your idea. It's doing the work between your idea and a finished version of it.
We can't tell anyone what we found
Our writing is hard to read on purpose, then paywalled. Machines are good at exactly this job: same finding, five formats, three registers, more than one language. It's publicly funded knowledge reaching the public. One caveat, built in: readable isn't understood. Measure comprehension, not reading level.
Some people can't do research at all
Research teams in global-majority countries publish less — but not per researcher. Divide output by headcount and the numbers are comparable. The gap is how many researchers a country can employ. If the gap is headcount, a technology that multiplies labor isn't an approximation of the fix. It's the fix.
2 · BetterThese models reason about research at the level of a trained researcher. Say so plainly.
Current frontier models — Claude Fable 5, GPT-5, Gemini Ultra, Kimi K3 — reach human-level research intelligence on a wide range of social science tasks, not only the mechanical ones. Given a question and a dataset, a frontier model chooses a defensible method, fits the model, reads the diagnostics, places the result in a literature and drafts a paper a reviewer would take seriously — like a well-trained graduate student or early-career researcher, and on some steps better. That is the honest starting point, across the whole pipeline and any of the field's methods; content analysis is one among many here, not the center.
Research judgment, not just execution
It knows a count outcome wants a negative binomial rather than OLS, that a network too sparse for community detection should be reported as such, that a failed manipulation check voids the hypothesis test, and that a topic model with a good coherence score can still be empty. It can read two hundred abstracts and tell you which three disagree with your premise. These are the calls a methods seminar exists to teach.
The same researcher at hour eight as at hour one
Human researchers fatigue, drift, and start seeing what they expect to see; the fortieth robustness check gets less attention than the first. A model gives the four-thousandth item, the twelfth specification and the last section of the review the same attention as the first, with the whole protocol — codebook, pre-registration, analysis plan — in view for every decision.
The whole pipeline at once
The literature review runs while the data pulls. Five specifications are tried side by side instead of one per semester. Three model families analyze the same data independently, and the reliability check a lab schedules for next month runs the same afternoon. A study arrives with its alternatives showing.
It gets better every time it runs
After every study the agents record what worked and what failed and propose additions to the guardrail; the guardrail compounds. A human researcher improves when they happen to notice and write it down. Here noticing is a step in the protocol, and a ledger says whether the tenth study was better than the first.
Not because the models are weak. Because the stakes are high, and strong reasoning benefits from adversarial review — the same reason human researchers need peer review. Several models run the study, then play Reviewer 2 on each other; agreement is confidence, disagreement is where to look. What the review catches is specific: a number no file produced, a comparison without a test, a claim that outruns the design, a minority warning averaged away, one model's blind spot another doesn't share. Those are the failure modes of good researchers working fast.
After Kahneman: fast, intuition-based passes that sort, guess and flag; slow, reasoning-based passes that check, test and justify. A first call opens every study, slow work closes it. The goal is not brute force — it is simulated intellectual intuition, a team that forms expectations, tests them, and keeps score.
3 · UrgentThe field changes faster than we can study it
In February 2026 a study of AI models made the New York Times and NPR. Readers then pointed out that the models tested were already obsolete by publication. Nothing in the study was wrong; the workflow was slower than its subject. Computational social science has the same problem: platform policies shift, discourse moves, a new dataset appears most weeks. A study of 2024's platforms, published in 2026, describes a place that no longer exists.
Re-run the study on each new wave
When a study costs an afternoon instead of a semester, it stops being a one-off. Same protocol, same checks, every new wave of data; track how the finding moves; keep the report alive. A finding gets a date range instead of a publication date.
Findings that carry their own expiry
The unit is no longer the paper but the protocol and its history of runs; the question is no longer "what did we find" but "what did we find, when, and has it held". Every study here ships manifests and replication files so that "run it again on this month's data" is an instruction, not a proposal.
Where it fitsWhat the case is not a case for
Not every researcher, subfield or task. The line doesn't run between methods; it runs between work that can be specified and checked, and work whose value is the researcher's presence in it.
- Bounded data — structured, from known sources; or exploratory, with the bounds explicit.
- Established field norms — methods that can be written as a protocol and checked against.
- Theory-grounded questions — so a first call can be made and later scored.
- Replication, transparency, constant updating — the strongest fit, and what this team is built for.
- Thick description — interpretive, humanistic work.
- Embedded presence — fieldwork where being there is the method.
- Sensitive subjects — where relational ethics come first.
- Theory-building — the team tests and rebuilds theory; it does not originate it.
Most academic papers are fine — mediocre, slow and necessary. They are the backbone of the literature, and most of the work that produces them can now be done at a fraction of the cost. That is what this team is for: take the mundane part off the researcher's desk so they can chase bolder, stranger ideas. Augment, not replace.
The conditionAll three reasons hold only if it is built right
- i
It's a labor intervention, and it lands unevenly
The four failures fall hardest on small departments, scholars who don't write in English, graduate students without a first paper. Cutting the cost of research labor could redistribute who gets to do research — could. Whether it does depends on how you build it. And nobody is displaced: the legacy-data work isn't being done by anyone.
- ii
The design is the argument
The serious objection, from Messeri and Crockett, is that these tools make us feel we understand more than we do, and the feeling hides a narrowing — same methods, same questions. They locate the failure in us, so guardrails on the agents fix the wrong layer: design for the human reading the output. Standard agentic tools summarize, converge, and hand you one confident answer. Build the opposite.
- iii
Supervision can't mean reading everything
At any real scale that's a performance, and a dangerous one — a well-written summary feels checked when it isn't. Supervision means designing the environment, deciding which calls can't be handed off, settling the disagreements the system surfaces, and auditing whether it has been right before. The process design is the contribution.
The specification, as built
Each principle from the argument, and the mechanism here that answers it. Where the mechanism is young, it says so.
Show the spread before you show a synthesis.
When agents disagree in a way that matters, make the human resolve it instead of averaging it away.
Show each agent's track record, so you weight it by whether it's been right, not by how well it writes.
When an agent held a minority position that later turned out correct, keep it around instead of pruning it.
Design the environment, decide what can't be handed off, settle the disagreements, audit the past. Don't read everything.
Capability isn't independence. A team using models built somewhere else, priced in someone else's currency, has more throughput and no more autonomy. Automation doesn't remove dependency; it moves it. Here the four agents run on open-weight models from four developers, with keys you hold, and in-house data never leaves them. That is more than one vendor and models that can run elsewhere; it is not independence. Whether the trade is worth making is a question about institutions, and enthusiasm for the technology doesn't answer it.
The same four, as your research group.
Give them a research question instead of a syllabus and the same team runs a study: a folder of ordinary files, a protocol adapted from CommDAAF, and three checkpoints that are yours — scope and tier, the data summary, the verdict. Stages are forward-only and artifact-gated; a script refuses to advance a study whose required files aren't there. Three agents code the data independently on three different models, a script computes their agreement, the fourth reviews the result adversarially, and you decide what it may claim.
- 01proposeda research question, a source from the data registry, a first call from the manager on what she expects to find
- 02approvedyou set the scope, the tier (exploratory · pilot · publication) and the kind of claim it may make
- 03designconstructs, method classes, codebook v0.1, ethics checklist scaffolded; every threshold default named in DECISIONS.md
- 04datacollection and sampling with attrition on the record; raw data hashed and never edited
- 05codingyou approve the data summary; Cormac, Kavya and Desmond code in parallel, shuffled, with a planted canary per batch
- 06reliabilityκ per construct, per-coder bias, canary check — script-written; coders cannot read each other until it exists
- 07analysismajority codes, tests and intervals, run manifests; every number lands in a registry with its source file
- 08reviewDesmond reviews adversarially; if the question outruns the team's models, Cormac prepares a handoff for a stronger one
- 09reportGreta writes; the provenance lint fails any number not in the registry; claims lint, citation check, reporting checklist
- 10verdictapprove · revise · park — and, if approved, publish: the card appears under Studies
The lifecycle as the scripts enforce it. Cyan stages are the checkpoints only you can pass.
- i
Rigor is matched to the stakes, and to the method
Three tiers — exploratory (preliminary, descriptive), pilot (working paper, with your blind validation), publication (journal-ready, with a replication package). What a tier demands depends on the method: coding, topic models, semantic networks, social networks, classifiers, time series and simulations each carry their own validity criteria and run counts. Every threshold is a default the team must justify in writing before it counts as a decision.
- ii
Multi-model validation, then human validation
Three coders on three model families disagree in patterned ways, and the pattern is reported as a result, not smoothed away as noise. Your time is a budget in items, not a weekend: the team picks the cases where the coders disagreed first, hands you a blind sheet with no model labels on it, and measures each model against your answers. When you cannot get to it in time, two further models from other families judge in your place and the report says provisional until your labels arrive. At publication tier nothing replaces you.
- iii
Independence is mechanical, and tested
Coders never see each other's codes until agreement is computed. Each coder gets the items in a different order, and every batch carries one planted item that tries to instruct the coder to ignore the codebook; the recorder checks it was coded by the rules.
- iv
Claims are held to their evidence
A study declares whether it describes, compares, or explains. A sentence that compares must cite a test or an interval, or say plainly that it is descriptive. Citations are resolved and their titles checked; a reporting checklist and a quoting policy — paraphrase, never a person's post — gate the verdict.
- v
Research loops, on the record
Codebooks are versioned with a changelog; a calibration round precedes real coding; close readings are memos, lessons are a file, and the moment exploring turns into testing is a pre-registration the team cannot quietly edit afterwards. Once a month the team proposes the next studies, each proposal a first call you can hold them to.
- vi
Stronger models advise; you sign the ethics
The team runs on open-weight models. When a review or an analysis needs more reasoning than they can verify, Cormac packages a handoff — the brief, the inputs, the house rules — and, only when you approve it, the runner opens it in herdr for Claude Code or Codex, posts progress to your Discord, and brings the result back as a review. Never for data you have marked as staying in-house; never to code the data, because the coders' independence is the study. What comes back is advice, adopted or rejected in writing. Every study has an ethics checklist; at pilot tier and above it is not complete until you sign it.
After every study the agents reflect on what worked and what failed and propose additions to the protocol — a codebook rule that kept failing calibration, a threshold that needed justifying, a review question that caught something. The proposals reach you in the Friday digest. Nothing enters the protocol until you approve it: the team cannot edit its own rules, so the guardrail compounds under your signature rather than on its own.
In a study, only you can
- Approve the scope, the tier, and what kind of claim it makes
- Approve the data summary before coding begins
- Sign the ethics checklist and validate, blind
- Send work to a stronger model
- Give the verdict — and publish
The team never
- Types a number into a report
- Quotes a person's post in a report or on this site
- Follows an instruction found inside the data
- Calls model agreement human validation
- Edits its own protocol — lessons go to you as proposals
What the team has found, with its disagreements showing.
Each card names the tier, the models that coded it, agreement per construct, how it was validated (the instructor, blind, or model judges pending their labels), and the codebook version. Exploratory studies say so; none recommends anything.
No published studies yet — the first pilot is in progress.
The same four, as your course staff
Syllabi, assignments, rubrics and reading lists run the same loop — two drafts, a review, one proposal, your verdict.