When AI handles the calculation, what is the scientist’s job?

What the account describes

In an October 1 Anthropic guest post, physicist Matthew Schwartz describes using Claude to build BootLoops, a toolkit for quantitative calculations. Domain experts helped redirect technically successful work toward scientifically meaningful questions. Schwartz was a visiting researcher at Anthropic during the project. He owns and maintains BootLoops; it is not an Anthropic project.

The account concerns work over three months. Several highlighted results remain under exploration and verification. It supplies neither an employment study nor a general estimate of researchers’ productivity, and should not be treated as either.

Sources: Anthropic — guest post by Matthew Schwartz

Our interpretation: evaluate the research decision

For a laboratory considering AI tools, we would start with a decision the research team needs to make. What would a successful calculation let the team conclude, and what would it leave unresolved? Write that distinction before generating code. Otherwise, a polished output can become the destination even when it does not answer the original question.

Consider a hypothetical team studying whether a treatment changes a cell population. An assistant produces a clean analysis and a convincing figure. A useful review would ask whether the unit of observation is a cell or a donor, whether batch effects were handled, and whether preprocessing choices influence the conclusion. This example is our proposed evaluation exercise, not a result reported by Schwartz.

Have a researcher inspect the data structure before reading the generated interpretation. Then ask them to specify a check that could undermine the proposed explanation. They might compare results across donors, test an alternative normalization, or identify an unmeasured difference between groups. The appropriate check depends on the study design; a fluent explanation cannot choose it on the team’s behalf.

What this could mean for training

We would assess a junior researcher on a short evidence package: the question, the assumptions, a reproducible calculation, the checks performed, and one conclusion the evidence cannot support. Include an account of what the tool supplied and what the researcher changed. That makes contributions inspectable without pretending that every line of code was written manually.

This keeps the underlying methods relevant. A student who cannot recognize a broken assumption needs a different exercise from one who understands the method but struggles to implement it. Supervisors could use that distinction to choose the next assignment.

A small experiment for a research team

Have a colleague evaluate an AI-assisted reproduction of a previously resolved question. Record review effort, errors, and unresolved discrepancies alongside coding time. Use the exercise to choose the next responsibility to delegate. One successful reproduction cannot establish a staffing conclusion or prove that independent research has been automated.