What part of being a scientist is essentially human?

QuestionStatus: Updated

If machines can search literature, propose hypotheses, operate instruments, analyse results and write reports, the interesting question is no longer which scientific tasks can be automated. It is whether anything about science is inseparable from human judgment.

Science is often described as a method.

Observe.

Form a hypothesis.

Design an experiment.

Collect evidence.

Analyse the result.

Revise the hypothesis.

If this description were complete, then sufficiently capable automation might eventually perform almost all of science.

And pieces of that possibility already exist. AI systems now generate and refine hypotheses; computational agents can execute substantial research workflows; self-driving laboratories can choose and perform experiments using robotics and algorithmic decision-making.

But the scientific method is not merely a sequence of operations.

Before any experiment begins, someone has to decide that a question is worth asking.

A result can be statistically significant and intellectually trivial.

An experiment can be technically flawless and conceptually irrelevant.

A hypothesis can fit existing observations while directing attention away from a more important explanation.

Science therefore contains another layer that is harder to place inside a flowchart:

selection of significance.

Why study this anomaly rather than another?

Why distrust this elegant explanation?

Why spend ten years measuring a phenomenon everyone else considers unimportant?

Why regard an unexpected result as a discovery rather than an error?

Humans do not answer these questions through logic alone. Scientific judgment emerges from accumulated knowledge, intuition, rivalry, aesthetic preference, institutional incentives, curiosity and historical circumstance.

Some of these influences improve science.

Some distort it.

That creates two very different possibilities.

One is that future AI systems learn enough of this structure that the supposedly human remainder gradually disappears. Scientific taste might be another cognitive competence that can be modelled.

The other is that science always contains an irreducible act of valuation.

Evidence can tell us what is true.

It cannot, by itself, tell us what is worth knowing.

Even the choice to optimize for useful discoveries instead of beautiful theories, human welfare instead of pure understanding, or short-term results instead of century-scale questions already contains values.

So perhaps the last human role in science will not be pipetting samples, reading papers or even generating hypotheses.

It may be deciding what deserves attention.

But that conclusion is not secure either.

If an artificial system begins proposing questions that humans had not considered — and those questions repeatedly produce important discoveries — then we will have to ask whether “scientific curiosity” was genuinely unique to us or merely difficult to automate.

The question should therefore remain open:

When every procedural stage of science can be automated, what would still make scientific judgment human?

What would change our mind?

Evidence that autonomous systems repeatedly identify important, non-obvious research questions before human researchers do.

Or the opposite: evidence that even very capable systems remain dependent on human framing for the discoveries that matter.

References

  1. The AI Scientist
  2. Co-Scientist
  3. Robin
  4. Self-driving laboratories