Criticism 1: The Decomposition is Arbitrary

The objection: The entire theorem rests on splitting a hypothesis h into (h ∨ e) and (h ∨ ¬e) and then showing the second part gets negative support. But why split it that way?

Critics argue this is a choice, not a necessity. Define "the part that goes beyond the evidence" differently and you get different results.

This is the most common objection in the literature. Ellery Eells argued the key assumption has been "almost uniformly rejected," because the propositions generated by Popper and Miller's decomposition contain content from both the evidence and the hypothesis tangled together, so they don't cleanly capture "the part that goes beyond the evidence." (Eells, 1988, British Journal for the Philosophy of Science 39, 111–116 — PDF)

Chihara and Gillies proposed "a new condition on what constitutes 'the part of a hypothesis that goes beyond the evidence' that is incompatible with Popper and Miller's condition, "arguing this refutes the impossibility of inductive support. (Chihara & Gillies, Philosophical Studies 58, 1990 — PDF)

The Popper-Miller Theorem

Bayesian epistemology says that knowledge works like this: you have a theory, you see evidence, and the evidence raises your confidence in the theory. That's how you learn. The math behind this is Bayes' theorem, a formula for updating probabilities when new information arrives.

In 1983, Karl Popper and David Miller published a paper in Nature titled "A proof of the impossibility of inductive probability" that used this exact math to prove something uncomfortable: the part of a theory that goes beyond the evidence never gets supported by that evidence. It actually gets negative support. In their words: "probabilistic support in the sense of the calculus of probability can never be inductive support." (Popper & Miller, 1983)

They expanded on this in a second paper: "although evidence may raise the probability of a hypothesis above the value it achieves on background knowledge alone, every such increase in probability has to be attributed entirely to the deductive connections that exist between the hypothesis and the evidence." (Popper & Miller, 1987)

Here's what that means concretely. Say your theory is "all swans are white." You see a white swan. Your overall confidence in the theory goes up. But Popper and Miller split the theory into two pieces:

1. The deductive piece: "this particular swan I'm looking at is white." The evidence directly confirmed that.
2. The inductive piece: "and all the other swans I haven't looked at are also white." This is the part that would actually represent learning something new about the world.

They proved mathematically that piece #2, the inductive piece, the part that matters always receives zero or negative support from the evidence. The only work the evidence ever does is confirm what it directly touched. It never reaches beyond itself.

The Math

What follows is a simplified sketch of the proof. For the full formal treatment, see the original paper.

Step 1: Define "support."

The support that evidence e gives to hypothesis h is defined as the change in probability:

s(h|e) = p(h|e) − p(h)

If this number is positive, the evidence raised the probability of the hypothesis. Bayesians call this "confirmation."

Step 2: Decompose the hypothesis.

Popper and Miller split h into two components:

• The deductive component: (h ∨ e), meaning "h or e." This is the part of h that is logically connected to the evidence. If e is true, then (h ∨ e) is automatically true, so evidence trivially supports it.

• The inductive component: (h ∨ ¬e), meaning "h or not-e." This is the part of h that goes beyond the evidence, the part that would still need to be true even if the
  evidence hadn't occurred.

The hypothesis h is logically equivalent to the conjunction of these two components: h ⟺ (h ∨ e) ∧ (h ∨ ¬e).

Step 3: Calculate the support for each component.

Using standard probability rules, the support for the deductive component is:

s(h ∨ e | e) = 1 − p(h ∨ e)

This is always ≥ 0, since p(h ∨ e) ≤ 1. The evidence always supports the deductive part. No surprise, the evidence is logically contained in it.

The support for the inductive component is:

s(h ∨ ¬e | e) = −(1 − p(e))(1 − p(h|e))

Both (1 − p(e)) and (1 − p(h|e)) are ≥ 0 (assuming we're not dealing with certainties), so their product is ≥ 0, and the negative sign means the whole expression is always ≤ 0.

Step 4: The result.

The total support decomposes as:

s(h|e) = s(h ∨ e | e) + s(h ∨ ¬e | e)

The first term (deductive) is always non-negative. The second term (inductive) is always non-positive. The evidence never positively supports the part of the hypothesis that goes beyond the evidence. Whatever "boost" h gets from e is entirely accounted for by the deductive connection between them. The inductive component, the part that would represent genuine learning about the unobserved, is always counter-supported.

Implication

The implication is devastating for Bayesian epistemology: the entire framework of "updating beliefs with evidence" is an illusion. The number goes up, but the going-up is entirely accounted for by deduction. There is no induction hiding inside Bayes' theorem. The Bayesians' own math proves it.

David Deutsch, who has been working with colleague Matjaž Leonardis on a more accessible presentation of the theorem (Deutsch on X/Twitter, 2020), puts it this way: "There's a deductive part of the theory whose credence goes up. But the instances never imply the theory. So you want to ask: “The part of the theory that's not implied logically by the evidence – why does our credence for that go up?” Well, unfortunately it goes down." (Joseph Walker Podcast, Ep. 139, "Against Bayesianism")

The Conjunction Problem

Deutsch also offers a separate, more intuitive argument: take quantum mechanics and general relativity, our two best physics theories. They contradict each other.

• T₁ = quantum mechanics
• T₂ = general relativity

Both are spectacularly successful. A Bayesian should assign high credence to each. But T₁ and T₂ contradict each other, and probability theory is absolute about contradictions:

p(T₁ ∧ T₂) = 0

Zero. The combined understanding that lets us build GPS satellites, which need both relativity for orbital corrections and quantum mechanics for atomic clocks is worth literally nothing under the probability calculus.

Meanwhile, the negation ¬T₁ ("quantum mechanics is false") tells you nothing about the world. It's the infinite set of every possible alternative, mostly nonsensical. Yet the probability calculus ranks it higher than the theory that lets us build lasers and transistors.

A framework that assigns zero value to our best knowledge is, Deutsch argues, not capturing what knowledge actually is. Instead: "What science really seeks to ‘maximise’ (or rather, create) is explanatory power." (Deutsch, "Simple refutation of the 'Bayesian' philosophy of science," 2014)

This is solved by actively doing some visible stuff you'd want to do anyway as an AGI researcher.

You could spend some time in a cheap country.

You say that trade-offs and scarcity are fundamental to biology. I agree, and this implies economics as a more fundamental science than biology or evolution. It still applies in our computer models, where biological details may not.

Undestanding does not flow from explanatory knowledge the way you imply. I understand Dutch and English, but a lot of my understanding of it is inexplicit.

Not a doctor. But it's not hard for me to imagine untainted memory but a script with an error such that it can't manage to look up the information.

I don't think a gene has problems. It does not have ideas.

Sure, philosophers and pedants do. But typically people use the word "know" in situations well short of being absolutely sure.

Copyright is stifling to creativity, as now people are not incentivised to write fan-fictions.

This is stifling to creativity, as now people are not incentivised to write fan-fictions as much as without copyright.

I fail to see how fan fiction is at all damaging to an original creator.

We have found an example where copyright is bad.

Where is copyright good?

I didn't know that. I figured linking to the tweet that posted it would be fine.

I have to admit I was unsure how many claims I actually made, and excused myself from the burden of having to figure it out with the following excuse: I expect that many potential users of your platform would make this error and therefore we should try to run with it. I also don't mind the bulk criticism.

I think it's different from Pascal's wager, as with Pascal's wager you have infinite, or many (all known religions) wagers. (Which god?) Whereas with animal consciousness we have only one wager, that we're currently not sure of, on which we're wagering a lot of potential animal suffering. Furthermore, we are not on our deathbed, and hence have the luxury of time to consider our trade.

The idea is: given that we know little about animal consciousness, it's better to err on the safe side given the asymmetry of inconvenience versus mass animal suffering.

I think you can attack this by either pointing out how that is a faulty way of thinking about a 'lack of evidence', or that there is indeed enough information on animal consciousness.

I'm mostly interested in finding evidence or thinking of cases that would be evidence, and less about the implications on morality.

Unless we are solipsists, we conclude that all human beings are conscious.

A simple extrapolation to animals would be to say that those with similar characteristics to humans, could also have consciousness.

I am not aware of any strong theories on animals being unconscious, other than intuitions of some AGI researchers who conjecture that sentience hangs together with unique learning capabilities of humans.

And suppose that we have a reasonable (best available) current explanation for why animals are not conscious, I don't think that puts us in a Pascal's wager situation, because considering our own (recognised) fallibility, and the asymmetry of being right and wrong with respect to moral outcomes: enormous suffering versus inconvenience, we should consider to tread on the safe side until we have more evidence.

This implies that we should treat animals carefully, as their sentience allows them to feel pain, until we have a lot more information. Interestingly, this also implies that wild nature is evil and that we should seek to get rid of it (if we continue to believe in animal consciousness).

TL;DR: We only have vague conjectures on animal consciousness

Unless we are solipsists, we conclude that all human beings are conscious.

A simple extrapolation to animals would be to say that those with similar characteristics to humans, could also have consciousness.

I am not aware of any strong theories on animals being unconscious, other than intuitions of some AGI researchers who conjecture that sentience hangs together with unique learning capabilities of humans.

And suppose that we have a reasonable (best available) current explanation for why animals are not conscious, I don't think that puts us in a Pascal's wager situation, because considering our own (recognised) fallibility, and the asymmetry of being right and wrong with respect to moral outcomes, we could consider to tread on the safe side until we have more evidence.

This implies that we should treat animals carefully, as their sentience allows them to feel pain, until we have a lot more information. Interestingly, this also implies that wild nature is evil and that we should seek to get rid of it (if we continue to believe in animal consciousness).

TL;DR: We only have vague conjectures on animal consciousness