Search Ideas

Conversely, non-inferential novelty does not follow by any inference or deduction; it is not implicit in existing knowledge, so it is not in any set of knowledge's implication space (except that of the laws of physics...).

Knowledge which is novel in this way is what constitutes creation in the sense we mean in epistemology. The two known examples of non-inferential knowledge growth are biological evolution and some of the thinking which takes place in the minds of people (human or AGI).

Any given set of knowledge cannot grow itself non-inferentially. There must be a source of novelty, of variation, that is epistemically external to it.
If we take that claim as general, then the mutation of DNA constituting the creation/novelty/knowledge-growth in biology is a specific instance: cosmic ray strikes and errors in copying due to chemistry are sources of variation not caused by the DNA’s knowledge).

Claim: It is useful (if not crucial) to distinguish between two notions of novelty: inferential and non-inferential.

Knowledge that is inferentially novel may never have been seen before, but was already implicit in existing knowledge; it results from application and composition of existing knowledge (e.g. the running of programs). What is novel in this way can be instantiated without any external epistemic (knowledge-related) influence. A program needs infrastructure to allow its computation to occur; it does not need another program, in an epistemic sense: If there's a dependency, we should consider the two to be part of the same "epistemic system." Let's say the dependent target has no "implication space" until its needed source is causally linked to it. The source may have its own implication space. Once "joined," a new implication space is defined.

Inferentially novel output (the instantiation of knowledge in an implication space) is all that current computer software (and all AI) is capable of. These systems "create" knowledge only in this sense.

I wonder if there is any significance to the "distance" the laws of physics are from us, in the sense of emergence. Each quantum application of the laws of physics (conceived as discrete steps) is so far down, but maybe it need not be so. Maybe there are people in other laws of physics, where the laws are much closer at hand. Do the laws need to have a certain minimum range of emergence for personhood to be possible..? Couldn't people like us exist in a universe where fundamental physics is no more complex than the operation of our minds, or is the "breathing room" we have here necessary?

If other criteria for the selection of conjectures were built into the system, such as in the genes of a human, the growth of knowledge would be bounded to only conjectures meeting those criteria, even if the variation/conjecture aspect of the process were proceeding via a random process with no built-in knowledge. Fixed criteria means bounded knowledge, means no universality. So for unbounded progress (universality), the criteria must also be varied. But how are they criticized? What criteria for their selection can there be to avoid a regress, except survival?

All subsequent knowledge in the system will be descended from the initial knowledge. So all knowledge in a knowledge-growing system must be either created by or comprised of replicators.

Knowledge growth (such as in genes in the biosphere, or in a person) can only proceed by conjecture and criticism -- variation and selection. In a system which can create knowledge, but has not yet, there are no modes of substantive criticism available, since that would itself require knowledge. So how can the system create knowledge at all? Because although substantive criticism is not yet possible, the crudest form of full-on refutation is: the destruction of the information medium on which a conjecture is based, i.e., death. Death is the only "undesigned" refutation. So any initial knowledge created in an otherwise empty knowledge-creating system can only be that relating to how to avoid death in the local environment. This is the knowledge which defines replicators.

There must be a minimum amount of available memory needed for a program to be a person. What can the program not do, below that minimum? What is that minimum M bits being used for that cannot be done with M-1 bits?

Gemini's response to that tweet: "The text uses legitimate biochemical concepts—like the fact that fluoride is a potent halogen that hardens minerals and can be toxic in high doses—to build a narrative based on skewed history. It fabricates a geological explanation for Hereford, Texas, to erase the very thing that actually protected the town's teeth: naturally occurring fluoride."

Is a quale a sequence of information states? Is this true of any given computation? Does the quale "care" if the states are computed afresh or read from memory?

Assuming qualia are classical, there is nothing stopping every step in the computation comprising them from being measured, like any computation. Then, nothing is stopping that recording/program from being rerun.

Any physical process can be simulated to arbitrary precision. A simulation is a program, and nothing is stopping any given program from being rerun.

But the mind has finite memory as well, and doesn't stop showing novelty. I was going to say it doesn't plateau in complexity, but actually I guess there is a ceiling... It's just that new and better ideas make better use of the finite resources; they aren't just more complex.

Not sure what you mean by the soup being more open to the world, or allowing for "more" open-ended evolution. Unless you mean the quantity. I think evolution is either open- or closed-ended, as a binary. And again, not convinced any external world or feedback is required for open-endedness (per Deutsch). It might merely be very helpful.

Could you say more about those regions?

I wish! I'll try.

creativity still could occur in total isolation, but I don't think they would generate much knowledge due to the lack of external feedback.

Maybe less overall, maybe just less by our standards. Maybe still a lot of knowledge can be made in isolation, just of a schizophrenic kind, more unstable without the regularities of external reality. But I'm wondering if isolated regions of the mind can provide "external" variation to each other in that one can make changes to the other, but not vice versa (like how the world makes blind changes to DNA but not vice versa). Maybe this is how evolution happens more quickly in the mind than by random copying error and mutations alone, because there are many more sources of blind variation...

Even in isolation, the real world applies [hardware constraints]

Agreed. So by isolation from reality we shouldn't mean isolation from physics. Just that the evolution taking place in the mind has no specific causal dependence on the external reality, I think. If it's "isolated" in this sense, nothing outside the mind, if changed, would impact the mind's evolution, its content. Even though mind and world are both determined by the same physics. Reality is not necessary, in this sense, so marking this as a criticism.

I also picture a primordial soup often. Maybe including these autonomous regions or layers I'm grasping at. Soups and "Turing gases" have been programmed and so far always plateau in complexity, even with no predefined criterion, so there's something else going on.

One neat paper, for example: https://arxiv.org/pdf/2406.19108
It sounded unbounded at first, but Claude explains that in a recent followup interview, plateauing has been confirmed: https://poggio-lab.mit.edu/blogsupdates/interview-blaise-aguera-y-arcas

Unless I'm mistaken, Deutsch has argued that a person in total isolation from reality would still be capable of creativity. Reality is a good source of problems/niches for us, but is not necessary. It seems to me we must have sources of selection in the mind. I picture autonomous regions, or layers, each of which is like an external reality from the perspective of the others, so they all scratch each other's backs.

One idea for a global criterion is simply productivity: If a program successfully produces an output which is then used by other programs (the more the "better"), it is replicated (by the global logic). Maybe it's replicated every time it's run as part of a greater process, in that sense. Sort of fits with capitalism: make useful products, get rewarded (replicated, here).
The bookkeeping may be an issue, though.

Do the compositions have to be self-reproducing? Or can they be replicated when a predefined global condition is met? I think we agree any predefined condition will keep the system closed-ended (the evolution of programs will be steered toward those meeting the condition). But maybe that's only if the condition is specific. If the criterion for replication to occur is non-specific, maybe this allows for open-endedness? It seems like we would want a condition that only rules out programs which cannot constitute a growth in knowledge.

Drawing on the framing of your question, it seems like a "universality of landscapes" is what's needed. Perhaps: the space of programs to be explored is only ever determined by whatever the system happens to have assembled so far, the pieces it has at any given time, which is never specified. What the system contains is determined by random/blind variation and composition (of some complete set of primitives) and selection for the most fecund compositions among them.

My suspicion remains that creativity and qualia come from some kind of blind exchange between layers of emergence, of which mutations and mistakes in copying, as seen in DNA, are an example.

That's what I get for suddenly returning to this discussion after a long gap. I was thinking that any computation can be recorded and re-run at will, in principle. So qualia can too. But indeed, I have suggested in this discussion that it is the evolution of programs that causes experience, and evolution cannot be scripted. This means a quale that is recorded will not "qualiate" if it is rerun exactly as recorded.

TL;DR: I don't.

A quale can be recorded and replayed later, arbitrarily many times. That looping program is not a person: it can only create whatever knowledge it originally did, not any possible knowledge, maybe until freed from the loop. Yet it still constitutes experience.
So people are not the only programs that can be qualia.

True that a person need not create new knowledge, but a looping quale is not a person, per #5035, so the claim survives this criticism.

True, knowledge could be created from each run of a quale, but a looping quale isn't a person, per #5035, so the claim in #5031 that not only people can undergo experience still stands.

Should clarify that personhood is: can create any new knowledge in principle (excluding resource constraints). A program consisting of a looping quale cannot create any possible knowledge, whether or not it is creating some each time it's run.

A looping quale is also not a person because personhood is not just: can create a given piece of new knowledge, it is: can create any knowledge (not just that created by this quale; again assuming knowledge creation defines qualia).

If qualia are only present when new knowledge is being made, then for a quale to be replayed it could be that there must be a reset each time: the knowledge produced from the previous run is erased. So w.r.t. the AGI system alone, new knowledge is being created each time (it doesn't matter if the knowledge is copied from a previous run and stored elsewhere, in which case it is not new w.r.t. that combined system).