Log in or sign up to participate in this discussion.
With an account, you can revise, criticize, and comment on ideas.Another way to approach AGI? (Very early, preliminary thoughts.)
Say you write an evolutionary algorithm, like the ones that have been written before. Then DD would argue it’ll get stuck because all it can do is explore a given landscape for its best features. Whereas real evolution creates new landscapes.
To address this issue, you subject your algorithm itself to variation and selection, by wrapping it in another evolutionary algorithm. But this approach just kicks the can down the road because now it’s the space of programs that’s limited.
How do you break out of this limitation?
You can’t just keep wrapping your programs in evolutionary algorithms like that because that only keeps kicking the can down the road. It’s like adding more and more entries to a multiplication table. It’s not the same as a multiplication algorithm. But for evolution, the problem is harder, in a way, because not even recursion solves the issue, and the starting point wasn’t as ‘flat’ as a multiplication table. The starting point is already an algorithm, not just a list.
What’s needed, in DD’s lingo, is a jump to universality. But a jump to what kind of universality, exactly?
cc @tyler-mills
In biological evolution, the landscape itself never changes directly. It only happens as a consequence of evolving genes.
Guess: The same is true in the "landscape" of the mind: Individual ideas mutate and evolve in relation to problems, and that's what constitutes the landscape.
That sounds right but how is it a criticism of my idea? It’s the ideas the mind evolves that change the landscape, no?
Hmm, I'll try to formulate in another way what I was getting at, not sure if you still disagree with this:
- My guess is that the selection mechanism can't be specified at all in the evolutionary algorithm, because every such specification is a restriction of universality. Reality has to do the selection.
Unless I'm mistaken, Deutsch has argued that a person in total isolation from reality would still be capable of creativity. Reality is a good source of problems/niches for us, but is not necessary. It seems to me we must have sources of selection in the mind. I picture autonomous regions, or layers, each of which is like an external reality from the perspective of the others, so they all scratch each other's backs.
Could you say more about those regions? :)
I agree that creativity still could occur in total isolation, but I don't think they would generate much knowledge due to the lack of external feedback.
Even in isolation, the real world applies constraints/selection mechanisms. Mainly in the form of hardware constraints (scarcity of memory/working memory).
I imagine the mind sort of like the "primordial soup", where the first replicators began replicating. To simulate that, I don't think there's any room for a specific "selection algorithm".
Could you say more about those regions?
I wish! I'll try.
creativity still could occur in total isolation, but I don't think they would generate much knowledge due to the lack of external feedback.
Maybe less overall, maybe just less by our standards. Maybe still a lot of knowledge can be made in isolation, just of a schizophrenic kind, more unstable without the regularities of external reality. But I'm wondering if isolated regions of the mind can provide "external" variation to each other in that one can make changes to the other, but not vice versa (like how the world makes blind changes to DNA but not vice versa). Maybe this is how evolution happens more quickly in the mind than by random copying error and mutations alone, because there are many more sources of blind variation...
Even in isolation, the real world applies [hardware constraints]
Agreed. So by isolation from reality we shouldn't mean isolation from physics. Just that the evolution taking place in the mind has no specific causal dependence on the external reality, I think. If it's "isolated" in this sense, nothing outside the mind, if changed, would impact the mind's evolution, its content. Even though mind and world are both determined by the same physics. Reality is not necessary, in this sense, so marking this as a criticism.
I also picture a primordial soup often. Maybe including these autonomous regions or layers I'm grasping at. Soups and "Turing gases" have been programmed and so far always plateau in complexity, even with no predefined criterion, so there's something else going on.
One neat paper, for example: https://arxiv.org/pdf/2406.19108
It sounded unbounded at first, but Claude explains that in a recent followup interview, plateauing has been confirmed: https://poggio-lab.mit.edu/blogsupdates/interview-blaise-aguera-y-arcas
If this is intended to answer my criticism, shouldn’t it be marked as a counter-criticism?
Drawing on the framing of your question, it seems like a "universality of landscapes" is what's needed. Perhaps: the space of programs to be explored is only ever determined by whatever the system happens to have assembled so far, the pieces it has at any given time, which is never specified. What the system contains is determined by random/blind variation and composition (of some complete set of primitives) and selection for the most fecund compositions among them.
Do the compositions have to be self-reproducing? Or can they be replicated when a predefined global condition is met? I think we agree any predefined condition will keep the system closed-ended (the evolution of programs will be steered toward those meeting the condition). But maybe that's only if the condition is specific. If the criterion for replication to occur is non-specific, maybe this allows for open-endedness? It seems like we would want a condition that only rules out programs which cannot constitute a growth in knowledge.
One idea for a global criterion is simply productivity: If a program successfully produces an output which is then used by other programs (the more the "better"), it is replicated (by the global logic). Maybe it's replicated every time it's run as part of a greater process, in that sense. Sort of fits with capitalism: make useful products, get rewarded (replicated, here).
The bookkeeping may be an issue, though.
At its core, I think evolution needs three variables: an environment, a mechanism for reproduction, and a mechanism for destruction. If either of these is missing, evolution will not happen. The algorithm for evolution is then definitively defined by the environment and the mechanisms of reproduction and destruction, with the current set of genes as the initial condition.
This sounds true but does it add anything new to saying that "AGI is an attempt to program Darwins theory of evolution"? 🤔
Please remember to mark your ideas as criticisms whenever appropriate. Although this idea is phrased as a question, it’s still a pending criticism as long as it doesn’t get answered.