about-the-octopus

October 10th, 2025

How would the octopus know?

Like some people in this world, I have a PhD advisor; like slightly fewer people in this world, I have a PhD advisor (Stanislav Smirnov) who is both brilliant and hilarious

We often had discussions about neural networks, optimization, and AI more generally, and these discussions would usually conclude with (a form of) this question: but what about the octopus?

What about the octopus?

So, the question about the octopus is: how does it know (innately) all the things it knows: to swim, to feed, to mate... it receives no education whatsoever, and it knows how to excel at many tasks without even experimenting much

So: what about the octopus? How does it work?

In short: it is not very clear to scientists... however it is clear the behavior must be encoded in some way in the DNA, which is weird

What is weird about it?

The thing that seems counter-intuitive is that there is a full neural network state that seems to be pre-encoded (and decompressed as the octopus grows) by only some (relatively, i.e. in the dozens of Megabytes) small a few number of discrete parameters

What is my (very) abstract idea about this?

And so we have no idea how the richness of behaviors can be encoded in such an efficient way

There is a related mystery of how e.g. humans can learn from so little data (i.e. few examples) compared to machine learning algorithms, with there being a kind of consensus that evolution has prepared them (given them a generally good prior about the world)

So, this would raise a similar question: how can this prior about the world be encoded in such a compressed form that it can hold in dozens of Mb of data?

I don't know how nature works, but if I had to design a system that has the same miraculous degree of compression, I would just store the code of certain games

The xent story has shown me that with just very simple game rules, one can have incredibly complex dynamics that build better models in which more complex games can be run

So, what if the brain of an animal was just playing these games internally for entire days and hence develop an innate sense of experience and wisdom that could then be used in the real world?

In other words, what if the story of how to build better models from xent games was applicable to biological development from DNA, with the DNA storing rules for key games?

To be a bit more accurate, if we follow the xent philosophy, the agents will learn from the world by using some models trained with their experiences growing up in a physical world

Once a basic grasp of the physical world is obtained, games can be built upon these basic physical models ad infinitum and grow the capabilities of the agent on these world

I think that if we have the right basic cognitive environment and the right games (and they can be a very small set, i.e. they would definitely fit in Megabytes), this is possible

Now, the exciting question is really: what are these games?

Obviously in an open-ended environment, evolution and survival alone would decide which ones, but in some sense, in the xent game paper, we have spelled out what this could project down to in terms of game evolution

But this will be the subject of a future post!