Forum: Author Hangout

This could be read as why definitions matter more than you think.

I am struggling with the words / language to explain all of this, partially because there is not as far as I know a public discussion that builds the jargon or vocabulary to support the evolving ideas. Evolution works primarily on a reward basis. I'm not going against 'natural selection' or 'survival of the fittest' but trying to describe a lower level mechanism that drives these very systems, via a reward feedback pathway. if 'something' gives an 'advantage' then that is performative, but why? The advantage itself is not a magical end pont but built on simpler sets of systems that eventually result in the 'advantage' via interactions.

If you look at the overwhelming design themes in A.I. since Turing and probably before, they are for lack of a better word 'backwards' or perhaps 'upside down'. What do I mean by that? They do not match the biological purposes of intelligence, minimising predictive errors. Yes they do use statistics and weighted values to arrive at a conclusion, but this is kind of a misrepresentation of how intelligence works and what the evolutionary pressure response pushes it towards. In physics a lot of 'work' is looking at things on scales so small they birthed the term / feild quantum mechanics. the opposite scale includes astrophysics. I mention these two as illustrative of different points of view comprable to understanding intelligence and A.I. specifically.

most A.I. work is like astrophysics, looking at problems defined at very large scales working top down rather than building bottom up. If you just give the A.I. enough information or data than it will become 'inteligent', but what if you reversed that? a quantum approach.

If you think of something that makes you 'uncomortable' this is a pressure signal, something an LLM cannot mimic, yet. consider this on a cellular scale and what is happening. The neurons are unhappy because their inputs are misaligned with expectations. If neurons expect 'order' a lack of either actual order or a poor signal will 'distress' them. Current A.I. systems rely on 'scale' to find the answers by mass instead of efficiency. Rather, 'learning' how to arrive at one because not having the answer is 'unpleasant'.

What is the point of the distinction? it completely changes the rules you base everything on. Under the traditional method your results are dependent on having enough resources. If you take a biological rather than statistical approach then the limit is efficiency and response not maths. I came to think of these opposite views because I was trying to reach a conclusion about the goals of an Alien Artificial Intelligence at the centre of a story. The original idea was absolute power corrupting absolutely and that is still the premise only because I changed the 'rules' of how learning and intelligence works I reversed the direction of corruption.

Now instead of a unlimited knowledge and power corrupting the user. A mistake at the core of the A.I. means that it is corrupted by humanity. the ending changes completely and the solution I struggled to reach became almost self evident and somewhat darker. I still need to think a lot more but by changing a base principal I have solved a core problem. Hopefully I have not created a worse one.

The hot batch of current human AI systems, the LLMs aren't "intelligence" by any means, they are systems that mimicry intelligence at their best. It is, indeed an upside down approach, trying to falsify outputs so well it becomes indistinguishable from intelligent behavior. However, at that point it becomes a philosophical question.

Bottom up approach is emergent behavior in very simple algorithms. The classic ant robot swarm gathering tea candles:
10: Move in straight line until either:
11: encounter a wall -> make a random turn,
12: measure a weight of 3× candle in the grabbing claw -> stop and move backwards to let them off, then make random turn.

That's all the program of each drone, there's no communication and no oversight, yet, swarm of such drones will, while not in perfectly optimal way but sooner than most would expect, gather the candles in one nice pile and keep tightening it.

Most biological systems rather build upon such. Even humans. Social structures are emergent, not designed, or rather, the emergent results differ significantly from conscious design goals (why communism will never work above limited and already driven community scale, among other things).

In a way, we perhaps expect the same thing from our AI: build a sufficient scale of matrix multiplication and the "intelligence" would, perhaps, emerge from depths of the scholastic parroting. At very least, observer will lose ability to tell the difference, that, we're perhaps not so far off.

What overachieving goals such emergent entity would define for themselves? We have no way to know, and that's the really scary thing. We can only hope it wouldn't end up paperclip maximizing (or similar). Far more likely, it will be extension not full overhaul of human culture it's feeding on, but in what way?

Trying to comprehend goals of alien AI built upon concepts and goals we may struggle to conceptualize in the first place, seems a futile excercise. In a story, the author has a huge advantage though, they may declare it any way they like, and since the premise is incomprehensible by definition, anything they say goes and should be "plausible" because (almost?) anything is equally likely.

However, I realize the struggle to justify the choices. One generally wants to fool themselves in thinking they didn't made them arbitrary, even though that's what likely happens anyway, It's perhaps a process somewhat similar to religious: the reinforcing scaffolding of confidence is circular in nature. But as we established above, that's how biological systems operate, so perhaps it's only natural indeed.

An "all-powerful" AI ascendant would rather care about information quantity and quality than physical aspects or outcomes. One could, for an example, decide to photograp absolutely everyone naked everyday for the next 90 years just to rectify perceived lack of coherence in some medical data. Or something similar. AI Doomsday will almost necessarily be nonsensical to human scale intelligence.