Header image: KF in Dall-E
Nathan Gardels interviews Yuval Noah Harari about his new book Nexus
https://www.noemamag.com/al-will-take-over-human-systems-from-within/
- YH: Our human superpower is the ability to cooperate in very large numbers. And for that, you need a lot of individuals to agree on laws, norms, values and plans of action. So how do you connect a lot of individuals into a network? You do it with information, of course, and most importantly, with mythologies, narratives and stories. We are a storytelling animal.
- In the body, you do that by transferring information, whether through the nervous system or through hormones and biochemicals. It is not just a single information network. There are actually several information networks combined to hold the body together…
- ﻼF: ONE brain
the truth is a very rare and costly kind of information.
- The truth can be unattractive. Fiction can make one’s image of reality as pleasing and attractive as you would like. So, in a competition between information that is costly, complicated and unattractive, and information that is cheap and simple and pleasing, it’s obvious which one will win.
If you just flood the world with information, truth is bound to lose. If you want truth to win and to acquire knowledge and wisdom, you must tilt the playing field. How? By building institutions that do the difficult work of investing the time, resources and effort to find the truth and to explain it and promote it.
- The key thing to understand is that order, in most cases, is more important than truth for societies to cohere and work together collectively.
- ﻼF e.g. China [The Chinese Mayor v live/trust government]
- Now, most of the time the people who understand nuclear physics get their orders from experts in mythology or ideology. If you go to Iran these days, you have experts in nuclear physics getting orders from experts in Shiite theology. If you go to Israel, the experts in nuclear physics are getting orders from experts in Jewish theology. If you were in the Soviet Union, the orders came from Communist ideologues.
This is usually how it works in history: The people who understand order are giving the orders to the people who merely know the truth.
Narrative Warfare
- NG: Networks of connectivity are a dual-use technology. They can foster social cohesion and collective action, but they can also divide. Particularly now with peer-to-peer social media, you have every group with its own identity, believing its own truth and spinning its own narrative.
- This creates a kind of archipelago of subcultures, a fragmented sense of reality, which actually subverts cohesion.
As the Korean-German philosopher Byung-Chul Han puts it, peer-to-peer connectivity flows from private space to private space without creating a public sphere. Without a public sphere, there cannot be social cohesion. So you have this dual dynamic of cohesion — whether it’s for good or bad purposes — and then you have complete fragmentation. Order breaks down.
Empowerment & Control
- NG: There is another dual aspect of information networks: they both concentrate and disperse power at the same time.
“The internet centralizes in a few hubs while also empowering billions of people. It creates behemoths and yet gives everyone the opportunity to join in. Social media created a few giants and a million tribes. Everyone can build a website, but there is only one Google. Everyone can sell their niche products, but there is only one Amazon. The disruption of the internet is largely explained by this tension, this potent, combustible brew of empowerment and control.”
DeepMind’s co-founder Mustafa Suleyman
- YH: We tend to think of democracy and dictatorship as different ethical systems that believe in different political ideologies. That is true. But at a more fundamental level, they are simply different models for how information flows in the world.
- So elections by themselves are not enough. You need the entire range of self-correcting mechanisms which are known as the checks and balances of democracy to make sure that the distributed information network remains distributed and not overly centralized.
Leninist AI
- What we are seeing is the creation of extremely centralized information networks, because the algorithms of AIs just make it far more efficient.
- Sooner or later, this kind of Leninist AI will make some terrible mistakes, and there will be no mechanism to correct it.
- If I’m a human dictator, I should be terrified by AI because I’m bringing into the palace a subordinate that will be far more powerful than me and that I have no chance of controlling.
Alien Intelligence
- NG: The most worrying thing about AI is how it hacks the master key of human civilization by appropriating what you’ve called the superpower of sapiens — language and the ability to construct stories that bind societies together.
- YH: I think of AI as an acronym not for artificial intelligence, but for alien intelligence. I mean alien not in the sense that it’s coming from outer space, but alien in the sense that it thinks, makes decisions and processes information in a fundamentally different way than humans.
The most important thing to realize about AI is that it is not a tool. It’s an agent. Every previous technology in history was a tool in our hands…But you invent an AI, and the AI starts to make the decisions
- We haven’t seen anything yet. Let’s remember that the AIs of today like the ChatGPTs are extremely primitive…But it won’t take it billions of years to get to the dinosaur stage. It may take just 20 years, because digital evolution is far, far faster than organic evolution.
By definition, AIs are not something that we can plan for in advance and anticipate everything they will do. If you can anticipate everything they will do, then it is not AI.
The idea that, “Oh, we can just build some safety mechanisms into it, and we can just have these regulations,” completely misunderstands that what we are contending with is an alien agent that can act on its own and is not a tool like all previous technologies.
- AIs discovered that the easiest way to increase user engagement is to spread hate and fear and greed since that is what catches the attention of human nature. You press the hate button in people’s minds, and they are glued to the screen.
We Need Checks & Balances On Distributed Power
Democracy, at least in the modern world of large-scale societies, requires patriotism and nationalism. It is very hard to maintain a democratic system without a cohesive national community. Lots of people get this wrong, especially on the left. They think that nationalism and patriotism are forces of evil, that they are negative; that the world would be such a wonderful place without patriotism. It won’t. It will fall into tribal anarchy.
If you start thinking of other people not as political rivals, but as enemies — a different tribe out to destroy my tribe — then every election turns into a war of survival. If the other tribe wins, that’s the end of us. So, we must do everything, anything legal or illegal, to win the war, because it is a war. And if we lose the elections, there is no reason to accept the verdict. And if we win the elections, we only take care of our own tribe.
Hollywood Miscasts AI As The Terminator
- What the cinematic image so far misses is that the AIs don’t need to start from scratch. They are inserted into our own systems, and they can take our systems over from within.
- ﻼF: Doh! This is exactly how Skynet took over