Superintelligence by Nick Bostrom (2014)
28th December 2018
A comprehensive review the existential risk posed by Artificial Intelligence and in particular by the possibility of a rapid intelligence explosion, and strategies to mitigate this risk.
Concludes that although the timing and form of Superintelligence is uncertain, the risks are real and argues that urgent research into various problems is needed to allow us to take precautionary steps.
Originally published in 2014, the 2016 paperback edition includes an short Afterword which reviews some of the subsequent developments in this field.
In broad outline the book examines:
- various possible types of AI
- the chance of a rapid take-off of Superintelligence, in the form of one or more AIs with reasoning and planning skills that are far in advance of humans
- the problem of controlling such an AI and the concept of an instrumental reason (i.e. a means to an end)
- the difficulty of defining correct goals for a Superintelligence
- possible outcomes for humanity on different scenarios.
Chapters 1-5
TBC
Chapter 6 -- Cognitive superpowers
Identifies six strategically relevant tasks for an advanced cognitive system and the skill-set that each task requires. These tasks are:
- Intelligence amplification
- Strategizing
- Social manipulation
- Hacking
- Technology research
- Economic productivity
Argues that a sufficiently capable AI with one of these capabilities might conclude that gaining other capacities was a good 'means to an end', and that it might be able to leverage the first capability to acquire the others.
Chapters 7-15
TBC