Interview by Lex Fridman - Steven Pinker (2018)
30th December 2018
Two kinds of existential threat have been discussed and both are incoherent:
- an AI take-over where a more advanced intelligence will take over..…confuses intelligence with a will to power..…humans have traits such as dominance but a pure problem solving systems won't have that as one of its goals..…"its goals will be whatever we set its goals as" and there is no reason to suppose it will evolve in that direction and we can assume that we won't give it stupid goals such as maximising its own power..…analogy between nuclear weapons and AI is fundamentally misguided since the whole point of nuclear weapons is to destroy things, which is not the goal of AI..…goals are external and if we don't design an AI system to maximise dominance then it won't..…we're just so familiar with homo sapiens
- that we'll be collateral damage such as a goal of maximising paperclips, curing cancer or world peace, each of which can have unwanted results (what Bostrom calls perverse instantiations) which Steven Pinker says he finds "utterly fanciful and in fact self-defeating" because it assumes we are intelligent enough to engineer a superintelligence but not enough to give it a sensible goal, and that the system is so smart that it can 'cure cancer' but so idiotic that it can't figure out what we mean by 'curing cancer' also implies other goals..…so the value alignment problem is based on a misconception. Furthermore all engineering systems are a trade-off between multiple goals and this is actually part of the definition of intelligence.
And on the question of the timescale of the threat of AI as existential threat..…it's fanciful..…culture of engineering is to test systems before using them..…assume magically powered intelligence that is connected to infrastructure without testing..…need some legal framework perhaps for irresponsible engineers..…but have never seen sufficiently credible argument / plausible scenario to spend much time worrying about it..…same culture of safety that is part of engineering mindset should not be discarded..…idea of a step-function, a recursive self-improvement, a foom is 'fanciful'..…the thinking about a step-function is magical thinking that is not in line with reality of current AI.
We should be thinking about pandemics, climate change, nuclear weapons instead. Sam Harris fingered the 'rationality' community as fatalists. Pinker says we need to prioritise between probable issues and imaginable but close to zero-probability ones, or this leads to fatalism or paralysis.
Accepts human effort is 'not well calibrated against risk' because it is based on what is imaginable, compare for example terrorism vs road deaths. Need to calibrate our budget of fear planning to probability.
Culture of engineering is to think about risks.
Psychology of negativity is a phenomenon, part of our negativity bias that is built in to our species.