I finally got my digital hands on a Kindle copy of Nick Bostrom's "Superintelligence: Paths, Dangers, Strategies", and the first thing I checked was how his chapter on "Superintelligent Will" compares to his 2012 paper on "The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents". This subject interests me a great deal for many reasons, among which is the observation that our expectations regarding superintelligence will affect our own attempts to achieve it ourselves -- to become superintelligent posthumanity, which I contend is an implicit aim of every life-affirming theology or pantheon or posthuman projection that has existed since the dawn of history.
On the morning of 24 July 1847, a group of Mormon pioneers broke camp for the last time. They traveled six miles through a deep ravine, across one last creek, and into full view of a great valley. Wondering and admiring, they gazed. The valley appeared vast and richly fertile, clothed with a heavy garb of green vegetation, adorned in its midst with a large lake from which islands rose, and entirely surrounded with a perfect chain of everlasting hills, mountains covered with eternal snow, and innumerable peaks like pyramids towering towards heaven. It was perhaps the grandest and most sublime scenery in the world.
At Welfare Square in Salt Lake City, I'm watching. Across the street is a half-naked malnourished man, smoking a cigarette; a backpack and large coat rest beside him in the shady grass, and he scratches at the air while talking out loud to himself, words I can't make out. A dirty brown pickup pulls in beside me, and a large man climbs out, speaking hurriedly in Spanish; four children -- no, make that a small woman and three children -- exchange a few words with the man, who beckons them off toward a building as he returns to sit in the truck. From the building, an aged women with a sparsely-haired scalp marches to the edge of the parking lot, her serious features relax as a car slows to a stop to let her pass, and she waits as a hunchbacked man, looking at his feet, slowly places one in front of another and again, and again toward her, and again perhaps to cross with her in front of the waiting car.