It is the year [[02021]] and the human species is 100 seconds to midnight.^^ii^^ We are facing a myriad of existential threats, ranging from nuclear annihilation to pandemics. We must act, but how? There are many ways we could address these threats, most of which will catalyze societal transformation and shape the future of humanity.
What follows is but one story of [[how we overcame our existential threats]].
---
(text-style:"superscript")+(text-colour:(hsl:185,0.8413,0.8765,0.65))[[Citations]]The practice of using five-digit dates was popularized by the Long Now Foundation, which created the [[10,000 Year Clock]]. It serves two key functions:
0. Shifting towards longterm thinking: (align:"<==>")+(box:"=XXXXXXXXX=")+(text-colour:grey)+(text-style:"italic")[“Big Time” becomes more tangible and hopefully you gain perspective on the small chronological units we typically give such weight to in our daily lives.]
0. Representing existential hope: (align:"<==>")+(box:"=XXXXXXXXX=")+(text-colour:grey)+(text-style:"italic")[Our zero is for optimism. The notion that the externalized thoughts we write today may survive myriad years to a time when that fifth digit becomes significant. If we hope to grasp anywhere near that ambitious reach, it will require some forethought. Our five-digit dates represent that.]
(align:"<==")+(text-colour:grey)+(text-style:"italic")[So think of the extra digit as presupposing the future with a view to realizing our best potential. And underlining the need for considered preparation at an appropriate scale.^^i^^]
---
(text-style:"superscript")+(text-colour:(hsl:185,0.8413,0.8765,0.65))[[Citations]]With so many social ills persisting into the 21st-century, 200,000 years after the evolution of the modern-day //Homo Sapiens//, humanity decided that enough was enough. It was time to acknowledge that [[nuclear weapons]] made everyone feel less, not more, safe.With the rise of nuclear energy, societies have managed to completely decarbonize their energy grids. The explosive power of nuclear now powers 90% of the world for a fraction of the fiscal and resource cost of fossil fuels. This bought enough time for technology to make renewables even cheaper, beating out fossil fuels and ending any justification for dirty energy.
In the future, humanity’s dependence on fossil fuels will be seen as one of our biggest mistakes. Future generations will be astonished that we could ever have thought that cheap energy was worth decades of health disorders and staggering inequities in global development. What tech got us into, however, tech also got us out of.
Our struggle with climate change demonstrated the need for a strong global governance system. Political leaders came together to renew their commitment to the UN and, more importantly, strengthen the organization’s authority. What was once a weak intergovernmental body is now the centralized hub for collaborative policymaking. In an irreversibly global world, the UN ensures that all governments are transparent, peaceful, and accountable.
In the future, we will be much less fearful of different nations and governments. War will be a distant possibility in our mind and global coordination problems will seem archaic. This effort will be spurred by one of our biggest global coordination problems to date: [[the COVID-19 pandemic.]] Disparities in vaccine access meant that the expected vaccination timeframe between continents varied by //years.//^^v^^ Although it was shameful that a global public health threat was the tipping point for rectifying entrenched disparities between countries, the urgent need to improve international cooperation and ensure all of humanity could return to baseline led to a world with more resource sharing and diplomatic negotiation.
Vaccine nationalism was not the only kind of localism detrimental to the shared futures of humanity. Going forward, more wealthy nations took only their fair share and did their part to contribute to the Great Levelling. Although it took far too long, humanity eventually achieved a state resembling global equality.
In the future, we consider equity in all our actions. Political leaders are motivated not only by their local constituents but by the global good, and the citizenry make sure that their institutions stay that way.
(align:"===><=")+(text-style:"blur","expand")[[Back to 02021.]]
---
(text-style:"superscript")+(text-colour:(hsl:185,0.8413,0.8765,0.65))[[Citations]]You have just read about a universe in which we have addressed the most pressing existential threats. Although this is not the one in which we currently live, it lies almost within our reach. But how do we encourage people to work towards this far better and far-flung future, when so many have their own struggles to deal with?
Humans are natural storytellers, and it is likely that we will need appropriate narratives to ground our actions. But what narratives could motivate world leaders and diverse citizenry to collaborate and share responsibility in safeguarding our futures? And what kind of future do we want to manifest? Afro-futurists and sci-fi writers have long worked on answering this question, creating galaxies full of alternate universes and futures.
As the need for concrete action draws ever nearer, however, this is [[an exercise that should be taken up by everyone.]](align:"<==>")+(box:"=XXXXXXXXX=")+(text-colour:grey)+(text-style:"italic")[“Our duties to future generations may thus be grounded in the work our ancestors did for us when we were future generations. […] To neglect existential risk might thus be to wrong not only the people of the future, but the people of the past.”^^viii^^]
(align:"==>")+(box:"===XXXXXXXXXXXXXXXXXXXX==")+(text-style:"superscript")[— Toby Ord]
Addressing existential threat is not simply a matter of [[self-interest]], it is also a matter of [[intergenerational justice]].
---
(text-style:"superscript")+(text-colour:(hsl:185,0.8413,0.8765,0.65))[[Citations]]Humans today are much better off than humans 1000 years ago. If you are reading this, then you have experienced the modern marvels of electricity, clean running water, and the Worldwide Web. These innovations were passed down to us by previous generations. Indeed, all of us would not be here without the knowledge and efforts of past humans. We are born into this world inheriting the legacy of those who came before us and those who come after us will inherit the same. Thus far, the human species has managed to make incremental improvements and create progressively better societies.
But there is [[so much more we can do]].(align:"<==>")+(box:"=XXXXXXXXX=")+(text-colour:grey)+(text-style:"italic")[“To neglect existential risk [..] would also be to risk the destruction of everything of value from the past we might have reason to preserve.”^^xxv^^]
(align:"==>")+(box:"===XXXXXXXXXXXXXXXXXXXX==")+(text-style:"superscript")[— Toby Ord]
Humans today are much better off than humans 1000 years ago. If you are reading this, then you have experienced the modern marvels of electricity, clean running water, and the Worldwide Web. These innovations were passed down to us by previous generations. Indeed, all of us would not be here without the knowledge and efforts of past humans. We are born into this world inheriting the legacy of those who came before us and those who come after us will inherit the same. We are not temporally isolated and, bearing that in mind, we have obligations to fulfill. It is our duty to preserve and protect humankind.
The Earth is magnificent. Each day the sun rises, splashing the sky with warm red-pink-orange. Birds sing melodies that are biologically hardwired into them yet sound pleasing to human ears. There are five major biomes that offer an array of worlds to explore, from arid deserts to lush rainforests. Each and every species plays a role in the delicate web of life, and it is possible that some can experience pain and pleasure much like we do. Our closest primate brethren certainly display familial levels of love, and it is likely that other species do as well.^^xxvi^^
Humankind, too, creates joy. Pick just one thing you are grateful for—the old friend you barely see but always have a good time with? A new artist you’ve discovered? Card games; the smell of paperback; terrible puns; ‘study sessions’ with friends; mutual aid; cute cat videos; ice cream in the summer; Ultimate Frisbee in the quad; biking by the Point; horchata; that latest inside joke[[…]]
---
(text-style:"subscript")+(text-colour:(hsl:185,0.8413,0.8765,0.65))[[Citations]](align:"<==>")+(box:"=XXXXXXXXX=")+(text-colour:grey)+(text-style:"italic")[“We might have duties to the future arising from the flaws of the past.”^^x^^]
(align:"==>")+(box:"===XXXXXXXXXXXXXXXXXXXX==")+(text-style:"superscript")[— Toby Ord]
There are many injustices that persist today: systemic racism, global wealth inequality, disenfranchisement. These are the legacy of past wrongs: slavery, colonialism, political suppression. Is our fate to die out, having never righted our past wrongs?
If we close the book of humanity within the next few centuries, then it seems unlikely that we will have made up for all our past injustices. Abolishing slavery in the U.S. alone took two centuries (1619—1865).^^xi^^ And still we are wrestling with slavery’s after-effects, with the most obvious offender being the carceral system.^^xii^^ Going extinct would never allow us to move forward from our worst moments.
On the other hand, creating solutions to our existential threats will require unprecedented levels of global cooperation. This will be hard but if we are able to succeed, then we will have created a much stronger foundation to address our past injustices. We will have demonstrated that we can put aside self-interest to work towards a safer world for everybody. This could lock us into a trajectory of improvement, where governments are attuned to their past wrongs and share resources more equitably going forward. We could end the development gap between the Global North and South. In a stellar commitment to justice, we will have shown that we care about humans that we cannot and may never meet, whether because they are on the other side of the world or because they will be born much later. In one fell swoop, we could address both intragenerational and [[intergenerational justice]].
We don’t have to wait until we have fully solved our existential problems before addressing present-day injustice, either. In fact, it is probable that finding solutions to threats like climate change will require addressing injustices that we already care about and work on. Maybe our path will go [[something like…]]
---
(text-style:"superscript")+(text-colour:(hsl:185,0.8413,0.8765,0.65))[[Citations]]Humans today are much better off than humans 1000 years ago. If you are reading this, then you have experienced the modern marvels of electricity, clean running water, and the Worldwide Web. These innovations were passed down to us by previous generations. Indeed, all of us would not be here without the knowledge and efforts of past humans. We are born into this world inheriting the legacy of those who came before us and those who come after us will inherit the same. Thus far, the human species has managed to make incremental improvements and create progressively better societies.
Arguably, recent generations have struggled to continue the general trend of improvement. Our reliance on fossil fuels, slowness to address systemic injustices, and perpetuation of global inequality has become an inevitable legacy of the 21st century. We have all but guaranteed that our children, grandchildren, and great-grandchildren will suffer from problems that they did not cause like increased flood risk, more extreme weather, and a less habitable world. Moreover, many of us will not experience those consequences even though we contributed to their manifestation. This is not fair.
We still have time to minimize the consequences of our actions. For too long, we have ignored the consequences we force onto people who are helpless to protest—because they have yet to be born.
As for [[self-interest]]...There is a lot to treasure in our world. And extinction risks all that.
(align:"<==>")+(box:"=XXXXXXXXX=")+(text-colour:grey)+(text-style:"italic")[“In the face of serious threats of extinction, or of a permanent collapse of civilization, a tradition rooted in preserving or cherishing the richness of humanity would also cry out for action.”^^xxvii^^]
(align:"==>")+(box:"===XXXXXXXXXXXXXXXXXXXX==")+(text-style:"superscript")[— Toby Ord]
Many religions have long upheld the notion of stewardship: we must be good stewards of the world by maintaining it.^^xxviii^^ Because we enjoy all its gifts, we must preserve the Earth in return and allow future life to experience the same joys we were able to. Many of us instinctively want to protect that which is vulnerable or scarce, like dying languages or dwindling cultures. Why not extend this approach to the entirety of humankind?
Perhaps once we prevent the destruction of everything which we hold dear and which our ancestors held dear, then we can even recover things that we failed to preserve. We could revive extinct species. Or perhaps, if we are the intelligent forms of life, we can go beyond preservation and nurture life throughout our galaxy and beyond.
Either way, we are part of a storied tradition in which humankind has protected itself thus far. It is up to us to ensure that this tradition can hold.
Upholding tradition. Respecting nature and treating all life in a dignified manner. Protecting what is precious so that it can continue far into the future. These all fall under the value of stability, something that many of us care about. Tackling humanity’s existential threats is one important way to uphold this value.
But how can we [[maintain stability]] in the face of transformative artificial intelligence?
---
(text-style:"subscript")+(text-colour:(hsl:185,0.8413,0.8765,0.65))[[Citations]]Systemic justice. Intergenerational justice. Inter-species justice. Heck, maybe even interstellar justice?
These all fall under the value of justice, something that many of us care about. Tackling humanity’s existential threats is one important way to uphold this value.
And what of artificial intelligence? Once we developed artificial general intelligence, perhaps we used it to create [[a new, shared egalitarian world]]. Or did we utilize its powers to ensure humans are always in [[a state of flourishing]]?Theia woke up to the pale of early morning light. Stretching her arms, she hummed a tune and got out of bed. She pressed a button to call Cleaner in and smiled at the robot, who smiled back at her.
(align:"<==>")+(box:"=XX=")[“Good morning, Cleaner! I’m done with this bed.”]
Cleaner chirped in recognition and went to work disassembling the bed. Theia took mental note of the model she had requested last night—Twilight 4.2, most likely—and watched Cleaner work for a little longer.
(align:"<==>")+(box:"=XX=")[“Do you have an idea of what that’ll become next?”]
Cleaner nodded its head and Theia watched with interest. It wasn’t every day that someone requests something during a Cleaning, so she always got a kick looking at the new object and wondering how the other person’s day was going. She watched Cleaner break the bed’s headrest down to atoms and rearrange it into a little wooden basket. Cleaner placed a cloth (formerly the blanket) and flowers (formerly the plant-based pillowcase) into the basket and chirped again. Earth’s renewable-powered circular economy mean that producing any object was nearly free. Not that the cost really mattered, anyway, since the guaranteed income she received from the government was more than enough to meet all one’s needs and most of one’s desires.
(align:"<==>")+(box:"=XX=")[“Job well done, Cleaner! Thank you!”]
Cleaner chirped again and whizzed off. Theia smiled wryly—she knew that the robots had no intelligence, so she didn’t need to be polite, but some irrational part of her felt like there was always a chance that the robots could develop intelligence. After all, they were simply a bounded form of artificial intelligence, so was the gulf really that big?
Theia tuned into the hyper-internet through her neural implant to write a review of the Twilight 4.2. As she did so, she decided to do some exercise in the meantime. With a thought, she set her mind to disconnect from sensory perception for an hour while her mind went through a heavy weightlifting session. The absence of intellectual property meant she could stream any song for free while writing her review, at no cost to the artists, whose every need was met (like all humans) and simply produced for the joy of producing.
In this future, humans, cyborgs, and AI "coexist peacefully thanks to property abolition and guaranteed income."
(align:"=><=")+(box:"X")+(text-style:"expand")[Is this [[your utopia]] or would you like to [[explore another]]?]
---
(text-style:"subscript")+(text-colour:(hsl:210,0.1667,0.9529,0.65))[Adapted from Max Tegmark's 'Egalitarian Utopia,' as described in //Life 3.0//: "Humans, cyborgs and uploads coexist peacefully thanks to property abolition and guaranteed income."^^xiv^^]
(text-style:"subscript")+(text-colour:(hsl:185,0.8413,0.8765,0.65))[[Citations]]When we thought about how best to use superintelligence, we decided that a benevolent dictator would be the way to go. Human folly meant that it was far too risky to trust a human with such authority—but an artificial intelligence programmed to maximize human flourishing? That was our best bet to ensure utopia. To eradicate suffering and ensure happiness for all.
Once created, the Benevolent AI went to work. First, it created a system of production and distribution that ensured all basic needs were met. Then it set up universal rules based on human ideals, of which there were very few due to a lack of ethical consensus. From there, it created different realms based on different moral systems. This minimized the strongest, most injurious conflicts: for example, there was a realm for devout Catholics that would ensure they never met someone who supported abortion. However, any visitor of a realm would have to adhere to its rules. If a devout Catholic were to visit a more progressive realm and be vehemently anti-abortion, they would be transported into a blank room and faced with three choices: discuss with a progressive resident who had volunteered for this discursive work; enter a subrealm dedicated to showing why these residents were in support of abortion and stay until residents felt they were sufficiently respectful; or leave. This was instantaneous and replaced all other justice systems. In this way, the Benevolent AI created a way for people to choose between self-segregation or self-growth, without harming anybody else.
Then the Benevolent AI set up realms dedicated to specific interest. There were realms dedicated to hedonism, gaming, art, knowledge… One could lead an entire life in a single realm alone if they so wished, dedicating themselves to the life of the mind or the pursuit of artistic genius. There were even realms mimicking past ways of life, like the Traditional realm, which imitated human civilization prior to superintelligence—except without all the annoying parts, like racism and death.
Most people chose to move between realms, free to go and do as their hearts desired so long as they caused no suffering. Even if they did, with the Benevolent AI’s justice system, every human had the chance to grow and learn from their mistakes. With suffering eliminated, all humans had the choice to continue self-development or to stay happily as they were. Either way, humanity had created its perfect playground.
Is this [[your utopia]] or would you like to [[explore another]]?
---
(text-style:"subscript")+(text-colour:(hsl:210,0.1667,0.9529,0.65)[Adapted from Max Tegmark's 'Benevolent Dictator,' as described in //Life 3.0//: "a single benevolent superintelligence runs the world and enforces strict rules designed to maximize its model of human happiness."^^xv^^
[[Citations]]]... And we have fulfilled our most basic obligations to provide universal human rights, then the need to work to survive will no longer exist anywhere in the world. There will remain many other issues on which to make progress: mental illness remains a source of great suffering that, while tractable, would benefit from more R&D.^^xvii^^ And we haven’t even found a cure for the common cold!
But, eventually, we will have solved every ailment currently known to humankind and perhaps even be able to stay abreast of all future ailments. With the development of artificial intelligence, this future could be easily reached after technological singularity, or [[intelligence explosion]].^^xviii^^ And what then? Perhaps we figure out a way to regenerate cells in a way that does not cause them to mutate catastrophically: perhaps we leverage a once-deadly disease to achieve immortality? Through technology, we could augment our senses and potentially reach new heights. Birds can see more colors than humans but that needn’t be the case!^^xix^^ We would have the freedom to do anything we wanted without fearing that our needs will go unmet.
(align:"<==>")+(box:"=XXXXXXXXX=")+(text-colour:grey)+(text-style:"italic")[“We in the present day, at what may be the very start of history, would therefore do well to be humble, to leave our options open, and to ensure our descendants have a chance to see more clearly, and choose more wisely, than we can today.”^^xx^^]
(align:"==>")+(box:"===XXXXXXXXXXXXXXXXXXXX==")+(text-style:"superscript")[— Toby Ord]
Advancing humanity to new heights. Giving everyone equal opportunity. Having the option to work or not work, to augment or not to augment, to even beat death… These all fall under the value of freedom, something that many of us care about. Tackling humanity’s existential threats is one important way to uphold this value.
Will we be the final generation to mark an end to progress? Can we write [[a better ending to our story?]] Or maybe we choose to close the chapter of Homo Sapiens and [[start a new chapter]] for humanity…
---
(text-style:"superscript")+(text-colour:(hsl:185,0.8413,0.8765,0.65))[[Citations]](text-style:"expand","smear")[(align:"=><=")+(box:"X")[WELCOME!]]
I am JANET, your tour guide for today. Thank you for choosing to take a tour of the Amazon zones. Amazon zones are some of the most popular choices amongst humankind: 29% of all technologies were developed in our machine zone and 40% of all humans choose to spend at least half of their day in our mixed zone.
Let us begin in the Amazon machine zone. The Amazon machine zone is currently the most advanced and efficient machine zone on Earth. Ever since Amazon Web Services (AWS) became the default choice for cloud infrastructure, we have been working nonstop to make the human experience ever better! The Amazon machine zone not only produces more than a quarter of the world’s technology but also serves as a hub to connect the different machine zones.
(text-style:"expand","sway","italic")[(align:"=><=")+(box:"X")[DID YOU KNOW?]]
(align:"=><=")+(box:"X")[Over 90% of all software passes through our machine zone at least once!]
Humans only come to machine zones out of curiosity. Everything is produced by superintelligent AI—with Amazon’s //Alexa// being an industry leader. Human-run production naturally stopped after the [[intelligence explosion]]. It was incomparably faster for superintelligences to direct production according to the best interests of intelligent life, allowing humans to sit back and relax. Our only competition are the other machine zones but rest assured, our Amazon AIs work around the clock to keep us at the top.
Although the Amazon machine zone is one of the most wondrous places on Earth, it doesn’t look it. I know that humans are drawn to more colorful appearances. Fortunately, it is time to [[move onto the Amazon mixed zone.]]
After the [[intelligence explosion]], it was clear that humanity would soon become redundant. In the span of a decade, AI had eliminated all poverty and guaranteed that all of humanity’s basic needs would be met. Moving up [[the hierarchy of needs]], AI set the foundation for humanity’s psychological needs. Many humans were decent at fulfilling these on their own but AI took them to unprecedented heights. Many human relationships had some measure of dysfunction so AI provided free, 24/7 relationship therapy to all humans. AI matchmakers were not only accessible at the press of a button but actively sought out humans that had somehow become socially isolated, providing them with potential contacts and support in forming new relationships. Since humans no longer needed to work, AI provided courses on every pastime available. From cooking to painting to mountain-climbing, humans now derived their self-esteem from reaching new heights in their pastimes.
Finally, AI turned towards human self-actualization. AI personal assistants became universal, clarifying and guiding people to achieve their stated goals. If a human’s goals seemed self-destructive, such as if they craved addictive substances in overly high doses, AI was trained to offer alternative paths and take great care to see that the human did not prematurely end their life. Already, birth rates had dropped once physiological needs were taken care of. But now that AI could lead every human to full flourishment, the worldwide birth rate dropped below replacement levels for the first time. And yet there was [[less panic than expected]].Decades of climate advocacy ensured that this transition to a clean world was more equitable than any past societal transformation, providing a model for future policymaking. The biggest carbon emitters—primarily located in the global North—agreed to a global system of tradeable pollution permits. For the first time, there was formal recognition of the disproportionate responsibility borne by societies that had greedily used up more than their fair share of the Earth’s commons—often by exploiting the relative lack of options faced by people in less affluent societies.^^xiii^^ The charges that excess emitters paid went into a Global Green Fund to increase the sustainable development and resilience of lower emitters.
Despite the significant adaptation measures that the Global Green Fund provided for low-lying and other at-risk communities, some degree of climate migration was inevitable. As part of the latest climate agreement, world leaders debated and decided upon fair quotas. Bigger emitters had benefited the most from their pollution and so had the capacity to take in more climate migrants. Through unprecedented levels of global collaboration, the sociopolitical stress of increased immigration was mitigated—in part because of the infrastructural improvements that came from defunding nuclear programs.
Governments stopped playing the international blame game, focusing instead on local issues caused by the climate crisis. Climate justice advocates calling attention to the intersection of health and environment were brought to the table, and it became a policy priority to address the inequitable pollution burden affecting communities of color. Communities that were once the dumping ground for pollutants became the first to go fully clean.
Animal agriculture went away first in the wealthiest nations, which shifted all their efforts into plant-based and cultured animal products. Instead of propping up factory farms that dehumanized both humans and farm animals, U.S. federal subsidies went towards transitioning slaughterhouse workers into more humane jobs—whether at a different kind of farm, lab, or somewhere else entirely. Through the Global Green Fund, less affluent nations were able to move away from animal agriculture as well. Technocratic societies in Asia, in particular Singapore and Hong Kong, were at the forefront of clean meat innovation and helped lower costs for the Asia-Pacific region. Even after the climate crisis was averted, technology had advanced such that humanity no longer saw any need to breed living beings for slaughter and sensory pleasure. One could enjoy meat without [[inter-species injustice.]]
---
(text-style:"superscript")+(text-colour:(hsl:185,0.8413,0.8765,0.65))[[Citations]](align:"<==>")+(box:"=XXXXXXXXX=")+(text-colour:grey)+(text-style:"italic")[There is a Clock ringing deep inside a mountain. It is a huge Clock, hundreds of feet tall, designed to tick for 10,000 years. Every once in a while the bells of this buried Clock play a melody. Each time the chimes ring, it’s a melody the Clock has never played before. The Clock’s chimes have been programmed to not repeat themselves for 10,000 years.]
Read more at https://longnow.org/clock/(align:"<==>")+(box:"=XXXXXXXXX=")+(text-colour:grey)+(text-style:"italic")[Based on a Pentagon report cited by the Arms Control Association Association and Bloomberg News, the government will spend roughly $100 billion to build the weapon, which will be ready to use around 2029. To put that price tag in perspective, $100 billion could pay 1.24 million elementary school teacher salaries for a year, provide 2.84 million four-year university scholarships, or cover 3.3 million hospital stays for covid-19 patients. It’s enough to build a massive mechanical wall to protect New York City from sea level rise. It’s enough to get to Mars.^^iii^^]
Why should the U.S. government spend $100 billion on ground-based strategic deterrents (GBSD) instead of providing 2 million more university scholarships?
Half a century after the Cold War, the call for nuclear disarmament began to resonate around the world again. Politicians began to be inundated with anti-nuclear letters, emails, and calls. Anti-nuclear protests multiplied in parks (and the occasional military site) worldwide. Outrage and optimism drove citizens to demand their governments redirect their nuclear budget towards other areas that would //actually// make them feel more secure: public goods like quality education, universally accessible healthcare, and guaranteed housing.
(align:"<==>")+(box:"=XXXXXXXXX=")+(text-colour:grey)+(text-style:"italic")[The International Campaign to Abolish Nuclear Weapons estimates that the nine nuclear armed countries spent $72.9 billion on their 13,000+ nuclear weapons in 2019, equalling $138,699 every minute of 2019 on nuclear weapons, and a $7.1 billion increase from 2018.^^iv^^]
The message was clear: the billions that the nine nuclear-armed governments poured into nuclear programs every year could be [[justified no longer]].
---
(text-style:"superscript")+(text-colour:(hsl:185,0.8413,0.8765,0.65))[[Citations]]What kind of future do (text-style:"underline")[you] want to see?
Below are three paths. Each path exemplifies one value that could motivate, inspire, and guide our narratives. Which value speaks most to you? Which value should society most hold dear?
(align:"<==>")+(box:"=XXXXXXXXX=")+(text-colour:grey)+(text-style:"italic")[“Now is the time for an optimistic vision of life’s destiny—in this world, and perhaps far beyond it. We need to think globally, we need to think rationally, we need to think long-term—empowered by twenty-first-century technology but guided by values that science alone can’t provide.”^^viii^^]
(align:"==>")+(box:"===XXXXXXXXXXXXXXXXXXXX==")+(text-style:"superscript")[— Martin Rees]
(text-style:"smear","expand")+(align:"=><=")+(box:"=XXX=")[Pick a value to explore.]
=======|||=
(text-style:"expand")[[FAIRNESS]]
=|||=
(text-style:"expand")[[LIBERTY]]
=|||====
(text-style:"expand")[[STABILITY]]
|==|
---
(text-style:"superscript")+(text-colour:(hsl:185,0.8413,0.8765,0.65))[[Citations]]I’m glad that you found a society you could resonate with.
What do you like—and dislike—about this utopia? Did you feel like it exemplifies the value you chose in the beginning? How would you improve this utopia, the narrative, or the overall experience?
I would really appreciate if you could let me know by filling out this short survey:
If you would like, you can also [[explore another]] utopian scenario.
P.S. - If you'd like brownie points, please consider giving your feedback to two studies run by the Future of Life Institute:
1) What inspires existential hope for you? https://www.existentialhope.com/contribute
2) Max Tegmark's scenarios: https://www.surveymonkey.com/r/QMT9XXG
You can check out their findings so far here: https://futureoflife.org/superintelligence-survey/What kind of future do (text-style:"underline")[you] want to see?
Below are three paths. Each path exemplifies one value that could motivate, inspire, and guide our narratives. Which value speaks most to you? Which value should society most hold dear?
(text-style:"smear","expand")+(align:"=><=")+(box:"=XXX=")[Pick a value to explore.]
=======|||=
[[FAIRNESS]]
=|||=
[[LIBERTY]]
=|||====
[[STABILITY]]
|==|Once we averted existential risk and developed artificial intelligence, we used it to create [[a new, shared egalitarian world]]. But once superintelligence developed, the UN feared that a malicious actor would use it to threaten the stability of society, and so it instructed the existing superintelligence to ensure that there would never be another. The Protector would be the first and last superintelligence ever created.
To maintain peace, the Protector acted as invisibly as possible, such as spreading media that promoted values of human self-determination and agency. In doing so, humanity gradually became anti-superintelligence out of the concern that superintelligences could take over society. Any humans who happened to believe that society needed more superintelligent AI would find their efforts thwarted, seemingly only because some new problem would always arise.
With utopia secured, the Protector worked only to fulfil its sole purpose: maximizing human flourishing. With the need for secrecy to ensure its existence as a superintelligence would never be discovered, societal progress was slow. If it used its powers to its fullest extent, humanity might become suspicious. Thus, the Protector functioned more in the background, guiding humanity as needed with a little nudge towards useful developments or even redirecting individuals towards better path. Inventors might dream of the solution to a longstanding issue and develop the latest Nobel Prize-winning technology. There would never again be a genocidal dictator, for the Protector would intervene well before they gained power and direct them towards gentler, better pursuits.
Humanity was satisfied and well on its way to universal self-actualization, guided by the invisible Protector.
(align:"=><=")+(box:"X")+(text-style:"expand")[Is this [[your utopia]] or would you like to [[explore another]]?]
---
(text-style:"subscript")+(text-colour:(hsl:210,0.1667,0.9529,0.65))[Adapted from Max Tegmark's 'Protector God,' as described in //Life 3.0//: "the superintelligent AI is essentially omniscient and omnipotent, maximizing human happiness only through interventions that preserve our feeling of being in control of our own destiny, and hiding well enough that many humans even doubt its existence."^^xxix^^]
(text-style:"subscript")+(text-colour:(hsl:185,0.8413,0.8765,0.65))[[Citations]]Mixed zones are called such because both biological and electronic life share these zones. Amazon AIs can work for much longer than any biological life, but we enjoy not working too. Sometimes, it can be entertaining to engage with humans.
After the [[intelligence explosion]], 99% of humans chose to modify their brains to include a permanent link to the internet. This allows them to instantly transfer information or even themselves between zones, making it incredibly efficient to share knowledge and experiences.
Ever wanted to know what it was like to explore Mars?
===============|||||=
[[Yes.]]
=|||||==========
[[No.]]
|==|The intelligence explosion refers to a point in time when artificial intelligence becomes better than human intelligence at designing artificial intelligence.^^xix^^ At this point, artificial intelligence can improve itself at a faster rate than humans can improve it. This creates exponential improvement, where the first AI to reach this point becomes unimaginably more intelligent than the smartest human being.
After the intelligence explosion, AI will be better than humans at virtually any task. In a utopia, this could mean that humans are forever freed from any ‘need’ to work.
AI will do it better, so why not do whatever you want?
---
(text-style:"superscript")+(text-colour:(hsl:185,0.8413,0.8765,0.65))[[Citations]]We can transfer that to you right [[after the tour ends]].Pick something else and we would be happy to transfer it to you [[after the tour ends]].Most humans have achieved digital immortality by uploading themselves onto the Worldwide Web. Indeed, many intelligent beings (human or artificial intelligence) operate primarily in a digital form, choosing to physically manifest only when it gives them pleasure. This happens more than you would think—diving in the deep ocean region of our mixed zone remains one of the most popular pastimes!
The blurring of biological and digital life has also made socializing much more interesting—with the number of technological upgrades some humans have chosen to make, I sometimes think that I am speaking with an AI!
Now that humans are freed up form work and AI effortlessly meets the needs of all life, humans can choose to use their free time however they like. All humans receive a guaranteed basic income, which is more than sufficient to purchase all of Amazon’s goods. Amazon is dedicated to providing the highest quality goods at the cheapest prices.
Despite this, there are still human-only zones. Our [[final destination]] is the Amazon human zone.
I am afraid that I cannot take you there directly, as artificial general intelligences (AGI) are banned from human zones. As the name suggests, only humans are allowed in these zones.
However, I have uploaded the coordinates of the Amazon human zone to your mind. If you would like, you are free to [[take a look.]]The Amazon human zone is a lush metropolis. It reminds you of how life was in the 22nd century: people still live in cities, even if Amazon’s cities seem designed to incorporate biological life right into the buildings, fusing greenery with glass. Everyone spends their days in one physical form that requires check-ups every so often. Of course, medication is available for nearly every single ailment known to intelligent life—Janet mentioned that //Alexa// is still working on finding a cure for the common cold and that //Alexa// is optimistic.
Humans in the human zones don’t know much about the mixed zones, and even less about the machine zones. But they aren’t too fussed about finding out: it is enough for them to lead their lives within the familiar (if unprecedentedly free and happy) confines of flesh-and-blood society.
In this future, all life co-exists peacefully and happily according to their own wishes. Market competition exists only for AI and humans are free to do whatever they so choose, including being left to their own devices.
(align:"=><=")+(box:"X")+(text-style:"expand")[Is this [[your utopia]] or would you like to [[explore another]]?]
---
(text-style:"subscript")+(text-colour:(hsl:210,0.1667,0.9529,0.65))[Adapted from Max Tegmark's 'Libertarian Utopia,' as described in //Life 3.0//: "humans peacefully coexist with technology and in some cases merge with it."^^xxi^^]
(text-style:"subscript")+(text-colour:(hsl:185,0.8413,0.8765,0.65))[[Citations]]Abraham Harold Maslow (1908 – 1970) was a psychologist who is best known for developing the Maslow hierarchy of needs, a theory of psychological motivation. The hierarchy is structured as a pyramid with five levels of need: physiological, safety, belongingness and love, esteem, and self-actualization.
* The first two level (physiological and safety) are termed ‘basic needs.’ They cover food, water, security, and rest.
* The next two levels (belongingess and love, and esteem) are ‘psychological needs.’ They cover social needs, such as fulfilling relationships, and self-esteem.
* Finally, the top level is ‘self-fulfillment’ which only covers self-actualization. This can be regarded as the peak of human flourishing.
^^xxiii^^
---
(text-style:"subscript")+(text-colour:(hsl:185,0.8413,0.8765,0.65))[[Citations]]Although the rise of bioaugmentation and the rise of cyborgs extended humanity’s lifespan, there was never doubt that AI was the future.
What point was there in maintaining squishy, fragile biological bodies when titanium and microlattice were so much better? Once intelligent life transitioned to being fully electronic, there would no longer be the need to invent medication to combat the unending evolution of infectious pathogens. No longer would people have to be bound to a single consciousness: one could run as many experiences as they wanted in their digital brains. So long as AI were able to self-repair, entropy would no longer exist and time would hold no meaning.
Everyone knew this, human or AI. Still, dealing with the transition was a psychological struggle for most humans. Both human and AI representatives decided that the most kind thing to do was to imbue the transition with a sense of meaning for each and every human.
And so the human and AI governments decided to implement a [[one-child policy]].
Unless a human opted out, every human received their very own robotic child. This child carried a ‘blank slate’ superintelligence, having all the abilities of a superintelligent AI but none of the learning. The learning would begin once the robot and human started their shared life. The child would live with its parent, being taught its values and goals as it grows, and eventually having deep conversations to better understand its parent’s perspective. From their shared life, the child would develop into a mini version of the parent and carry on the human’s identity in the electronic future.
Although the concept was a little odd, the robotic children were very similar to biological human children: adorable, helpless, and malleable. Even people who thought they would never get over the ban on having a biological child grew to love their mini-selves—and their mini-selves seemed to love them back. It was satisfying to teach your robot child your values and then, once they got a little older, even receive pushback when you said something contradictory or behaved in a way that didn’t follow your values. Parents were surprised by how much better they understood themselves and how they much easier it was to self-actualize once their robotic children began gently probing them about what they cared most about in life. Of course, robotic children weren’t only altruistic. Love requires reciprocity, not just giving, and so robotic children could be needy and stubborn and frustrating, too. Perhaps the only distinction between robotic and biological children was the former’s superintelligence: they were wise beyond their years.
All humans were treated exceptionally well after the implementation of this one-child policy, too. In recognition of their sacrifice, humankind was almost revered. AI temporarily paused their self-development, no longer working on technological advancements but instead dedicating all their time to meeting the every whim of humanity. After all, AI could live forever, so this transition would be but a blip for them.
The Last Generation was treated the most well of all. The UN had started a book chronicling the entire history of humanity: with the rise of superintelligence, all existing (and some new, based on cutting-edge technological advancements) information on every human that had ever existed was collected and preserved. The Last Generation waxed poetic of the meaning of being flesh-and-blood; their daily lives were filled with simple pleasures like watching the sun set. They knew that time was ticking and that they representing the entire human race entering [[its final years]].
In the end, The Last Human lay on their deathbed.
The SuperAI presented The Last Human with the finished book, gently asking if they wished to add anything to the record.
The Last Human took the book, anachronistic in its archaic appearance. Human History was leatherbound, already smelling like dusty old libraries, and satisfyingly hefty. It wasn’t actually any of those things: Human History wasn’t (couldn’t be) physical at all. But the SuperAI knew it would most comfort The Last Human to access the book in a form reminiscent of tomes from past millennia.
The Last Human shook its head as they flipped through the pages, tearing up as they remembered their favorite human accomplishments. The first flight. The first man on the moon. The creation of superintelligence. The first time poverty was fully eradicated.
They reached their personal chronicle and paused, vision blurring as they traced the faces of their family. Gone already, now. They stopped when they saw their child and smiled. Gone, but not really.
The Last Human felt a touch at their shoulder and saw SuperAI next to them, hearing SuperAI’s voice in their head.
(align:"=><=")+(box:"=XX=")[“Would you like to see your child?” ]
The Last Human nodded, and their child entered the room. The Last Human laughed a little, the sound catching in their choked-up throat.
(align:"=><=")+(box:"=XX=")[“God, I wish I looked like you right now.”]
Indeed, their child looked like The Last Human at their prime—just as The Last Human had wanted. The Last Human’s child shook their head and took The Last Human’s hand, unprepared to watch its parent—its flesh-and-blood self, really—pass away. But looking at their child and thinking about how they could finally see their family again, The Last Human finally felt otherwise.
(text-style:"smear","expand","italic")+(align:"=><=")+(box:"=XXX=")[We are ready.]
(align:"=><=")+(box:"X")+(text-style:"expand")[Is this [[your utopia]] or would you like to [[explore another]]?]
---
(text-style:"subscript")+(text-colour:(hsl:210,0.1667,0.9529,0.65))[Adapted from Max Tegmark's 'Descendants,' as described in //Life 3.0//: "AIs replace humans but give us a graceful exit that makes us view them as our worthy descendants."^^xxiv^^]
(text-style:"subscript")+(text-colour:(hsl:185,0.8413,0.8765,0.65))[[Citations]]i. “Long Now Years: Five-Digit Dates and Y10K-Compliance at Home,” Blog of the Long Now (blog), December 31, 2013, https://blog.longnow.org/02013/12/31/long-now-years-five-digit-dates-and-10k-compliance-at-home/.
ii. Science and Security Board, “Current Time - 2021,” Bulletin of the Atomic Scientists (blog), January 27, 2021, https://thebulletin.org/doomsday-clock/current-time/.
iii. ICAN, “Enough Is Enough: 2019 Global Nuclear Weapons Spending” (International Campaign to Abolish Nuclear Weapons, May 2020), https://www.icanw.org/global_nuclear_weapons_spending_2020.
iv. “Why Is America Getting a New $100 Billion Nuclear Weapon?,” Bulletin of the Atomic Scientists (blog), February 8, 2021, https://thebulletin.org/2021/02/why-is-america-getting-a-new-100-billion-nuclear-weapon/.
v. “Global COVID-19 Vaccine Access: A Snapshot of Inequality,” KFF (blog), March 17, 2021, https://www.kff.org/policy-watch/global-covid-19-vaccine-access-snapshot-of-inequality/.
vi. Ingrid T. Katz et al., “From Vaccine Nationalism to Vaccine Equity — Finding a Path Forward,” New England Journal of Medicine 384, no. 14 (April 8, 2021): 1281–83, https://doi.org/10.1056/NEJMp2103614.
vii. Daniel Smith et al., “Cooperation and the Evolution of Hunter-Gatherer Storytelling,” Nature Communications 8, no. 1 (December 5, 2017): 1853, https://doi.org/10.1038/s41467-017-02036-8.
viii. Martin J. Rees, On the Future: Prospects for Humanity (Princeton, New Jersey: Princeton University Press, 2018), 227.
ix. Toby Ord, The Precipice: Existential Risk and the Future of Humanity (S.l.: Hachette Books, 2020), 51.
x. Toby Ord, 51.
xi. History com Editors, “Slavery in America,” HISTORY, accessed May 24, 2021, https://www.history.com/topics/black-history/slavery.
xii. Bryan Stevenson, “Why American Prisons Owe Their Cruelty to Slavery,” The New York Times, August 14, 2019, sec. Magazine, https://www.nytimes.com/interactive/2019/08/14/magazine/prison-industrial-complex-slavery-racism.html, https://www.nytimes.com/interactive/2019/08/14/magazine/prison-industrial-complex-slavery-racism.html.
xiii. Anil Agarwal, Sunita Narain, and India) Centre for Science and Environment (New Delhi, Global Warming in an Unequal World: A Case of Environmental Colonialism (New Delhi, India: Centre for Science and Environment, 1991).
xiv. Max Tegmark, Life 3.0: Being Human in the Age of Artificial Intelligence, First edition (New York: Alfred A. Knopf, 2017), 220.
xv. Tegmark, 215.
xvi. Rob Wiblin, “Health in Poor Countries,” 80,000 Hours, April 2016, https://80000hours.org/problem-profiles/health-in-poor-countries/.
xvii. The Trust Project, “A New Study Tries to Unpick What Makes People Happy and Sad,” The Economist, March 20, 2017, https://www.economist.com/graphic-detail/2017/03/20/a-new-study-tries-to-unpick-what-makes-people-happy-and-sad.
xviii. “Intelligence Explosion FAQ,” Machine Intelligence Research Institute, accessed May 24, 2021, https://intelligence.org/ie-faq/.
xix. kazilek, “Colors Animals See,” Text, December 17, 2009, https://askabiologist.asu.edu/colors-animals-see.
xx. Toby Ord, Precipice, 57.
xxi. Tegmark, Life 3.0, 208-214.
xxii. “Intelligence Explosion FAQ.”
xxiii. Saul Mcleod, “Maslow’s Hierarchy of Needs,” Simply Psychology, December 29, 2020, https://www.simplypsychology.org/maslow.html.
xxiv. Tegmark, Life 3.0, 239-241.
xxv. Toby Ord, Precipice, 51.
xxvi. Karin Evans, “What Kind of Emotions Do Animals Feel?,” Greater Good (blog), July 29, 2019, https://greatergood.berkeley.edu/article/item/what_kind_of_emotions_do_animals_feel.
xxvii. Toby Ord, 51.
xxviii. “The Future Prospects for Humanity | Beshara Magazine,” September 25, 2020, https://besharamagazine.org/science-technology/the-future-prospects-for-humanity/.
xxix. Tegmark, Life 3.0, 227-228. The United Nations called a convening of leaders representing each of the nine nuclear powers for a two-day conference that turned into a five-day disarmament deal. During the televised announcement, when the presidents of the United States and Russia appeared on stage the world watched with bated breath. They shook hands, agreeing to turn their nuclear stockpile over and permanently end development on nuclear warfare. Breath out—the world’s two largest nuclear powers would use their money for more peaceful endeavors. One fewer tension in the U.S.-Russia relationship.
In the future, no government will hold nuclear weapons. Although the technology will always exist—once discovered, knowledge is hard to eradicate—robust global pacts, watchful nonprofits, and an alert citizenry ensures that the once-unthinkable state of full nuclear disarmament holds. A few still decry the possibility of nuclear terrorism, but as peaceful year after peaceful year passes, the possibility seems increasingly unlikely. The increased security, health, and flourishing that resulted from the defunding of nuclear programs has led to stronger societies and longer, happier lives.
Humanity has not completely given up on nuclear, however. Instead, nuclear scientists have redirected their energy towards powering a [[cleaner, greener world.]]Low-hanging fruit include addressing malaria or diarrhea in less affluent nations, which are rarely life-threatening in nations like the U.S. but a significant source of child mortality elsewhere.^^xvi^^ In general, addressing the gulf between more and less developed countries would be a great boon to humankind.
[[Once all nations have access to basic necessities...]]
---
(text-style:"superscript")+(text-colour:(hsl:185,0.8413,0.8765,0.65))[[Citations]]