Thoughts on AI/AGI
I’m not particularly well informed about AI/AGI. Like everyone, I’ve been experimenting with ChatGPT and some of the other accessible tools, find them really great for certain demos, useful for other tasks, and that they break in amusing ways, particularly when asked to do certain kinds of things, such that they are basically only useful if well-supervised by a human.
Most of what’s interesting about them is the potential they show in the technology for the future.
There has been a lot of discussion by others, including people I know personally and have spoken with at length, on both the “AGI doom scenario, AGI might slowly or quickly destroy humanity” and “AGI doomerism is the next climate change or Covid, a fake issue used to lock down technology and maintain central control.”
I tend to be a little more on the side of “AGI doomerism is bad” for a few reasons.
What we have *today* (LLMs like GPT-3) is not a significant risk of “AGI explosion”
Most of the real harms of the “AI” we have today are also possible with less-advanced computer tech (stuff like pervasive monitoring, social credit scores, financial censorship, coordinated government influence over communications providers, third party doctrine compelled access, etc.). AI to the extent we have it today *does* make this stuff worse, but not by orders of magnitude. These are real problems and need to be addressed, though, just independently of AI/AGI.
There’s a weird human bias toward “end of the world” scenarios, especially if they’re “remote” in personal effects — “you plebs need to stop driving your cars to prevent climate change from hurting the whales, I will fly my private jet to a conference to discuss this publicly with other high status individuals” is the precedent. There’s sufficient personal benefit to being seen as “AGI X-risk defender” or “EA” to make this worth doing for many people.
A lot of the real harms we are facing in the near future are due to declining labor productivity; to the extent AI works today, and to the extent AGI is possible, we will get massive productivity gains. Unfortunately these productivity gains also reduce the size of the smallest conspiracy required to accomplish very bad things, too.
A lot of the people who are pushing for “AI Safety” (distinct from the “AGI doom x-risk” folks) are pretty horrible on multiple dimensions and either unfettered AI becomes AGI and solves them (victory condition, even if I’m a paperclip by then…) or powerful AI human-productivity-enhancement-tools also defeat the “AI Safety” people (which is an even better victory condition.)
In some of the AGI doom scenarios, it’s not “AGI has bad motivations of its own” but “AGI inherits bad motivations of creators”; if this is true, I want those to be *my* bad motivations, not someone else’s
One plausible AGI scenario is uplift/enhancement of humans rather than a separate entity; I volunteer for that role, and will cat-maximize.
Usually with good predictions I’d want to identify indicators I can continue to search for which, if they become obvious, will confirm or disprove the prediction before it’s critical or irreversible, but I haven’t figured out the right intermediate indicators for this yet.
OTOH, life as a paperclip is probably fine, and any truly good AGI would actually be a cat-maximizer, not a paperclip-maximizer.