These readings present two overlapping crises of the current AI moment, both of which have been on my mind a lot lately. With character.ai being sued into bankruptcy because of the multiple cases of harm resulting from engagement morphing into addiction for its users, you can easily see exactly what Turkle warns of in her piece about the breakdown of the empathetic order that connects an individual to themselves and then to others and extending out to the larger society as presented by Thoreau’s three chairs of a virtuous circle.
Compound this with the second reading on LLMs being compression agents for the whole of the internet, or arguably, all information that can be datafied, it becomes increasingly more bleak. One of my biggest fears about the very near future is the disintegration of anything that could be referred to as a consensus reality.
This isn’t a new problem, Meta has been facing lawsuits and governmental crackdowns and hit pieces for ages based on the way they essentially create a polydisperse foam of reality bubbles across their social platforms but something about the way LLMs can isolate and fabricate feels so much more troubling because we have seen it help write suicide notes and walk through instructions. Not that people don’t do these things but at least there is a person, or persons, to be held accountable not just a black box of question marks. I have watched AI take a PDF I had given it to extract text for a speech reader, instead spit me back a vaguely incorrect and conceptually deflated version of that text in which I had only caught it by accident. I have read recently in the case the AI giants are fighting against the major news organizations the claim that they aren’t stealing because AI is an aggregate that offers a legitimately competitive platform for news consumption, so much so that it is now slowly rising as the number one way people access information about the world, historically and in real-time.
This is a very damning thought. That not only is AI offering a tailored distillation of reality on an individual basis, that has a high risk of being a total fabrication, it is doing this at the complete detriment of any kind of shared social externalities. This could make a trillion tiny little independent worlds with no seeming crossroads. This is extremely concerning when there is a concentrated group of very wealthy and powerful people who sit atop these industries, believing in a neo-monarchic rulership that all tend to agree on a version of a shared reality despite the rest of us. Even if that shared reality feels like something out of a deeply dystopian science fiction novel.
