Enthusiasms

Where’s My Goddamn Jetpack?

This is SHRDLU. Terry Winograd developed SHRDLU at MIT from 1968-70. It’s an experiment in artificial intelligence: a virtual world filled with geometric blocks, and an AI operator that you can ask about the block world or command to do things—stack the blocks, say—in natural language. The AI understands a large subset of natural language pertaining to moving the blocks around, and has a memory. Here is some of what it can do:

Person: HAD YOU TOUCHED ANY PYRAMID BEFORE YOU PUT THE GREEN ONE ON THE LITTLE CUBE?
Computer: YES, THE GREEN ONE.
Person: WHEN DID YOU PICK IT UP?
Computer: WHILE I WAS STACKING UP THE RED CUBE, A LARGE RED BLOCK AND A LARGE GREEN CUBE.
Person: WHY?
Computer: TO GET RID OF IT.
Person: WHY DID YOU DO THAT?
Computer: TO CLEAN OFF THE RED CUBE.
Person: WHY DID YOU CLEAR OFF THAT CUBE?
Computer: TO PUT IT ON A LARGE GREEN CUBE.
Person: WHY DID YOU DO THAT?
Computer: TO STACK UP THE RED CUBE , A LARGE RED BLOCK AND A LARGE GREEN CUBE.

SHRDLU is the pinnacle of artificial intelligence. It’s forty years old, and we are no closer to the jetpack of artificial intelligence, the one that was supposed to arrive in a decade or two, the elusive strong AI. I don’t claim to be an expert on artificial intelligence, and I’m sure people more knowledgeable about this will correct me on this. There have surely been advances in AI since 1970, but it seems to me that those advances have been in one direction, while SHRDLU is a step in a different direction. We haven’t been able to go much further in that direction, but I think SHRDLU’s general direction is the road we must ultimately go down if we’re going to have strong AI. Right now, it seems like we won’t have human-level artificial intelligence until we simulate a human brain completely, which seems like an admission that we just can’t figure out intelligence—that nature has beaten us.

SHRDLU has an ontology, i.e., a model of the things there are in its world and their relations. It has a memory about its past actions, and the ability to reason about its world. It can learn things like the fact that you can’t stack pyramids by trying and seeing that it doesn’t work. And it understands enough natural language to isolate some meaning from it and connect it to this model of the world. Today, our best machine translators are still based on statistics rather than any sort of semantic model—they don’t even try to represent the meaning of the words they’re translating. Chess computers are all about tapping into precalculated databases and brute-forcing through every possible outcome. It seems like the default way of solving an artificial intelligence problem is to find a way to make the problem dumber. And some problems simply can’t be made dumb, so this approach won’t work.

I was reminded of the sorry state of AI when reading a rant on fake freedom in games at the Rock, Paper, Shotgun blog. Jim Rossignol starts off with his annoyance at the way some linear games present the illusion of an open world, only to foil the player every time he tries to go off the beaten path. Want to go over there? Sorry, invisible wall! Went out of bounds? Surely you didn’t mean to, so we’ll teleport you right back where you started. The post evolves into a lamentation over the lack of true simulation in games:

The second thing that makes me and others dissatisfied with games like Rage (and other open but unfree worlds) is that we understand that games can actually be simulations. Games could, if game designers so chose, do more than create the illusion of our presence in a world, they could actually create systems in which our actions caused ripples, rather than triggering responses. We know complicated NPC responses are possible because we’ve seen games like Ultima and the Gothic games on one hand, and we know entities in the world can get on with their own shit because we’ve seen games like Arma 2 and Stalker on the other. We’re aware that the systems can be more complex than a series of pop-up targets draped across a linear story. Because we can see this, games that don’t even attempt to offer the “life” of simulation, never quite meet with our understanding of their potential.

I am neither an expert on AI nor a hardcore gamer, but this is a sentiment I seem to share with many who are either or both. Today, game worlds are not made out of substances, they are made out of infinitely thin textures draped across bodies made from abstract polygons. I understand that this is how graphics work, but games could do a much better job of hiding it. I don’t want to feel like the world is hollow and all surface, even if the technical implementation is. If you give me explosives, let me blow up things that don’t have blow-up cutscenes. If a game gives me a shovel, it’s usually because the game designers want me to dig in a very specific place. In real life, if someone gives me a shovel I can dig anywhere I want, or I can run into the house next door and slam the shovel into the precious wooden floors, splinters flying everywhere. I don’t live in a 3D painting, and when a game gives that impression, it shatters immersion. There are certainly issues with storytelling in open, fully-interactive worlds, but who says making great art—and video games are art—should or could be easy?

Not only are game worlds not interactive, they’re dead. Usually, nothing happens outside a very small radius centered around the player. This is as true in FPSes and platformers as in many RPGs and strategy games. In the comments at the RPS blog, there are many examples of games that have tried to make a living world, which is fascinating, but many of them aren’t recent, and very few come from the major game companies with the resources that could most easily pull it off. It’s like modern game studios have given up on all ambition outside of graphics. Occasionally, you’ll get an attempt at an epic story, but only in the sense that a novel may be epic: mostly linear, predestined to come to the same conclusion. What about simulation?

With every generation of hardware, brute force approaches become more viable. Even if we don’t invent any ingenious ways of making intelligent problems dumb, or making intelligence out of dumb parts, we have so much more power than we had in the seventies, we can still make much more impressive simulations. You could run a thousand SHRDLUs on your phone! Imagine what a major game studio could do with simulation if they had to fully use current-generation hardware, but could only use 1990s graphics. Imagine what they could do with all that leftover power, if they chose.

I’m not a game designer either. These are just the ramblings of some guy on the internet who feels entitled to the jetpacks promised to him by the science fiction of decades past. But imagine: drop me into a tiny the Village-style village. Surround the village with an infinite, procedurally generated forest populated by monsters that will devour you if you stray too far (no invisible walls needed). There are only a hundred villagers, but every one of them has some kind of relationship with everyone else, everyone has their own agenda, and every relation is updated almost in real time. Only that is 10,000 AI agents every tick. Continue by simulating the every animal and plant in the vicinity, the growing of crops and weathering of houses. I don’t know what the story is, or the gameplay, but I’m sure the world would be more alive, and thus more engaging, than most anything else out there.

Contrast, say, Elder Scrolls: Skyrim, which will probably, like its ancestors, contain a huge playable area with thousands of NPCs. Nothing will happen outside the frame, and none of the NPCs will do anything that the creators of the game didn’t plan beforehand. The world will be a pantomime centered around the player, like in almost every other game, even games like The Sims that explicitly try to simulate the world.

All I’m saying is, I want my goddamn jetpack.