I’ve been thinking a bit lately about storytelling with smart homes and the Internet of Things – what the MIT Media Lab‘s David Rose calls enchanted objects. My Lighthouse in the Woods experiment at the Annenberg Innovation Lab was a first step in this direction, using the Oculus Rift to experiment in storytelling using smart picture frames in a living room:

The Lighthouse in the Woods is an immersive ghost story. Donning an Oculus Rift, the user finds herself locked in a windowless study as a disembodied voice tells how, one by one, the narrator’s family fell victim to a mysterious curse. Portraits on the walls illuminate as each family member is introduced, dim as each one falls, and re-illuminate as they come back. The audience member can move around the room, but cannot leave it, and has no control over how the story progresses – replicating the narrator’s experience of impending doom and helplessness.

The Lighthouse in the Woods leverages the Rift’s unique sense of 360-degree immersion, but by removing nearly all of the audience’s agency, it subverts the assumption that the Rift is a gaming device and delivers a storytelling experience more akin to film, TV or theater. This project also explores how storytelling could evolve for a connected home. The Lighthouse in the Woods uses portraits to tell its story because connected digital picture frames are one possible additional “screen” to use in a living room.

It was a fun experiment, and it gave me an opportunity to work with some old friends (huge props to Talon Beeson, Jesica Avellone, Sara Anderson, and Andy Rozsa) and some new ones (similarly huge props to USC Games’ student Austin Drexler), but it barely scratched the surface of the topic.

Since Black Friday is this week, I’m seeing a ludicrous amount of ads for sales on smart home tech, which is probably geek dad bait of the highest order. Since we only rent our place, this particular geek dad is limited in how much I can really drill into this subject (har har), but if we did own this joint I’d first be tempted to install some of the more utilitarian stuff: a Nest thermostat and smoke detector set, for example, or some simple equipment to tell me if the mail’s arrived or if the garage door is closed. (Rose’s excellent chart aside, I suspect there’s a pretty clear Maslow’s hierarchy of needs for the Internet of Things.) However, once that was sorted, I’d definitely want more fun stuff in the house. I keep thinking about the possibilities of the house as a storytelling and entertainment platform, which leads to more thinking about how a home already functions as a storytelling engine.

Think about it:

• Framed pictures are stories. Photographs are captured memories. Paintings are captured visual stories.
• Kitchens and pantries are full of stories. Many of the recipes we cook are handed down to us over generations. The leftovers in our refrigerators are memories of places we’ve been. Many of the packages of our food tells the stories of where that food came from (true or not, that’s a different matter).
• Cars accrue memories. Not only do beautiful vintage cars have stories of prior owners, but all cars have embedded stories about how they were designed and where they were made. Dings on their bumpers tell stories of fender benders. (In my case, somewhere out there is a car with a story about backing into a piano.) Family cars could tell brilliant stories about family vacations, especially cross-country drives.
• Toys tell stories. Of course they do.
• Clothes tell stories. Much like food, many clothes have stories attached to their brands, although the actual stories of where they were made may be much less romantic. Still, shirts can be souvenirs of concerts or other locations; jewelry commemorates occasions; unique items can represent the skill and story of an artisan.
• Furniture can literally tell stories. I’ve sketched up a device I call the Storyteller’s Clock, which would rattle off a story for every day of the year…
• Lighting can tell stories. Or, at the very least, it can set the stage for other stories to emerge (hello, mood lighting).
• Architecture can tell stories. Every home has a story about who built it, when and why. The manner in which houses are designed and built reflect certain cultures or periods of time. The more artistic ones can themselves evoke particular moods or create their own little worlds (think homes that look like castles). Homes converted from other buildings, like warehouses, barns, windmills, and so on, are full of stories from their previous lives.

When you look at a home through a storyteller’s lens, you begin to realize that homes (or at least the most interesting ones) have stories everywhere. There are a couple things, though, of particular interest.

One, many of these stories are a balance of push and pull – the pictures and souvenirs in a house, for example, may be completely silent and ignored until you invoke stories when you choose to look at them – for example, many of the pins shoved into the bulletin board over my desk are souvenirs from our honeymoon at Walt Disney World, but a few are souvenirs from the opening of OSU’s Billy Ireland Cartoon Library and Museum last year, another’s from the 2010 Game Developer’s Conference, two more are from Artist’s Alley at the San Diego Comic-Con, two more are from a trip to San Francisco this year, a few are collector’s pins from the Broodhollow Kickstarter last year, one’s from a Cirque du Soleil show, one’s a rare Walt Disney Imagineering Blue Sky Cellar pin my wife got me, and another’s a rare Imagineering pin I was given as a thank-you for doing a little work for Imagineering last year. However, these items need me as a translator. When this office serves as a guest room, few people would know what the stories behind these pins are, unless I (or, in some cases, my wife Laura) were there to tell those tales. I keep thinking of the scene near the end of Return to Oz, in which Dorothy has to go through the Gnome King’s ornament room, lay her hand on an object, and say “Oz” – if the object is actually one of her friends, transformed by the villainous Gnome King into a true enchanted object, then the friend would be changed back. What if a visitor in this guest room could lay a finger on one of the pins on my bulletin board, say the equivalent of “Oz”, and hear – or somehow even see – the story behind it?

Two, I think the kitchen is actually a great place to tell stories with a smart home. Something like SARAH, the Self Actuated Residential Automated Habitat from Eureka, could sort of cook, or at the very least do an ingredient check in the pantry. We’re starting to see steps in that direction with things like Microsoft’s Kinect or Amazon’s Echo, which will clearly hook into things like Apple’s Siri or Microsoft’s Cortana really quickly. Asking a smart house to cook a meal is beyond us for the time being, but we could – and can – ask it to start the coffeemaker. However, hooking up a little screen with speakers in the kitchen that could, upon scanning a barcode or detecting some other NFC widget, play a little video of your grandmother saying where a recipe came from, or some celebrity chef explaining where the inspiration for this particular TV dinner came from, would be low-hanging fruit. There’s even a possibility of making the cooking itself into a narrative experience, either as a kind of extended metaphor a la the whole “rabbit runs around the hole” thing when you’re teaching a kid how to tie her shoes, or as a play-by-play by a really great storytelling chef like Julia Child, Anthony Bourdain or Jamie Oliver, explaining how they learned to cook this recipe.

Three, and again this gets back to my Lighthouse in the Woods prototype, I think there’s a lot of room for storytelling experiences in a smart house without screens. Imagine playing a game of hide-and-seek with a virtual cartoon mouse that scampers throughout the house via embedded speakers, “rustlers” akin to the vibrate function on a phone, or tiny projectors. Imagine a ghost story told in a similar fashion, or a romance, or a drama. Imagine an installation-style murder mystery, or something like Sleep No More or Then She Fell, the installation stories from Punchdrunk, only with the actors projected into the room? Could you tell a similar story with disembodied voices coming from location-aware speakers? What happens when you layer AR into the mix?

Four, what happens when you add performative objects, like the next generation of Teddy Ruxpin dolls, as “actors” in the experience? We’ve done a little experimenting with this at the Annenberg Innovation Lab already (more on this in a bit, once it’s fully unveiled) but imagine a whole network of smart toys that are aware of each other and react to each other, like Skylanders without a TV, built on a platform similar to the one I hypothesized over ten years ago? Imagine the Beauty and the Beast style experience you could create.

Five, what voice would you want your house to have? Getting back to the Siri and Cortana thing for a moment, would we still want our house to be an AI of sorts if we’re already carrying an AI around in our pockets? Would we want our house’s AI to be a slightly stuffy librarian, and our phone’s AI to be a wisecracking raven? Perhaps we would want multiple AIs haunting our houses, so we have a chef AI character installed in the kitchen to help with the cooking, a mechanic AI character in the garage to ask for help with the car, and a fashion designer AI in the closet to make suggestions on what to wear? What stories would they all tell? How does the modularity of separate AI characters impact how we think of them – would we feel shortchanged if we had to pony up for a separate AI for each function instead of simply buying one master house AI like J.A.R.V.I.S, or would we feel weirder if we knew our JARVIS was incomplete because we hadn’t bought the “mechanic” DLC pack to make him more useful in the garage? What if those DLC packs were themselves stories for our J.A.R.V.I.S. to learn?

Anyway, this is the kind of thing I’ve been thinking about lately. How would we Imagineer our everyday homes? What would that experience platform be like? Our houses are already full of stories, but what would you really want your house to do if you were to say aloud, “House, tell me a story”?

I just tried it. Surprisingly, I’m a little disappointed it didn’t respond. Maybe that in itself is telling me something.