Divergence

To figure out how to tell stories in an interactive fashion, I felt it essential to work with the stories themselves. This turns out to be a two-way street: the interactive features should both support and enhance the story. The story dictates what interactive features are needed, but the interactive features affect how the story should be told, and further what kind of stories fit the features.

The process, in short, was a long, divergent phase full of enthusiastic sketching, story writing and basic prototyping, which resulted in a ring binder full of sketches, exploring interactive storytelling and story development, looking at locations, characters and interactive sequences.

This ended with the transition to a convergent phase, trying to make sense of my ballooning sketch binders (fig. 4.1). Thankfully, some patterns emerged, resulting in some “dos and don’ts” and eventually the foundations of a framework for interactive storybooks.

Multiplayer

Interactive shared reading is inherently “multiplayer”, with two participants engaging in a common activity. Multiplayer games can be local or online, turn-based or real-time.

I went on a short tangent looking at the possibilities for online multiplayer, where parents could engage in remote shared reading with their children. The prototype devices ended up talking to each other via an online server anyway, so it would “only“ require some means of communicating via text, audio and/or video. In the end, I felt that it led to an entirely different project; I would design for local multiplayer, as it felt more relevant for conventional dyadic reading.

For a local, turn-based game, there is only need for one shared set of controls and one shared screen. With actions happening in real-time, there might be a need for a second controller and, in some cases, a second display.

Since the first computer game¹, “Tennis for Two” (1958), through “Pong” (1972) and a heap of arcade machines, from the first Nintendo (1983) to the latest Playstation (PS4 Pro, 2016): the convention has been separate controllers for each player. In the case of the iPad, I felt it would be fair to assume that households that own an iPad (tablet) also own at least one iPhone (smartphone). The iPhone could serve as a separate controller and an extra display, opening up a lot of possibilities for both gameplay and storytelling. It also allows one player to keep secrets from or surprise the other player, a very interesting mechanic for social play.

The way the devices are held “has implications for how easy the device is to share” and can “influence the closeness of the interaction”², so I found it interesting to see what sort of impact a secondary device could have. Would it only lead to more individual screen-staring, or could it be used to coax the participants into close interactions?

I started sketching out some scenes and scenarios for two players, with screen layouts for tablet and mobile. I tried to represent as many play types as I could, and explored themes like negotiation, cooperation, competition and dialogue. Some sketches focused on the secondary device as a remote control, others tried using the mobile device as a synchronous “viewport” for storytelling, showing the current scene from a different perspective.

Some examples include:

• A second iteration of the origin idea, where the one player was stuck in a completely alien environment and tasked with finding the “crombobulator”. When the player clicked on various items, the other player took on the role of expert, getting access to detailed explanations of what was clicked. The story would progress once the “crombobulator” was discovered.
• A scenario in which one player has to defuse a bomb and the other has access to a cryptic and randomized bomb defusing manual (fig. 4.2, left), forcing the two to cooperate (or sabotage each other). A timer would add urgency.
• Negotiation between the two players (fig. 4.2, middle). One player takes the role of a guardian blocking the path leading to the next page, the other plays the protagonist who has to convince the guardian to let her pass. The protagonist player offer items to the guardian player, who decides if the offer is met with anger, spite or acceptance.
• Ways to tell the story from different perspectives simultaneously (fig. 4.2, right). Though interesting-looking, it could easily make the story hard to follow. It was also hard to make it interactive, instead making both players passive spectators.

These led to more involved storyboards of action sequences, with multiple outcomes depending on player input.

For instance, one storyboard (fig. 4.3) depicts a sequence where one of the players are given a description of an apartment, and the other pilots a drone to peek into people’s windows. If they think they’ve found the right apartment, they can crash the drone through the window to enter. If it’s the right apartment, the story continues. If not, someone gets very angry.

Proof-of-concept apps

I made some very early tech prototypes to see whether or not it was plausible to get the two devices to talk to each other. The first working prototype was “platform-agnostic”(i.e. my iPad communicated with an Android phone), written in JavaScript and packaged for the devices with Apache Cordova. The apps were synchronized via a local server, sending instructions back and forth using node.js and websockets.

This soon out turned to be a slow workflow with bad app performance, so after some tests I swapped my Android phone with an iPhone, focused on iOS development and learned how to program apps with Swift and Xcode. I then made some unsuccessful attempts at getting the devices to communicate without my node.js server, based on proximity and via Bluetooth.

In the end, I had tested a dozen different approaches and found the most effective way of prototyping with “real code”, and proven the multiplayer idea could work.

Interactive storytelling

Each scenario sketch was a snapshot that could be a single page in a storybook, and I started thinking of ways to string them together. Many scenarios involved the players making decisions, which would only make sense if those decisions were reflected in the story. That, in turn, meant the story couldn’t be the same every time.

I looked at story archetypes (“The Hero’s Journey”, "Seven Basic Plots”) and tried to apply them to look at non-linear and branching storytelling. Though no surprise, what I discovered was the sheer amount of planning and writing needed to drive these kinds of plots successfully. Respecting a standard story structure is easy when the story is linear, but gets difficult fast when the introduction sets you up for different conflicts, where the protagonist meets different characters for each one, and you suddenly try to time a simultaneous climax for a dozen separate storylines branching out to even more endings.

You don’t need many branches before a story gets completely out of control. Add illustrations on top of this, and you’ve got yourself a nightmare. After various attempts, I ended up close to adventure games again – quite linear stories that take interesting detours. Instead of big branches, smaller variations based on player choices seem more viable.

There’s also the question of how close to stick to a book and its pages. In a book, you can always turn back a page, but once you introduce player choices, the choices should have some weight and mean something. If you can undo all choices by turning back a page, you render them meaningless. It doesn’t matter for the plot, but it has a large impact on how it feels to read and play.

Locations and characters

Again inspired by the tradition of adventure games, I wanted the story to include a lot of traveling between locations. If the story is told in a non-linear fashion and you still want to retain some control of it, you need to limit where the players can go. The easiest way to do that is placing the story in small, naturally “walled-off” areas (see also: Agatha Christie).

I also explored how the players could leave an impact on the locations they had visited, and how to design a location where a branching story was represented by literal crossroads and branching paths. This led to a small, and very steep, city location, where the players started at the bottom and progressed upwards. Each location they stopped at would have some sort of interaction with the environment or another character, and each of these interactions would leave a visible mark in the environment. For instance, the players could end up starting a fire in a bakery, or completely flood the lower levels of the city. When the story concluded, the camera would zoom out to reveal the visual traces of all the players’ decisions. It could also serve as a nice bookmark if you returned to the book halfway through the story, as the trail would mark how far you’d progressed.

The scenario sketches also hinted at the players taking on various roles, so I looked into character archetypes (fig. 4.6) and classic pairings (“mentor and apprentice”, “good cop, bad cop” etc.) to come up with ideas for role-play and dramatic interactions between the players.

It also led to a host of ideas about character creation and customization, with the possibility of the players feeling closer relationships with the characters they experience the story through. Again, it’s a balance of writing a story beforehand, and how big blanks the readers should fill in on their own.

I tried explaining the use of the secondary device by having characters interact with the player’s phone, for instance via simulated calls or text messages. I even played with housing a character in the phone through supernatural means (fig. 4.8).

Character personalities were simulated through their phone interfaces. “Geeky” characters might have some very odd or advanced apps on their phones, while a social, friendly character might have a lot of apps for sharing pictures or sending messages. If the secondary device played the part of the characters’ phones, their apps could even be used for storytelling, through reading their mails and messages, or peeking at their selfies.

Pre-written characters can be quite complex, leading me down a rabbit-hole of backstories and relationships. In general, storytelling had become a daunting and time-intensive task.

Story development

This is where the explorative nature of the project almost took it off the rails and into a vicious circle. The various attempts at stories and setups would affect the form of the product and its interactive elements. A certain story sequence would be a good fit for a certain minigame. When developing these elements, the story would need adjustments to accommodate the interactions. Maybe a location would change, but then the story might not make as much sense, so it would be rewritten again, which might affect the minigame, etc.

For instance, let’s say a group of characters are on a hiking trip and camping in tents. One character loses something important, a MacGuffin, that needs to be recovered for the plot to progress. Finding the MacGuffin is then played out via a minigame, that consists of clicking hotspots in an interactive environment. One of the tents would be a natural place to look, but tents aren’t particularly interesting environments, so maybe they should be camping (or glamping) in an RV instead. That opens up a lot of possibilities for story locations, too.

Now the minigame starts to get interesting, but the characters are no longer hiking – they are on a roadtrip. They are all traveling in the same RV – what if one of the characters stole the MacGuffin, and has hidden it somewhere? That creates an interesting conflict, the minigame can be expanded to include both players – one to hide the MacGuffin before the other must find it. But who are these characters..?

And so the process went, ad nauseam. Normally, this would just be a normal creative process, but a diploma project has a pretty short timeline. It’s a ton of fun to work this way, but I got caught up in storytelling and lost sight of two very important points:

1. The goal of this project isn’t writing a compelling story, it’s the parent-child interactions.
2. I would never have time to prototype an entire story anyway.

Though not a shining example of project management, the process was by no means a waste of time – working with the story this way did generate a lot of ideas and discoveries that might have been hard to come by in a less slapdash manner. It also underlined the idea of interactive sequences both driving the story and being driven by it.

In retrospect, when looking at the product as a storytelling framework, I think it was essential to have explored “real” storylines to get a feel for what sort of scenarios the framework might have to support.

Patterns emerge

All the sketches, ideas and storyboards could either be described as purely story-related, or categorized by play type using Hughes’ taxonomy. Many of the play-related ideas fit under role-play, while others didn’t. Thus all my ideas could be presented as:

• Story ideas, for reading
• Role-play ideas, for acting
• Game ideas, for playing

The principle that games should not interfere with the story made it necessary to separate the three somehow, especially if any product were to represent and contain all three categories. This meant everything from the storytelling category should be kept on its own, interspersed with interactions relating to the other two. The game ideas were only loosely connected to storytelling, and had the most potential to be distracting, so maybe they should be kept completely separate too.

This led to idea of a storybook app with three “modes”: Read mode, Talk mode and Play mode. Each page could be one of the three modes, depending on the what kind of interaction suited the plot. To accommodate any kind of story, I went general: I started thinking less about one, definitive product and more about a sort of framework that could be used to build storybooks.