Convergence

Sorting through the research and sketches and trying to make sense of it all.


The Framework

The framework app provides the “building blocks” with which to tell a story. It’s like a book with blank pages: a container for the story, where the story itself comes from the applied text and images. In this imagined framework each page can be represented with on of three modes: read mode, talk mode, and play mode.

To facilitate stories with branching plots and player choices with story consequences, the framework needs to react to both real-time user input, and keep track of previous actions. Did Snow White eat the poison apple or not? What item was used to murder Professor Plum, and in which room? From a technical standpoint, this can be solved by storing “flags” – small pairs of information that can be used to store and reference choices made by the readers.

The framework also needs protocols for device connection and communication. This handles how the devices synchronize their data, for instance what page the readers are currently on, and how choices on one device are communicated to the other.

Modes

Read mode

This mode focuses on reading and story progression. One player takes on the reading role, and reads the story on the primary screen (iPad). Illustrations and animations are shared between the devices, but the secondary screen (iPhone) may also contain interactive elements that lets the listener influence what’s being read.

Wireframe sketch of an iPad screen with text on the left half and an illustration on the right.

Fig. 5.1 | Read mode initial sketches. The iPad has a split view for text and illustration.

Talk mode

Children are more engaged when reading than listening, so I wanted to find a way to ensure both participants got to read something. Talk mode is about dialogue and dramatic play. It uses one device for each participant, who are given character roles and dialogue lines to read. While one player reads, the listening player can change their character’s reactions to what’s being said. These reactions are reflected on both screens, and determine the next lines and the final outcome of the conversation. This outcome then has consequences for the rest of story.

Wireframe sketch of a tablet and a smartphone during an interactive conversation between to characters.

Fig. 5.2 | Talk mode initial sketches. Characters talking to each other based on input from the secondary device (right).

In a conversation between two characters, they might end up best friends or angry at each other. Talk mode also opens up possibilities for bartering, negotiation and diplomacy. A character could gain a valuable items, or miss them completely, depending on the chosen dialogue options. One character could be tasked with persuasion and try (and fail) to convince the other into something. The roles could even be without regular language, where one player or character has to communicate via sign language, robot movement or alien noises.

Play mode

The point here is to keep distracting games away from the story, a little time-out “between” pages. The games should be closely connected to the story, for instance letting the players explore locations, develop characters, or just determine what happens next.

Wireframe sketch of a tablet and a smartphone. The left half of the tablet displays text, while the right shows a game. The smartphone shows the same game and is used as a separate controller.

Fig. 5.3 | Play mode sketch, with both devices as controls for the same minigame.

Framework guidelines

Story-driven, and driving the story

Every event, action and reaction should always connect both players to the story. Interactive “hotspots” and choices should be directly related to the story at present or near future.

Requires active participation by both players

Try to encourage social interactions. Make sure both participants are always involved in the same event or activity (whether with or against each other). Even as passive participant, a listener should have something interesting to watch or choices to make.

Non-distracting interactions

Interactions must not distract from the story being told. Engaging illustrations, animations and games should be used carefully, and not when important story details are being read.

An action on one device demands an immediate reaction on the other

It’s important that the participants feel connected, especially when looking at separate screens. Input on one device should always be indicated or mirrored on the other. For instance, when making a choice on the iPhone, that choice should be reflected on the iPad. If the choices are secret, the iPad should indicate that a choice is being made.


Up next:

Final prototype and user testing

Observing and getting feedback from real readers.

Table of contents

  1. About project
  2. Introduction and background
  3. Research I: Children and media
  4. Research II: State of the Art
  5. Divergence
  6. Convergence
  7. Final prototype and user testing
  8. Results / final conclusions