Research II: State of the Art

Looking at interactive children’s media, the products range from books to games – from game-like paper books to digital games inspired by children’s picture books – with a whole spectrum of categories in between.

Printed books are alive and well, with no decline in sales over the last 20 years¹. According to one author I spoke to², a normal first edition is printed in 2500 copies, with expected sales around 200 books a year.

It’s hard to gauge how well children’s apps sell based on the app store charts, but throughout this project, they have taken up 80–90% of the “paid app” bestseller lists. (Poio, a constant chart-topper, has reportedly “helped 30.000 kids learn how to read”³, but since their software is used in schools, that doesn’t translate to sales figures.) Google’s Play Store distinguishes between sales numbers and highest grossing apps, but Apple’s algorithms are unknown.

The charts are localized, so we can make one of two assumptions about the Norwegian market:

1. We spend a lot of money on children’s apps.
2. We buy very few apps, but when we do, they are for our children.
   I suspect the latter is more plausible. (As a sidenote, I saw an obscure and quite expensive iPad app shoot right up into the top 25 “best-selling” paid apps only hours after I had purchased it.)

Paper books

Though their themes and topics are as diverse as adult books, traditional children’s books are normally categorized by age, ranging from picture books to “middle-grade” books. (I do not include “young adult” (ages 12–18) literature in this study.)

3–5 years, Picture books

Books teaching fundamental language and concepts (for instance: vocabulary, numbers, emotions, civic services) through big letters and colorful pictures. Often “interactive” features using paper in novel ways, like the holes eaten by “The Very Hungy Caterpillar” (Eric Carle, 1969) or the paper flaps in Eric Hill’s “Spot” series.

6–8 years, Storybooks

More focus on text than on pictures, though most often illustrated. Classic books by Roald Dahl & Quentin Blake (“The BFG”, “Matilda”), Astrid Lindgren ("Pippi Longstocking") and Ole Lund Kirkegaard ("Gummi-Tarzan").

9–12, “Middle grade” books

Novels, pretty much, where pictures are mostly reserved for their covers. Lots of fantasy and comedy, often serialized. The fact books for this age group are more advanced and diverse, both related to school and children’s interests. Current best-selling series about “Captain Underpants” (Dav Pilkey), “Diary of a Wimpy Kid” (Jeff Kinney), “Harry Potter” (J. K. Rowling).

E-Books and “enhanced“ digital storybooks

In its simplest form, an e-book is the screen version of its print sibling, with static text and illustrations. Cahill & McGill-Franzen (2013) mention four minimum criteria for classifying an e-book:

1. Text presented visually on a screen
2. Book-like configuration (table of contents, pages, etc.)
3. An organizing subject matter or topic
4. Multimedia enhancements

Several Norwegian publishers have their own “bookshelf” apps for purchasing and viewing their children’s books. (You’d think the publishers could work together on this one, but I’ve yet to see a bookshelf-app that covers several publishers’ children’s books – it seems troublesome to have a large library of digital children’s books when you have to remember on which publisher’s bookshelf to find a given book.)

“Enhanced” storybooks make more use of their interactive platform, and usually contain animated elements, interactive “hotspots”, music and sound effects, and voice-acted narration. These seem primarily made for solo use, where the children can explore the books on their own⁴.

Audiovisual and interactive features can even be effective stand-ins for parental “scaffolding”:

As motion and zooming may direct children’s attention to a detail of the illustration in a similar way as an adult pointing at the detail and providing comments or explanations, multimedia may be just as beneficial in supporting story and language comprehension as interaction with an adult explaining the meanings of the story and sophisticated words in the narration.

Adaptations of the classics (Dr. Seuss’ books, for instance) are well represented on the iPad, but stories specifically written for the format still seem to be rare.

Educational apps and games

Apps with a focus on teaching: often ABCs and math, but also about topics like space and dinosaurs. The iPad has become a common teaching tool in Norwegian primary education, and there’s a whole ecosystem of software designed for classroom use and as support for school textbooks, not always found on the app stores. Some of these apps stay close to a book-like presentation, while others are closer to games.

“Poio” (fig. 3.3, top) is part game and part storybook with a unique approach to teaching 4–9-year-olds how to read. The children control a character that collects alphabet letters, and can then spell words with these letters and use those words in a story. Getting rave reviews, Poio has topped the paid app charts⁵ for the entirety of this project, even at a quite high price point (kr 199,-). It’s compatible with the curriculum for Norwegian first graders and used as a teaching tool in schools⁶.

[Update, 2019: Poio is now on both iOS and Android and has versions for Swedish and English readers.]

“Dragonbox” (fig. 3.1, bottom) is a series of mathematics games for children of all ages that teach basic and advanced maths in fun ways. Developed by WeWantToKnow AS in collaboration with the publisher Aschehoug, it comes in a school edition where the app is bundled with textbooks and manipulatives (i.e. building blocks⁷).

Pretend play

Games that have more in common with toys like LEGO or Barbie than books. These “sandbox” games have no set goals and leave the storytelling up to the player via interactive play sets. These are entertainment products, but valuable for social interaction and fun to play with others. (Not “multiplayer”, but playing together with/on the same device.) Some apps have features that let the players record and share their stories.

This is the best-selling category on the paid app charts, with series like “Toca Life” (fig. 3.2), “Sago Mini” and “Dr. Panda” filling several of the top spots⁸.

Computer/video games

I’ve chosen not to examine the whole field of video games in detail, as it’s huge on its own – broadly speaking, every game with a story can be said to be an interactive narrative. Still, there are some genres to take note of (and inspiration from), mainly stemming from the adventure games of the 90s.

Some adventure games back then tried to play with live-action video (“Under a Killing Moon”, “Voyeur”, “Phantasmagoria”), earning the moniker “interactive movies”. With adventure games enjoying somewhat of a renaissance lately, with games exploring novel takes on interactive storytelling, we have come full circle back to movie-like games like the “Uncharted” series and this year’s “A Way Out”. They exist alongside games that are visually removed from Hollywood blockbusters, but first and foremost tell a story.

Adventure games

Adventure games (Fig. 3.3, 1) had their heyday in the 90s and early 2000s, with titles by Lucasfilm/Lucasarts (“Monkey Island” series, “Grim Fandango”) and Sierra On-Line (“King’s Quest” and “Leisure Suit Larry” series) now regarded as classics. The genre has had somewhat of a revival in later years, with a lot of new games and “remasters” of old ones.

These “point-and-click” adventure games tell semi-linear stories and requires the player to solve puzzles and problems. Progression is typically made through exploring environments, talking to NPCs and collecting items in an inventory. Inventory items can then be used with environments and NPCs to solve puzzles.

I enjoy games in which the pace is slow and the reward is for thinking and figuring, rather than quick reflexes. The element that brings adventure games to life for me is the stories around which they are woven. When done right, it is a form of storytelling that can be engrossing in a way that only interaction can bring. [...] If any type of game is going to bridge the gap between games and storytelling, it is most likely going to be adventure games. They will become less puzzle solving and more story telling, it is the blueprint the future will be made from.

“Stories you play”

A collective term for games that focus solely on telling a story, with various degrees of interactivity. Usually quite linear stories with little problem solving, the gameplay is closer to reading a book or watching a movie.

I have nothing against stories you play, though they’re currently shelved with video games only because we privilege unreliable terms like “interactivity.” [...] in all cases they involved me watching some things happen, clicking a button, then watching some more things happen. [...] That’s put the burden of invention on audiovisual novelty.

These are nothing new – laserdisc games like “Dragon’s Lair” were popular in the 80s, and CD-ROM brought on a wave of “Full Motion Video”-games (“Voyeur”, “Phantasmagoria”) – but there’s been a lot of interesting releases over the last few years. Advances in graphics might have made “interactive movies” viable in new ways (see: “Uncharted” series, “A Way Out”), but it’s could also be seen as a sign of video games maturing as a medium. Not everything is about action, horror and adventures anymore: “Kentucky Route Zero” (Fig. 3.3, 2) is a surreal and poetic story about loss and death, “Firewatch” is a romantic thriller and “Night in the Woods” is a drama comedy about returning to a the dying small town you grew up in.

More to the point: just like “Robinson Crusoe”-like adventure novels a became genre popular with children, these story-focused games are starting to become more children and family friendly. Brain&Brain’s “Burly Men at Sea” (Fig. 3.3, 4) from 2016 is a “story you play” presented like a children’s book. It was very well received, with some reviewers even bringing up shared reading:

It’s a gentle seafaring tale I’m looking forward to playing through with a child when I next see my smaller family members but which I’m more than happy to play for my own enjoyment as well. I think I’m on my sixth distinct playthrough at the moment and still discovering new things.

It’s the dreaded “product you discover during the diploma that you wish you had thought of first”, but it’s at least comforting to see how far you can take interactive children’s books and that there seems to be a big interest in them.

Case studies

I took a closer look at a few enhanced e-books, thinking they were the closest to my own project.

I was surprised (and nostalgic) to find “Arthur’s Teacher Trouble” (Marc Brown, 1986) on the App Store. This is part of Brøderbund’s catalogue of “Living Books”, a popular CD-ROM series from the mid 90s, now ported to the iPad as “Wanderful Interactive Storybooks”. It is literally, pixel by pixel, the exact same digital book from my childhood, with the same voiceover. I had a lot of fun with this book when I was a child, and it was instrumental in teaching me English.

There are two modes, with and without voiceover. Every word on the page can be clicked to hear it out loud, and almost every little element in the background illustrations can be interacted with, triggering sounds and animations. The pictures are pixellated and the voiceover is compressed into a crunch, but the title is more than 20 years old. (I’d say it’s still a charming book, but then again, I’m wearing rose-colored glasses for this one.)

Jakten på Teddy is the only storybook-app I found that I know was specifically written for iPad. It was “Jakten på Teddy” tells the story of Theo, whose teddy bear gets stolen by the evil witch Ipswitch. Theo gets transported into a magical realm where all his toys have come to life, and has to search for Ipswitch to rescue Teddy.

Theo travels through three locations before confronting the Ipswitch in her castle. In each location, he meets a living version of one of his toys, all broken in different ways by the witch. The locations all have interactive hotspots that reveal short animations. Theo and the toys have a short talk, the reader plays a little minigame to mend the broken toy, and it’s off to the next location. When confronting the witch, the toys you have mended come to your help in defeating her.

“Kubbe lager skyggeteater” is the least “enhanced” of the case study e-books. It tells the story Kubbe, a piece of wood, who is spending a day with his grandmother in the forest. Grandma tells Kubbe about the art of shadow theatre, inspiring him to make his own play.

“Kubbe...” has beautiful illustrations and excellent voice acting, even a long animation about shadow theatre, but interactive hotspots are few and far between. There’s very little to do but listen to the story and turn the pages.

I’ve included Telltale Games’ “Puzzle Agent 1 & 2” (Fig. 3.3, 3) because it’s presented like a children’s book and has a lot of minigames. It tells linear, humorous mystery stories interspersed with brain-teaser puzzles, with full voice-acting and illustrations by Graham "Grickle" Annable. The player can travel between locales and talk to NPCs, but progression is made by solving puzzles with increasing difficulty. The minigames are well suited to touch controls, and both Puzzle Agent games have been ported to the iPad.

Findings

These enhanced e-books are well executed products, but not taking full advantage of the all the opportunities of interactive storytelling.

Dynamic stories

In the case of “Jakten på Teddy” the story branches, but only slightly. The app is inspired by “Choose Your Own Adventure”-books and its marketing really plays up how the story is different each time you play, but the truth is that there is very little variation between each playthrough. There are six different locations to visit, and four different orders in which to complete the story, but the climax is always the same. Player choices have very little impact, it doesn’t matter what choices you’ve made or which toys come to your help in the end, the ending plays out exactly the same.

“Arthur...” has a ton of interactive hotspots, but the story is the same every time. “Kubbe...”, though a charming e-book, might as well have been a movie. “Puzzle Agent” has a linear story, but progresses like an adventure game, making it feellike it’s based on player actions.

If the story is completely linear, it’s a good idea to break it up with events that require player input (beyond turning a page) to progress. Branching stories must be linear enough to write, but to offer any real replay value to anyone but the youngest children, they need to be dynamic enough to offer up some real variations based on player input.

Interactive elements

In “Jakten...”, the interactive hotspots are telegraphed with a distracting sparkle animation, yet offer little in return for pressing them. Neither “Kubbe...” nor “Arthur...” gives any special focus to hotspots, but where almost every element on the screen (including words) offer a reward when clicked, almost none of Kubbe’s do. “Kubbe...” makes it worse by having very detailed illustrations, at times with tons of elements that you want to interact with, but very few that do anything.

It might be hard to predict what the user thinks, but everything that looks like an interactive element, should be an interactive element. At the same time, a hotspot should manage to look like one without drawing too much attention to itself, for example using odd-looking borders or animations. Both “Arthur...” and “Puzzle Agent” gets this right.

Interactive elements and minigames should not be active or playable while the story is being read, but become available during the break between pages. This is easy to solve when using a pre-recorded voiceover, but is a tricky problem for “live” shared reading.

Research conclusions

• Children aren’t awed by iPads any more.
• All kinds of play types can be recreated digitally
• There is nothing indicating that iPads are inherently bad for children, children’s books or shared reading, but one needs to be aware of distracting elements and decide on when to teach and when to entertain. Younger children are especially sensitive to distractions.
• Interactive media for children has its uses in the classroom.
• If the app’s interface is too fun to use, it will be used for transgressive play. Keeping close to a paper book format lets the children develop trans-literacy skills.
• The biggest challenges with shared reading on iPads, are children obstructing the parent’s view and children’s perceived “ownership” of the medium.
• Digital children’s books are seeing some interesting developments, but the evolution comes from interactive narratives circling back from games, not “growing up” from e-books. The genre of “stories you play” is starting to come into its own, with a lot of novel takes on interactive storytelling.