It has been more than 7 years since the New York Times published its landmark multimedia story Snow Fall: Avalanche at Tunnel Creek and we’re still talking about it in our Journalism 341: Multimedia Storytelling Design class. We’re stressing to the students that understanding the structure of the compositional elements in a multimedia story is as important as knowing how to code the website for the story. ย
In past years of this class, our students have posed thoughtful questions. Figuring out the story comes before any coding. Composing the flow, or the pathways, of the story starts before designing any graphics.
A few questions to get started: What elements make a composition in multimedia? How do you arrange those elements into a story? In what ways do different elements enhance or distract from the storytelling?
The Snow Fall team at the New York Times included Jacky Myint. In a TED talk, she described the importance of finding ways in which multimedia and interaction can allow stories to unfold naturally.
Identifying a vocabulary that a team uses in talking about the story elements aids in getting and keeping everyone on the same page, so to speak. Myint uses the term full-browser-width ambient videos. The important word there is ambient, which recalls the element of ambient sound that is so common in documentaries in evoking a sensation of immersion.
Myint reminds us that we are creating an experience. We use the term UX, user experience, but often the emphasis is not on experience or even the user. The focus too easily goes to the shape, color, or placement of a minuscule widget. Certainly, we do know that those details and that attention to those pixel-perfect graphics are factors that distinguish smooth interactions from chaotic episodes of frustration. Yet, we always need to bring ourselves (our designing selves, our storytelling selves) back to what we envisioned for our users.
Any search of the Web for UX design is going to turn up dozens of articles and sites about branding, marketing, and product design. Branding is storytelling. What about UX design for storytelling that is not a product? Or, rather, what about UX design when the product is the story itself?
Digital storytelling in the form of journalism, documentaries, or book-life form all presents the story itself as the product. In this way, the users are readers. For this type of material: is the story about the story or about the reader?
How does design change when shifting the focus from a user contemplating the purchase of a product to a reader contemplating the lessons of a story?
Must your story conform to an existing visual identity? Is it part of a larger journalistic site where a corporate style guide governs many elements? Does the story have to be contained within a brand? Or is it a standalone piece? Even if part of a larger entity, do you have the leeway to craft the design and style in a separate way?
Digital media is defined by software and the capabilities of computer code when played through a hardware device. At the core as we experience it today (in 2019), digital media is software that is presented through a pane of glass. What do readers want in this medium? Do they want a book, a magazine article, an essay? Words? Thousands of words telling a story? Words and images? Thousands of words and a dozen images? A few videos tossed in? Or are they seeking another form of representation that holds their attention?
Filmmaker Andrew Stanton mentions that every good story meets certain primal aspects of human needs, such as creating a sense of caring, presenting a promise, and invoking a sense of wonder.
How does multimedia give form to those needs? What does the reader gain from finishing your work? Or, even just sticking with it for a while? How do you keep the reader from giving up in frustration or just out of boredom?
Is the path through the story a cohesive narrative? Is it a logical navigational structure? Does the story create a sense of engagement, a narrative flow either through the menu or visuals (including words) of the story. Yes, words are visuals. (Obviously, the visually challenged experience stories in an entirely different ways that need an alternate mode of storytelling.)
You have all these parts of a story that you have to shape into a form that flows naturally for the reader. You do so through a process of prototyping and iterating design after design. In writing composition class, we call that revising and revising.
There’s not just the story itself. Consider the screen, that pane of glass which actually sits between us and the story. The screen is the intermediary. The screen mediates our understanding of the story. Designer Frank Chimero wrote about what screens want, an essay worth rereading every year.
Multimedia storytelling is about understanding the possibilities of the screen. In designing a multimedia story, you are forced to consider the constraints (resources, time, bandwidth, user attention, assets). Are there possibilities for bespoke imagery that sets your work apart? What role can maps, diagrams, animation, and actual puzzles play in your multimedia story?
The story is never just the story.