Interactions of the Reading Mind
On the three different levels at which readers are interacting with digital texts — and what this means for the design of digital publishing
The interactivity of the digital medium
The digital mediation of information (be it textual or other kinds of content) consists of two different phases: The first phase is the provision of the information, which is the embedding of the information into a medium (i.e. the thing “in between” the content provider and the user). To succeed in this phase, we as designers must pay careful attention to the interaction between the information and the technological context in which it gets embedded. The interactive technology will have to “understand” the underlying semantic and hierarchical structures of a text in order to provide the best visual presentation of the information to the readers. A lot of what makes 2K/DENMARK pioneers and leaders in the field of digital publishing is our expertise in the complex tasks of making the interaction between the text and the technology as fluent and frictionless as possible.
But what about the next phase in the mediation process; the consumption of the information? Here the interactivity of the digital medium is defined by the interactions between the user (which in the case of texts would be the reader) and the information presented on the screen. This phase is at least as crucial in establishing the successful mediation of information from provider to reader, which is why it is beneficial here to take a closer look at some of the different aspects of the interactions between the text and its reader—the processes of the reading mind.
Windows, Mirrors and Cognitive states
Since the publication of Bolter & Gromala’s Windows and Mirrors back in 2003 it has become common practice to distinguish between two different ways for users to interact with technology: When the technology is acting like a mirror, the user is interacting with the interface, and the interface will respond by reflecting the users actions and perform the given tasks. The goal of this interaction will often be to get to a state of interacting through, where the technology becomes transparent, functioning as an unobtrusive window, through which the user interacts with the given content or with other people using the same technology as a window.
These two kinds of user interaction also entail two different states of cognition for the user. When interacting with the technology by the manipulation of the interface, the brain is in the perceptual cognitive state, where it is attentive and prepared for external perceptual stimuli to which it can react. When interacting through the technology the brain should ideally be in the reflexive cognitive state, where the brain is immersed in internal activity, thinking and processing the information it almost subconsciously obtains through the (preferably) passive and unobtrusive “technological window”.
Jean-Honoré Fragonard La Liseuse (c. 1770)
Three levels of interaction in reading
Applying this thinking on the specific context of interacting with texts, we should keep in mind, that there are many different forms of reading. We would not want to read a table of contents in the same way as we would read a poem. The goal of the first would be informational and navigational, whereas the goal of the second would be immersive and hermeneutic. Therefore it would be clarifying in this context to introduce a third level of interaction. The result would be a model for the process of user interaction with texts visualised below.
The ordering of these three levels of interaction reflects the natural sequential process of the reader’s goals with the given interactions. At first, on the navigational level of interaction, the reader would want to locate the needed information. Secondly, on the hermeneutical level, the reader would need to be “left alone” with the text in order to read it and to obtain its information. Part of understanding the text would also find its place on the manipulatory level of interaction, where the reader will be allowed to process the information by annotating, note-taking, sharing or archiving. At the end of this sequence of interactions the reader would be ready to start over again with new texts to read.
Level of interaction Technology as ... Intention Cognitive state
Navigational Mirror To locate information Perceptual
Hermeneutical Window To read information Reflexive
Manipulatory Mirror To process information Perceptual
Designing for the reading mind
What do these insights mean for the design of digitally distributed texts? As mentioned, the shifts of interaction levels also mark cognitive shifts in the readers brain. These shift are, although inevitable and essential for the interaction with all kinds of technology, increasing the mental workload on the readers mind and should therefor be carefully and purposefully implemented in the design of the technologies for reading digital texts.
When the reader is in the perceptual cognitive state, focusing and reacting on the possible interactions presented on the screen, the technology should be the intelligent and suggestive guide providing the user with relevant information and directions. (Again: in order for the technology to be competent and successful at this job, it is of great importance that the structural and semantic metadata of given information is understandable and actionable for the technology.)
But when the reader is in the reflexive cognitive state, immersed in obtaining the information presented in the given text, the technology should be the discreet and tactful butler, who is letting the reader do the reading, while passively waiting to assist. This assistance should begin when the reader is shifting to the manipulatory level of interaction in which he in some ways wants to engage in the processing of what he has read. Here the technology should provide ways to easily, and with as little disturbance as possible, let the reader take his own notes, share a passage, send a comment to a friend…
In other (biblical) words: the tools we make for digital reading should know that there is “a time to keep silence, and a time to speak” (Ecclesiastes 3:7b) in order to make the most out of the “cerebral real estate” available in the reading mind.