Doug Engelbart stated that his mission was to augment human intellect, later expanded to our collective IQ.
In his seminal Augmenting Human Intellect (Engelbart, 1962) paper he elaborates on what he meant by this; more rapid and better comprehension for more complex situations which result in speedier and better solutions for problems which before seemed insoluble – and he later emphasised collectively.
He further specified that he was concerned with urgent, complex problems, not run of the mill problems.
One of the key components of his approach was the importance of what he called ViewSpecs. ViewSpecs are short for ‘View Specifications’ and are specified views of a document. In his NLS/Augment system he included options for showing only headings, first line of every paragraph and for specifying how the text should be displayed, such as with extra space between the lines and hiding or showing the ID number of every statement. Doug discussed the possibility of further view options with me many times and we even mocked up one of his ideas to useful effect; He thought it could be valuable to colour code words by category, so for example words which relate to technology could be yellow, business blue and people green for example, using a (admittedly quite subjective) glossary of terms. This allowed the reader to rapidly skim a document or see thumbnails and get an idea of what section covered primarily what subject. He also suggested making common words slightly greyed out so that it would be easier to skim to see what words were less used but this turned out to be less useful, basically making the document appear more grey.
His insight that more powerful views could us more powerful connections with our information is supported by how powerful the human visual system is and how small our working memory is.
I have chosen document discourse as my focus since there are significant efforts underway to in the fields of images, video, 3D and AI but very little is happening with what Doug Engelbart referred to as ‘symbol interaction’ – text. My concern is the whole document process of finding, reading and authoring but what I am concentrating my research on is creating a non-linear space for discovering what the user already knows, so that this can be mapped out, and subsequently worked into increasingly useful patterns.
The vehicle I will be using to perform my research is what I call Dynamic Views in my word processor called Author: www.liquid.info I have developed this platform so I have full control over how to use it to better understand what visual modes actually meet Doug Engelbart’s requirements and which are simply impressive visuals.
It is important to note that these Dynamic Views are views of word processor text which the user can instantly toggle in and out of, these views are not a separate system since that would add too much cognitive overhead and variables to take into account of.
In order to support the Criteria, the system should support the user’s ability to connect their neural net to the visual net on the screen to allow for flexible interactions to see relationships and groupings, resulting in increased comprehension and thus solutions – explicitly to mirror and extend their minds.
Author is a commercial word processor for macOS and iOS which I have developed for the purposes of furthering Doug’s augmentation philosophy and the basic build of Dynamic Views are also clear and useful enough to be a commercial product. My research however, is about what happens when we have a basic Dynamic View; What works and what doesn’t and more importantly – how far can we delve into the potential of richly interactive, connected cognitive workspaces?
The Views are a research platform, not a single pre-decided implementation, to learn the effects of interactions in a text space which is not constrained by traditional fixed columns of text. The research questions then are:
• What aspects of the visual presentation and the interactions contribute to supporting the specified Criteria and which detract?
• What elements are necessary to include in order to deliver on the Criteria? Only Heading Nodes or also elements from body text, Citations, Links and Glossary items?
• What layers of information and meaning (semantic and visual) can be employed and how can the author and reader be given interactive access to them and how do they relate?
• What dynamic options are possible? Such as multiple searched saved with colours (based on glossary) to allow a different kind of insight.
• How multidimensional can the views be?
• Furthermore, since information is inherently relationships, can the system be designed so that nodes and connecting lines are interchangeable. How can the be turned into a practical interface, it at all?
Research in this field, also dubbed Spatial Hypertext, reached its peak at the close of the previous millennium but the research community is still very much interested in furthering this field and through my continued hosting of The Annual Future of Text Symposium I am building a dialogue network to draw on decades of insights into what views we should consider building and testing and I will also consult to a degree with other fields, including the study of human perception and cognition, for insights into design strategies.
In order to test approaches I will need to build (through design, I a not self a programmer) a flexible prototyping environment for the macOS (and soon iOS) Author word processing application. Since the basic word processing capability is developed and developed with Dynamic Views in mind, different Dynamic View modules can be developed and slotted in for testing, resulting in a stable and rapidly available prototyping environment.
Evaluation will be based on Doug Engelbart’s original Criteria, as listed above, with qualitative and quantitative testing and all findings will be made public as the project progresses. The goal is to produce results which other word processor builders can also use.