On the Primacy of Interactions

Where I preach for a minute:

The stuff of work for a knowledge worker is not information, it is interaction.

This is because information is the result of interaction – interaction is more fundamental than information and I think this is crucial to consider.

Information has its name because in order for something to be information it has to be (to play with words); in-formation, it needs to have some relation to other information and this is where the power of information comes from, as well as its complexity and, let’s be honest, the messiness.

The energy needed to organise information, to put it in-formation is greater than the energy needed to produce the information to begin with and when organised it is only organised based on finite criteria so to focus on the organisation of information should be thought of as a dynamic process where we think of the desired interactions we need to make possible.

This then has knock-on effects for how we design the user experience and how we store and share our knowledge – do we share our knowledge in frozen form or in richly interactive ways which allows for limitless connections, associations and links to be seen and re-drawn?

That is not a rhetorical question, I would greatly like to know what you think.

Proposal to wider group after weekly call 13 December 2017

Everyone, it was a good call and we continue next week.

We coalesced around a list of threads for the work/competitions: Link Improvements, Classifications, Interoperability, Dynamic Views & Far Horizons.

However, through emails after the call the ones who remained towards the end continued and we are now leaning towards using Doug’s notion of a DKR as our focus. If we agree on this then what Jack calls semantic interoperability seems to be a good aim for how we aim to glue these disparate pieces together. Here is a bit of framing for the DKR:

Doug told me he had an epiphany in 1951: Boy, the world is complex, jeez, the problems are getting more complex and urgent and have to be dealt with collectively- we have to deal with them collectively. He would go on to write the seminal Augmenting Human Intellect paper, where he put a stake in the ground. He stated that his mission, what you might call his core principles was to augment human intellect, by which he meant:

Increasing the capability of a knowledge worker to
approach an urgent, complex problem,
and gain more rapid and better comprehension
which result in speedier and better solutions
collectively.

In his words, from our documentary : These ‘means’ can include many things–all of which appear to be but extensions of means developed and used in the past to help us apply our native sensory, mental, and motor capabilities– and we consider the whole system of a human and his augmentation means as a proper field of research for practical possibilities. It is a very important system to our society, and like most systems its performance can best be improved by considering the whole as a set of interacting components rather than by considering the components in isolation. So the job then became a matter of finding the factors that limit the effectiveness of the individual’s basic information-handling capabilities in meeting the various needs of society for problem solving in its most general sense; and to develop new techniques, procedures, and systems that will better match these basic capabilities to the needs’ problems, and progress of society. This is important, simply because mankind’s problem-solving capability represents possibly the most important resource possessed by a society. The other contenders for first importance are all critically dependent for their development and use upon this resource.
http://invisiblerevolution.net/thestory.html

For this he invented a whole ecosystem which later come to refer to as a Dynamic Knowledge Repository, a ‘DKR’, which is concerned with; mental structures, concept structures, process structures and physical structures. The result of this dynamic environment was NLS/Augment where capabilities were developed within and across these structures in order to deliver on the augmentation goals.

Can we, as a group, re-define the DKR for the 21st Century in order to invite others to come and play and invent with us? I suggest we take Doug’s core principles as our main aim and inspiration from his implementations to catapult us far beyond what we can imagine with our current capabilities.

If we agree on this then what Jack calls semantic interoperability seems to be a good aim for how we aim to glue these disparate pieces together.

Shall we proceed?

 

Dynamic View Research Proposal

Augmenting Human Intellect

Doug Engelbart stated that his mission was to augment human intellect, later expanded to our collective IQ.

Criteria

In his seminal Augmenting Human Intellect (Engelbart, 1962) paper he elaborates on what he meant by this; more rapid and better comprehension for more complex situations which result in speedier and better solutions for problems which before seemed insoluble – and he later emphasised collectively.

He further specified that he was concerned with urgent, complex problems, not run of the mill problems.

ViewSpecs

One of the key components of his approach was the importance of what he called ViewSpecs. ViewSpecs are short for ‘View Specifications’ and are specified views of a document. In his NLS/Augment system he included options for showing only headings, first line of every paragraph and for specifying how the text should be displayed, such as with extra space between the lines and hiding or showing the ID number of every statement. Doug discussed the possibility of further view options with me many times and we even mocked up one of his ideas to useful effect; He thought it could be valuable to colour code words by category, so for example words which relate to technology could be yellow, business blue and people green for example, using a (admittedly quite subjective) glossary of terms. This allowed the reader to rapidly skim a document or see thumbnails and get an idea of what section covered primarily what subject. He also suggested making common words slightly greyed out so that it would be easier to skim to see what words were less used but this turned out to be less useful, basically making the document appear more grey.

His insight that more powerful views could us more powerful connections with our information is supported by how powerful the human visual system is and how small our working memory is.

My Scope : Document Discourse

I have chosen document discourse as my focus since there are significant efforts underway to in the fields of images, video, 3D and AI but very little is happening with what Doug Engelbart referred to as ‘symbol interaction’ – text. My concern is the whole document process of finding, reading and authoring but what I am concentrating my research on is creating a non-linear space for discovering what the user already knows, so that this can be mapped out, and subsequently worked into increasingly useful patterns.

ViewSpecs inspire Dynamic Views

The vehicle I will be using to perform my research is what I call Dynamic Views in my word processor called Author: www.liquid.info I have developed this platform so I have full control over how to use it to better understand what visual modes actually meet Doug Engelbart’s requirements and which are simply impressive visuals.

It is important to note that these Dynamic Views are views of word processor text which the user can instantly toggle in and out of, these views are not a separate system since that would add too much cognitive overhead and variables to take into account of.

Mental Process to Support

In order to support the Criteria, the system should support the user’s ability to connect their neural net to the visual net on the screen to allow for flexible interactions to see relationships and groupings, resulting in increased comprehension and thus solutions – explicitly to mirror and extend their minds.

The Host : Author

Author is a commercial word processor for macOS and iOS which I have developed for the purposes of furthering Doug’s augmentation philosophy and the basic build of Dynamic Views are also clear and useful enough to be a commercial product. My research however, is about what happens when we have a basic Dynamic View; What works and what doesn’t and more importantly – how far can we delve into the potential of richly interactive, connected cognitive workspaces?

Research Questions

The Views are a research platform, not a single pre-decided implementation, to learn the effects of interactions in a text space which is not constrained by traditional fixed columns of text. The research questions then are:

• What aspects of the visual presentation and the interactions contribute to supporting the specified Criteria and which detract?

• What elements are necessary to include in order to deliver on the Criteria? Only Heading Nodes or also elements from body text, Citations, Links and Glossary items?

• What layers of information and meaning (semantic and visual) can be employed and how can the author and reader be given interactive access to them and how do they relate?

• What dynamic options are possible? Such as multiple searched saved with colours (based on glossary) to allow a different kind of insight.

• How multidimensional can the views be?

• Furthermore, since information is inherently relationships, can the system be designed so that nodes and connecting lines are interchangeable. How can the be turned into a practical interface, it at all?

Approach

Research in this field, also dubbed Spatial Hypertext, reached its peak at the close of the previous millennium but the research community is still very much interested in furthering this field and through my continued hosting of The Annual Future of Text Symposium I am building a dialogue network to draw on decades of insights into what views we should consider building and testing and I will also consult to a degree with other fields, including the study of human perception and cognition, for insights into design strategies.

In order to test approaches I will need to build (through design, I a not self a programmer) a flexible prototyping environment for the macOS (and soon iOS) Author word processing application. Since the basic word processing capability is developed and developed with Dynamic Views in mind, different Dynamic View modules can be developed and slotted in for testing, resulting in a stable and rapidly available prototyping environment.

Evaluation

Evaluation will be based on Doug Engelbart’s original Criteria, as listed above, with qualitative and quantitative testing and all findings will be made public as the project progresses. The goal is to produce results which other word processor builders can also use.