Skip to content

Interacting with our future

In the same way that McLuhan pointed out (McLuhan, 1964) that a bicycle is an extension of our feet and augments our physical mobility, computers are extensions of our brains and augments our mental mobility. How we connect to the world, and information about the world, determines how we hold ourselves to the information and how well we do that will determine the future of our species.
          Evolution has been the mechanism through which we have been able to adapt to fit changing landscapes over millennia. We have moved from water to land and from jungle to savannah and further, changing slightly at every step of the way. Now we are moving more and more into the digital realm when we work and communicate. We no longer only exist where our bodies are. We no longer only communicate at a distance through passive substrates. We now exist through computer networks as much as we exist through our central nervous systems. How we adapt to this will frame how we can meet our opportunities and challenges as we move forward. Our evolutionary pressures have changed and this is the kicker–we are entirely in charge of what the selective pressures for our future evolution will be–how can we not respond to this in an intelligent way and invest in how we exist in digital information?
          The lack of development in word processors and text reading software frightens me. I am a gamer and what I experience with my PlayStation vastly outperforms what I experience with my macOS Work Station. The connective tissue of the Internet remains robust in my opinion, with an inherent layering of redundancies but the first knowledge web weaved around the world, that of the World Wide Web, remains fragile (servers can go down, domain records can stop being paid for, lies can be spread, social media interactions are focused on 1-click propagation without any effort on interaction for comprehension and so on) and I believe, as I first stated a few decades ago (though which I cannot find a citation for), that the web browser is the biggest barrier between the human brain and the biggest collection of knowledge there has ever been.
          This is of fundamental import: At the core, I assert that information arises out of interactions and as such, interaction is the most fundamental aspect of the universe and interaction makes information accessible and useful. To extend the student mind we must extend the tools the student’s use their minds to interact with their work.
          In my effort to work on the software aspect of the future of text I have focused on the student user since I believe students need to be supported in their learning and thinking as much as possible and if we augment students they will hopefully remain augmented throughout their working lives. It is not enough to design software for quick sales with impressive ‘first-use’ experiences.
          As more and more of us become digital knowledge workers we must design tools commensurate to the challenge which digital knowledge entails. This means that there will be some learning curve, as there is with any other professional tools. We can no longer afford icon-click simplicity for first use. Imagine learning to speak a language only by clicking on pictures–you would not get far. This is how many are learning the language of how to speak with our computer systems.
          Most of the information a student interacts with is in digital text form. I therefore believe that in order to augment the student’s thinking we need to develop better text interactions.
Images, including video and 3D, will continue to also be a vital media but creating them requires skill and resources far beyond typing and reading, and while powerful software has been developed for this, the power of grammar and syntax in text will keep text as a particularly powerful means of expression and thought.
          AI will play a part of the evolution of how we interact with our knowledge, that much is sure, but AI is receiving a large amount of investment whereas text interaction is not. Our goal should be IA (intelligence augmentation), not simply offloading thinking to AI. Having someone else, or something else, do all our thinking for us is never a viable solution–we cannot outsource our responsibility to understand.
          I accept the importance of the document. An art teacher told me in high school that at some point I had to stop adding to a picture (I was extending the frame to be bigger and bigger) and accept it as done. This has stuck with me. A document is an important construct, it is a statement framed. This is different from the ever receding social media texts and ever editable document types such as Google Docs and Microsoft Word. This is a core aspect of academia: ‘Publishing’ a document which then future documents can address/refer to, or cite and this is how academic discourse takes place.
          In terms of how we should go about researching and delivering enhanced interactions with digital text, I developed a philosophy of ‘Liquid Information’ and the creation of a Liquid Information environment, as I wrote about in Wired (Hegland, 2005). Taking a look at the notion of ‘Cyberspace’ is a useful way to colour in what I mean by liquid information. The word prefix ‘cyber’ in ‘Cyberspace’ comes from the Greek kybernetes ‘steersman’ (metaphorically ‘guide, governor’), from kybernan ‘to steer or pilot a ship, direct as a pilot,’ figuratively ‘to guide, govern,’ (Harper, 2021). I think this is central to how we should think about digital text and academic research: Cyberspace means navigation space. More than the academic focus on the term ‘hypertext’, this is what holds the paradigm which guides my own work.
          The potential for how digital information can be represented is almost endless. If we look at digital information as being stored in zeros and ones and therefore needing interpretation in order to be represented, which is another aspect of interaction, then this opens up a fluid world of possibilities for how we can interact with the information.
          However, there are very real constraints, the first of which what we as physical beings can best interact with, then what it is possible for computer systems to present, which is a moving question as computers get increasingly capable, and the current standards, both technical and social, for how things are done.
          The current academic standard for document interactions is PDF, which is great from the perspective of robust data storage in a digital environment but not so great for rich interaction since the documents are as flat and as devoid of metadata as is possible. The current academic standards also specifies how PDF documents should be presented in terms of visual formatting and use of citations.
          The goal of my work is to enable a user to be able to interact with their information as though it is magical clay, that is the mental model I have of the interactions. Examples of this include expanding and contracting how much information is displayed, and being able to follow any explicit or implicit connections at will.
          I am looking at improving (augmenting) how people navigate in information space, both in terms of how they can relate themselves to the information (navigation) and how they can relate the information to themselves (interaction).
          To make this possible infrastructures have to change. Documents need to carry more information about what they are–metadata.
          So far I have developed a way to turn PDF documents into rich containers of metadata without affecting compatibility. I call this Visual-Meta.
          I have built interactions which allow the user to effortlessly fold documents into, find a view of only sentences they are interested in, define terms for readers to access without even thinking about it, cite with copy and paste, think freely in a non-linear space which is connected to their word processing view and let users to follow a spark to look up any text in an search engine or reference work within half a second.
          Doug Engelbart wrote in 1962 “We do not speak of isolated clever tricks that help in particular situations. We refer to a way of life in an integrated domain where hunches, cut-and-try, intangibles and the human ‘feel for a situation’ usefully co-exist with powerful concepts, streamlined terminology and notation, sophisticated methods, and high-powered electronic aids.” (Engelbart, 1962) This is the paradigm which guides the work.
          To me the need to augment how we interact with text is self evident. I simply cannot understand how it is not obvious that we need to seriously upgrade how we interact with text in order to upgrade how we think. I find it very hard to express in ways which will persuade–I have tried for decades–but having managed to produce a word processor, a PDF viewer and a liquid text tool, I can let the ‘inter’-actions speak louder than words. I have a 3 min demo at https://www.augmentedtext.info which presents this in ways which static text cannot. Please have a look.
      ­
So far precious little work is being done. Searching the web for “the future of text” results in only my work, with a single exception, on the first page. Job one has to be to get more people involved, hence the annual symposium and the book on The Future of Text.

Published inNotes On...ThoughtsVisual-MetaWhy?

Be First to Comment

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.