Skip to content

Category: Why?

Collection of posts over the years as to why the work to make digital systems more richly interactive for knowledge work.

The Old Man

old man inventor?

As my first major software application reaches 16th most popular Productivity Application on the macOS App Store I reflect again on what Alan Kay said over lunch: “But you CAN do anything!”. I know he was being sincere, it’s the same thing I would say to a curious and passionate student. My concern is that my brain is fully myelinated, all the fatty optimisers have been laid down, my axons and dendrites are not as much in the dating market as before. I have now managed to produce one instance of my vision in Liquid | Author, which includes Liquid | Flow. It’s a different type of word processor where I had a few medium sized ideas (tag for meaning such as for headings, a different Find command and so on) and did a lot of polishing on use, reducing as many button presses and unexpected frustrations as I could see.

So now I have a smidgeon of credibility, which is great, but what I have produced is only a very small incremental step in a different direction. How can I possibly move symbol manipulation along at a usefully large step? How can I invent something amazingly powerful? This is the question any inventor or artist would ask. And my stodgy old brain, is it still able to fluidly create new and useful connections?! I have noticed that my memory has gotten better over the last few years, which worries an artist-type such as myself. I should be able to be flexible and see things afresh. So I worry.

old man navigator

But my worry gives way to perspective. It’s not my ability to invent something which is why I must keep working and why I invite others to join me. It’s not what I build, it’s about where I am going. I am a navigator, not an inventor.

My work is to augment our ability to orient ourselves in our information, to see and make connections. These are all navigation metaphors and they apply to the process of building the tools as well.

I take responsibility of the question, of the goal, not the answer or a particular way of getting there. Sure, my world is that of visual symbol manipulation (text and associated visual communications) but to navigate we must first choose which world to venture forth into.

future

My future work will be to further improve the navigation and display of text in Author and on the web, including through compressed scrolling, dynamic views and more atomic authoring, including though the user of hyperGlossaries. Much of the work will be to implement small, incremental changes to make the information flow more liquid. The reach of my research and implementation–the twins of progress–will venture as far as my mind will stretch and collages will entertain.

In the distance I see a more liquid information environment where users can become deeply literate because their tools are powerful and they can view and interact with their knowledge in visceral ways, like moulding magical clay.

This is not a solo effort. It is an effort in building dialogue in addition to systems and approaches. This is what I have been doing for almost a decade now with The Future of Text Symposium and look forward to continue.

My premise is simple: Continue to work to employ the occipital lobe to support the prefrontal cortex–to use our eyes to think. And that is my journey.

5 Comments

We can do anything

Question, Inspiration

I have been lucky enough to have a few conversations with the magical Alan Kay this year. When we last had to wrap up our lunch he said to me, plainly and clearly: “But you can do anything!” At first this bothered me since it’s obviously not true (of course I was wrong) and not feasible (again, I was wrong).

This morning of the last day of 2018, the last day of the 50th anniversary of Doug Engelbart’s demo, I get some time to put down my thoughts through my laptop and onto your screen, while my beautiful baby boy who has just started skiing (Edgar, 19 months old) is reading for his mum in the early, dark hours of our Norwegian mountain top cabin.

I have thought quite bit about what Alan said and it’s become clear (though I may still be wrong) that the point is not to come up with a neato-design and build it, but to come up with a design direction with clear goals and frames:

Frames

Let’s start with the frames. I feel it’s out of scope to look at radical new hardware at this point since we already have high-resolution, fast, networked devices, from large, stationary setups to mobile laptops and tablets, phones and even watches, all of which can contribute as access means to a system.

What I do not feel we need to worry about is legacy file formats or legacy ways of working since file formats can always be translated at the point where they need to meet the wider world and legacy ways of working will quickly and easily be seen as such when they come up against something radically better, which is of course our goal.

The only real frames we have to develop within is the human nervous system and that which can be computed and displayed by a computer system.

Goal

The goal must necessarily be quite loose in order to support a long evolution so let’s start with Doug Engelbart’s stated goal in his 1962 paper and add to it based on his further work. I paraphrase:

We must augment our capabilities to
approach complex problem situations,
to gain comprehension and to
derive solutions
collectively.
And we must keep improving our capacities to do so.

The way it is stated, this could apply to numerous fields and aspects of human life and it’s clear that is what he intended. However, in this context I will apply it to the ‘fire’ of giving us the ability to flexibly interact with symbols. I immediately go to the notion of symbols since I do not think that the most effective route to dealing is with the issues above is to present a ’natural’ scene to the user. From the very earliest moments of human recording onto a substrate we have looked at ways to record that which has the most symbolic meaning, not simply slavishly copying down what is visually there in the world.

Symbols

Symbols are the ‘stuff’ of recorded thought and deserve our outmost attention. We can further focus our design goal:

How can we design and co-evolve systems to
augment the readers ability to grasp the authors intention,
in a critical and contextual way,
while also citing the original work usefully,
and augment the users ability to produce further clarity in their own work?

Many will criticise my rabbit hole and might say I have simply provided directions to my own work and I will accept such criticism with the comment that there are of course other avenues to pursue, such as persuasive cinema and educational games and open, free-flowing graphs of variously encoded knowledge but the human ability to express oneself, to record and justify ones position and to make this interpretable and understandable by oneself in the future and by others will surely remain one core aspect of our intellectual and moral work.

Directions forward

To simply reproduce the marks on a passive substrate in the digital environment is not enough. We must experiment with various ways of increasing the interactivity of the symbols–as I have written, interactivity is the core aspect of symbols and of the universe.

As we experiment, with free-form non-linear spaces connected to linear reasoning (LiquidView), with colour coding based on velocity, and with the means through which we can integrate the resulting knowledge product into existing knowledge flows, we also need to look at the spaces of symbols themselves and how the symbols can most powerfully relate with each other in a document on a screen and also between spaces–how to connect and link them, how to experience pattens and movements.

We must think freely, with open minds and share our work.

And we must think with the user, not just for the user. We must listen to the user but also encourage a deeper literacy not heir part. If we, the tool people, can demonstrate more powerful ways of working, they will come. History is witness to this, but only when the more powerful ways of working a persuasively presented to a receptive audience, as audience we need to continually foster a hunger within.

We must think fresh thoughts.

We must also read more of what the founders of the field wrote. We we living in their shadows as well as in their light, we need to better understand their un-fogged boy daily use prejudice perspectives. Something I have promised Alan to continue doing.

Personally, I will continue to experiment with Liquid | Author and LiquidView, blog my findings and thoughts here on the wordpress.liquid.info blog and to host the Future of Text Symposium. How do you, dear reader, wish to produce and connect? Let’s make 2019 a banner year for breakthroughs in employing powerful computer systems to the aid of our thinking, not only to our distraction.

My dream is to look back within a few years and smile at how naive this all was and that it was obvious that the best solution lay somewhere completely different from where we looked, but it was only by going into the first of possibilities that we found this out.

Happy New Year!

P.S., please have a look at my recent, related blog post
Two New Year Wishes.

1 Comment

In the Information War : Arm the Citizen (part 3) Pointing Out Solutions

1 | 2 | 3

For this final chapter on arming the citizenry I am writing en-route to the cradle of alphabetic writing: Egypt. After three days of being bed-ridden and sweating rivers, I am seated at the back of a Boeing 77-300 on Egyptair with good food, plenty of water (though no ice for my brought on-board UK/Norwegian iced mocha), wearing headphones over my Stetson brand baseball cap and my large hoodie over that. I’m awaiting the completion of Liquid | Author 3.5 with the ability to post to WordPress and I’m looking forward to Chris’ work on his ‘weblaux’, or liquid space, or link view or card space or whatever name it ends up being…

That’s my context. This is my point:

Text

To truly arm the citizenry one thing which deserves serious consideration is how we can truly unleash the power of interactive text. The passive medium writing evolved over the first five millennia or so–clay, papyrus & paper–provided scope for powerful innovation which can now be further explored and developed in the truly interactive media of powerful networked computer systems.

We are still stuck within the legacy of the passive medium though, looking, as Marshall McLuhan said, at the future through the rear view mirror. In as much as the inherent interactive character of computer systems can bring real life experiences to hyper-real life, such as visually rich multiplayer combat simulations and fantastical but real looking movies, the interactivity potential of digital text offers rich opportunities.

To be clear, by interactive text I not only refer to the characters of the alphabet but also their connective potential as well as their layouts and any symbols or other visual means to connect and separate them.

Augmenting Text Interaction Augments Us

The core of what I am putting forward here I feel I can best express in bullet points:

  • Text is a visual augmentation of our long term memory
  • Basic interaction with text, such as using a pencil on paper to write and connect makes the text also more of an augmentation of our working memory
  • Making the text more interactive allows the text to augment both our long term and working memory and thus our thinking
  • In physiological terms the interactive text allows the occipital (visual) lobe to augment our prefrontal (higher thinking) cortex

State of The Art

We have seen the very early implementations of this with features such as spell check and the ability to change the look of the text through changing fonts and line breaks on the fly. We have furthermore experienced weblinks where an author can type in an address in a text document to another document and have a high degree of confidence that the reader will be able to ‘click’ in the text and that a ‘web-browser’ will proceed to connect to the location and load the document hosted there.

These are truly wondrous innovations but they are only the very surface of what we can expect to mine the potential of digital text for. Text, as Yuval Noah Harari points out, was invented to solve different problems than speech.

Digital Is Different

On paper some of the inherent qualities of text, such as permanence and linearising of thought, can also be limitations. In digital media these limitations not only disappear they open up new avenues for communicating and thinking with text.

For many who are not involved with thinking about interactive text on a daily basis much of this can understandably be a bit abstract. This is why I have followed the advice of my mentor Ed Leahy: Don’t tell ‘em, show ‘em. So here are some examples of what I am working on. Please refer to my annual symposium on The Future of Text for more work. http://thefutureoftext.org

Examples

At an immediate and visual level we can allow the user to ‘pinch in’ when reading a document to automatically collapse the document to hide all body text and show only headings. Instead of headings and the table of contents being separate, they are now only different views of the same text. This is what we have built into our Liquid | Author word processor for macOS. Further ‘magical’, or digital-only innovations we have built include the ability to select text and hit cmd-f to hide all the sentences in the document which does not have the selected text, a very handy way to check on something which you know was introduced earlier in the document but you forgot where.

With the companion application Liquid | Flow the user can also select any text and have it looked up in any of a myriad of sources within half a second. (Speed matters: As Marshall McLuhan pointed out: The difference between still pictures and movies is the frequency of which they are updated). This is what Doug Engelbart considered to be ‘implicit’ links, in the way that a word is implicitly connected to its entry in the dictionary for example.

In collaboration with Christopher Gutteridge I am working on another one of Doug Engelbart’s ‘obvious’ functions which we are missing today, which highlights our bumpy road into the future: High-resolution links. In most cases today we can only link to whole documents. We don’t even have the ability to point to pages (particularly in commercial eBooks). This severely retards our ability to usefully assess citations which means that academia continues to work on the principles that work known to an advisor or editor is useful but work to a new source is unlikely to be consulted.

Links are one way to move around between documents (and inside documents with internal links) and so is scrolling, since page number don’t mean much in the case of digital documents. However, even scrolling is a simple legacy behaviour. We are not signalling to the system our desire to move up or down in a facsimile of a papyrus scroll when we put two fingers together on a trackpad and move them up or down. We are signalling our desire to view a different part of the document. This is where something like compressed scrolling can come in where what happens is that the body text is gradually compressed so that the user sees more headings and less of illegible body text. More opportunities are described in the blog post Compressed Scrolling including work I hope to carry out with Howard Oakley: http://wordpress.liquid.info/compressed-scrolling/

It is not difficult to think of how we can allow the user to ‘tear’ up the traditional, wood-block on cotton paper rectangles of text and play around with a mind map, Post It note, KJ-Ho or any other visual notes-on-a-board style thinking. As we see on countless drama series on TV, the user could turn their text into a ‘murder wall’ with any media grouped in any way the user thinks is useful with (traditionally) red thread connecting pertinent parts. This is partly what the forthcoming Dynamic Views feature of Liquid | Author aims to deliver : http://www.liquid.info If we go further into the digital mode of interactions, we can choose to not only work with the text directly in front of us authored at the time, but to connect to external data sources at will, such as blogs and academic document repositories and connect and layout as our curiosity takes us, only to rein it back in when it becomes to messy, for then to add another level of connections and views. This is related to the work Christopher Gutteridge and I are doing in the WAIS (Web And Internet Science) department at the University of Southampton where I am currently a PhD student.

Opportunities

Beyond this small menu selection there is a huge scope of opportunities if we choose not to focus on the technological implementations, but simply use what we build to further explore what the most useful interactions can be with text, and with other media. We already have high powered computers in general use and the displays on our desktop are already at high enough resolution and large enough size to cover most of our useful visual range (any larger and we’d have to more our necks more). We need to keep developing using the best understanding we have, in order to understand the principles behind interactive text.

We already know that interactive text is interactive, which is such a core aspect of what digital text is that we named it this. Everything flows from the interactions, otherwise we are simply recreating non-interactive text in an interactive medium, such as we do with PDF documents which are only images.
We furthermore know that text is visual and thus we should keep experimenting with how the text is visually portrayed.
The inherently connective aspect of text, from simple keyword searches to linguistic analysis, is something we know and take for granted in ordinary speech which can be transposed onto digital text.

We need two ask: What is it possible to do with interactive text and what is useful, and for what? How far can we enable the user to sculpt their text to their liking? How deeply can we augment their ability to create and see connections?

Purpose

I asked ‘for what’ at the end of the previous section and to me it it’s simple: The purpose of my work is to foster and give opportunities for ever deeper literacies. It’s not enough to simply know how to read and write today, we must also know intimately the tools at our disposal. In ‘The Ironies of Automation’, Lisanne Bainbridge wrote that a hidden danger of automation is that it will reduce our abilities. An example cited is how we don’t remember phone numbers anymore and struggle with handwriting. I would beg to differ though. As we climb the hill of what systems can do for us, it is both our opportunity and obligation to actively learn how to use the new capabilities handed to us. With the case of phones storing our numbers a new literacy become knowing when to call and when to text, a complimentary functionality opened up by the same smart devices.

Our ‘war’ against misinformation and navigating all the issues thrown up by the term ‘fake news’ as The Road To Unfreedom’ describes, we need to consider that our mental freedoms are rooted in the constraints and augmentations we have in how we interact with our own thoughts and those of others.

Those who does not have the ability to read is at a disadvantage to those who can, when it comes to accessing the thoughts of others and one’s own in the future. Similarly, we talk of the different abilities of someone who has barely started cobbling together letters into words and someone who is a successful author or researcher. It is not outrageous then to look at different levels of literacy of how we use the digital opportunities afforded to us at this point. And beyond.

Aspirations, Questions

Will we get to a point where a document which does not feature high-resolution citations to back up assertions seems as spurious as a schoolyard taunt?

Will we get to a point where a document which only offers one way to view it is viewed as quaint as a Sumarian cuneiform tablet?

Will we get to a point where a document which does nor explicitly make it clear to wherever it is stored what it contains and how it’s connected looks as integrated as a back of an envelope drawing?

Yes, I use the term ‘document’ since it is simply a framing of a human intention. This does not imply a lack of interaction, a lack of flow, simply a frame to hold on to.

The Dream

In the end, if the populace is better informed, better connected to their neighbour and those beyond, in deeper understanding of the contexts of their lives and of our world, I firmly believe we will see a human race with kinder intentions, wiser actions and more harmonious collaboration for a better future for all. This is not a naive position, we will not become angels with more freedom, we will still err and we will still have disagreements and lack of empathy but since it’s clear that isolating ourselves from other people and from other points of view is clearly destructive, I say that integrating and connecting ourselves can be powerfully constructive.

I don’t think we can preach freedom, we must enable it and watch the human mind bloom.

2 Comments