I’ve started my PhD studies and as such I have started diving in to the literature. One of the first things I did was look at Craig Tashman’s PhD thesis for LiquidText since it clearly has some commonalities to my own work. I am inspired by his work but it is also quite different from what I am trying to do. Anyway, I think I should spend a little time on defining the State of My Art in terms of how I see the issues and potential solutions at this stage. This is a bit of a hodge lodge at this point, with reading and writing features mixed in, and I hope I can stretch it quite far, then maybe edit, but this is what I have today…
• It takes less energy to produce information than it does to organise or assimilate the information. This leads to information overload.
• What is important to note however, is that information is not most fundamental – interaction is. This is because information arises out of interactions, and this gives us a deep insight as to how to most efficiently deal with the information overload: increase the interaction we have with the information.
Information is stored in many forms. My particular interest however, is documents. To really start from first principles I have to ask what a document is, what the point of a document is:
• A document is the framing of a human perspective. This highlights why documents are important: the process of thinking, justifying and presenting a coherent argument
• The intent of a document is to convey this perspective.
Document Component Connections
It is important to note that the useful information in the document is not only contained in the text – it is also contained in the relationship between different pieces of information – and in the structure of the document. This highlights specific needs for specific types of interactions.
Furthermore, it is important to note the different motivation and processes which lead to writing vs what leads to reading:
• Writing is a way of recording information for the author’s reference later, for ‘publication’ or a combination of the two.
The Issue With Writing
• Writing is the process of transcribing thoughts into coherent, grammatical sentences and sentence fragments. Our mental constraints means that a fully structured and cohesive model of what we are saying or typing does not come out smoothly in one continuous stream – we simply don’t have the mental space for this to occur. A result of this is that editing the writing becomes an inherent part of writing but because the reason that editing is needed is our lack of mental space, this becomes mentally taxing and software systems have not developed rich editing environments. The result is too much redundant text and not enough connections and structure.
• We read to gain an increased understanding of a subject. As such, we can greatly benefit from well-ordered, clear, properly cited documents, but this is not what writing naturally produces, so we need to build tools and systems to help the author, who has different motivations and energy, produce what the reader will benefit from, whether the reader will be themselves in the future or someone else.
• When we read to understand, rather than simply for the entertainment value, regardless of whether the material was well written and well structured, we benefit from active reading (Craig Tashman term, central to LiquidText), including making annotations.
Key Issue : Too Much Text, Not Enough Structure
A key issue, which we must not ignore, is that there is way too much text to read. There is too much crud around the important, meaningful text and the meaningful text itself can be belaboured and repeated.
Also, the text is not explicitly well structured.
Approaches to Deal With This
Here is a list of a few of my actual ‘inventions’ or ideas I have somehow gotten from others. Either way, these are concrete approaches for helping us deal with the key issue of too much text.
It is important to let the reader and the writer flexibly change the view of the document. This includes, and is quite a familiar theme in the industry most of this, the ability to:
• Zoom to an overview and zoom back in, where the action of zooming in and out support different needs, such as showing more information when zooming in (summaries, notations etc.).
• Change the view of the document to show specific text, information or relationships.
• Remove unnecessary material through using summarisation techniques to remove repetitions etc.
• Look up any text as with Liquid | Flow and more advanced.
• Compare sections of text with others in the document or elsewhere. This likely calls for both a mind map view and a hybrid mind map view.
The bridge between the underlying fundamentals and specific ways to deal with them are not always clear. In some ways the fundamentals are merely inspirations for solving very specific human information interaction needs. I have suggestions to how Author can be, which are mostly modest features but I feel it’s important to have a track for a relatively normal improvement so that I can relax and think bigger… http://www.liquid.info/author-project.html for the list.
More Advanced Approaches
Writing should be very quick to change between typing and speech.
Action on Text
The user should be able to select text, say a word, a sentence or a paragraph and act upon this, not just to look it up, but as a basis for manipulating the document. This should include actions such as:
• Show only sentences with this text (though sentences which refer to the text should also be included, as the High Performance Text Interaction Libraries should allow)
• Show relevant occurrences elsewhere
Pleasure of Pinch
Pinching out to see an overview is not the same as pinching in:
You would pinch out to collapse the document into headings only, then pinch more and more to collapse the levels one by one.
You would then tap on to ‘jump’ to or just read/skim then dismiss to return to where you were reading.
If you have pinched out and collapsed all levels of heading apart from the top level, you can pinch back in to reveal the underlying headings, one by one. Once you have all levels open you should be able to pinch yet again, to have a single sentence summary of each level appear. Pinch yet again and the document opens. Any notes the user has added should appear on this level as well.
The writer should be able to choose to collapse a full document and only work/see a section, to make it easer to get an overview.
Lists should be collapsible and should be able to be horizontal as well as vertical.
I define author added extra information as being Comments and reader added extra information as being Notations.
It would be valuable to take as much power and flexibility of paper-notation into the digital document, while retaining the power of the digital substrate. As such, a separate layer for notations should be provided for, while keeping accurate relationship with the underlying text.
When selecting sections of text, the user should be able to:
• Annotate with text and links etc.
• Draw anywhere
• Create connections with other passages of text
It is important that the notations remain with the document when the user chooses to enter mind-mapping mode, below.
A commentary layer for discussion of the content following a global standard, maybe the w3 standard, will be an important addition to the discourse.
In support of multiple views, the document should be possible to turn into a card-based mind-map view, where headings are cards (which can be nested based on levels) and body text collapses into the cards for this view, though of course they can be expanded in this view as well.
How to implement this view I am not sure, but it should feel tactile, like a pinch or a swipe. Maybe 4/5 finger pinch out?
There is no inherent reason why the document should not be somewhat pre-processed for such things as adding definitions next to words which are specialised words in a subject area the reader is now familiar with (the reader would need to request this, or the system could learn over time what the reader knows). Also, basic summarisation could be done, to for example grey out all text in the document with is repeating information.
For writing for students, a type of assistant which the student can invoke which will ask questions about the document, starting with what type of document is being written, then asking where certain information is, as a template would dictate should be there, then the student selects the text and the system tags that section with that topic, or the student acknowledges that the section is not done and writes that section, then writes it and does the process again.
Much is being said about AI and this is one area where it can be useful. On ‘Publishing’ a document, rather than simply saving it, a series of AI modules can check for plagiarism, writing level and so on, as well as summarising the document so that the writer knows what was attempted to get across is actually what came across.
Masses of Meta
Meta-Information should not be a crude add-on, but a centrally thought through aspect of the visual marks on the screen. Tags assigned in HTML editors today can easily be broken when you cut and paste. Tags should be robustly attached to the text and breaking tags through selecting text without consideration of the tags should be technically protected. Tags should record times of writing, location and much more, to help the reader go through the material later.
Imaging the teacher having a spreadsheet with al students listed and updates on how they are doing with all their readings and all their papers in one view. It could even be colour coded to show how they are feeling, based on deep learning of large corpuses of documents written by happy, sad and people with other known states of mind. Many other pieces of information can be tracked here, with student consent, such as how often they read and more, to help the tecaher get a better understanding of students (particularly quiet ones) are doing.
Stan Gould wrote to me: “I believe you have a nascent paradigm-shifting offering, whose greater value to society simply awaits the ability to frame it that way. It is my opinion, that in our new era you will find that explicitly designing “frameworks, platforms & ecosystems” and conceptually placing your Apps within those conceptual organizational objects will both resonate with the marketplace and amplify your success efforts.” (docs.google.com)
He is right. It’s not enough to cobble together features in apps, we need to change the flow of education, to increase the value of the act of writing and citing well, to increase the power of active reading, while integrating with all the media accessible. This will be a big challenge.
It is clear that citations will essentially need to be re-invented in the digital world, to help both student make them quicker and more clearly, and for teachers to better check them. This is a key issue.
I was lucky enough to attend the launch of the UK and Ireland W3C office on Friday and among the brilliant people and brilliant projects, I heard mention of a W3C workshop on VR (the first one…) in Silicon Valley this coming week and then things just exploded in my head: In the same way we need standards and security – in the same way we need the w3C for ‘2D’ rendered web, we certainly need the same for the upcoming VR rendered web. I hope to attend the workshop, bit they are out of space and it’s very unlikely they will allow me in, but I hope I can be there, at the very start of this… Please also refer to my Universe Sculpture post: http://wordpress.liquid.info/wp-admin/post.php?post=2269&action=edit for issues related to this.