Making Information Self Aware

Published by Frode Hegland on July 14, 2019

We can fight fake news and find more useful information in the academic and scientific publishing tsunami if we make the information self aware–if the information knows what it is. This is not a suggestion of Harry Potter level magical fantasy but a concrete act we can start with today and lay for foundation for future massive improvement.

the intelligent environment

Many years ago I read an interview with one of the developers of the computer game Crysis where he was lauded with the quality of the AI of the opponents in the game. He said that making the AI was not really the hard part, making the different parts of the environment aware of their attributes was key. If a tree trunk is thick, then the enemy can hide behind it. If it is dense then it will also serve as a shield, up to a point.

the self aware document

This is what we can and must do to documents. We must encode the meaning in documents as clearly as possible so that the document may be read by software and human. The document must be aware of who authored it, when, what its title is and so on, to at least provide the minimal context for useful citations.

It should also know what citations it contains and what any charts and graphs means what glossary terms are used and how they connect. Of course, we call this ‘metadata’ – information about information and the term has been used in many ways for many years now, but the metadata has so far been hidden inside the document, away from direct human and system interaction. We should maybe instead call it ‘hiddendata’. For some media this is actively used, such as the EXIF data in photographs, but it is lost when the photograph changes format, is inserted into other media or is printed. For text-based documents this is certainly currently possible but seldom actually used and not usefully read by the reader software and lost on printing.

bibtex foundation

You may well feel that this is simply a call for yet another document format but it is not. This is simply a call for a new way to add academic ‘industry-standard’ BibTeX style formatting of metadata to any document, starting with PDFs, in a robust, useful and legacy friendly way, by simply adding a final appendix to the document which follows a visually human-readable (hence BibTeX) and therefore also machine parseable format.

As this will include who authored the information, which the reading software can ‘understand’ and make it possible for the user to simply copy text from the document and paste it as a full citation into a new document in one operation, making citations easier, quicker and more robust. Further information can be explained for reader-software parsing, such as how the headings are formatted (so that the reader software can re-format the document if required, to show academic citation styles in the preference of the reader if they are different from the presence of the author), what citations are used, what glossary terms are used and what the data in tables etc. contains and more.

more connected texts

This is making the document say what it is, where it comes from, how it’s connected, what it means, and what data it contains. This is, in effect, making the document self aware and able to communicate with the world. These are truly augmented documents.

This will power simple parsing today and enable more powerful AI in the future in order to much better ‘understand’ the ‘intention’ of the author producing the document, by making documents readable.

This explicitly applies to documents and has the added benefit that even if they are turned into different formats and even if they are printed and scanned they will still retain the metadata. The concept is extensible to other textual media, but that is beyond this proposal.

visual-meta

I call this approach Visual-Meta and it’s presented in more detail here liquid.info/visual-meta.html. I believe this is important and I have therefore started the process of hosting a dialog with industry and I have produced two proof-of-concept applications, one for authoring Visual-Meta documents and one for reading and parsing them: Liquid | Author and Liquid | Reader: www.liquid.info

paper

Digital capabilities run deeper than what previous substrates could, but even in the pursuit of more liquid information environments we should not ignore the power of the visual symbolic layer. We hide the meta at our peril – we reveal it and include it in the visual document and gain robustness through document format changes and even writing and scanning, gaining archival strength without any loss of deep digital interactivity, something which matters more and more as we live and discover how brittle our digital data is and how important rich interactivity is to enable the deeper literacy required to fight propaganda and to propagate academic discoveries often lost in the sheer volume of documents.

Furthermore, with the goal of more robust formats and supporting reading of printed books and documents, addressing information (as discussed in the Visual-Meta addressability post) can be printed on each page in the footer to allow for easy scanning of hand-annotated texts to be OCR’d and entered into the user’s digital workflow automatically. Digital is magic. Paper is also magic. One day they will merge, but until then there is value to be had to use both to their strengths.

As we make our information aware,
we increase the potential of our own awareness

Liquid | Reader is finally live

Published by Frode Hegland on July 12, 2019

Nice.

What this means is initially that when someone copies text from a PDF and pastes into Liquid | Author the text will automatically be a citation.

What it means for the future, if this approach is taken up, is that context, which is crucial for any information, will no longer be buried inside the document with all the fragility that entails, but surfaced for access by the human reader to ‘manually’ interact with, software to parse and servers to analyse.

It means that archiving becomes one step less frail, with all the contextual metainformation kept on the same level as the actual body of the document. A Visual-Meta document can change formats, be printed, scanned and OCR’d and it makes no difference for future metainformation extraction.

I am not one for blowing my own trumpet, I have barely been able to ‘sell’ Liquid | Flow (which gives the user a myriad of useful commands instantly responsive) and Liquid | Author (which provides a unique dynamic view of the users text) but I feel strongly that the Visual-Meta approach is important and needs to be taken forward by the community.

Stages for what to visualise for LR

Published by Frode Hegland on July 2, 2019

My PhD work is to “analyse current methods of performing a Literature Review and known hypertext opportunities and design specific visually augmented views to augment the student ability better deliver on the University PhD requirements assessment criteria and build and live in such a system to ascertain what is particularly useful for specific use-cases” as agreed with Les Carr and Wendy Hall.

University PhD Literature Review Requirements

The guide to conducting a literature review on the University of Southampton website is clear: “Conducting a literature review enables you to demonstrate your understanding and knowledge of the existing work within your field of research. Doing so allows you to identify any undeveloped areas or unexplored issues within a specific debate or field of study. This, in turn, helps you to clearly and persuasively demonstrate how your own research will address one or more of these gaps.” This guide is without an author itself so it is, a bit ironically, difficult to cite by anything more than it’s web address: http://library.soton.ac.uk/ld.php?content_id=31944998 It follows with specific recommendations on writing your literature review:

“Show the connections between your sources. Remember that your review should be more than merely a list of sources with brief descriptions under each one. You are constructing a narrative. Show clearly how each text has contributed to the current state of the literature, drawing connections between them.”
“Engage critically with your sources. This means not simply describing what they say. You should be evaluating their content: do they make sound arguments? Are there any flaws in the methodology? Are there any relevant themes or issues they have failed to address? You can also compare their relative strengths and weaknesses.”
“Signpost throughout to ensure your reader can follow your narrative. Each time you bring up a new source it should be made obvious to your reader why you are doing this and where the discussion is headed. Keep relating the discussion back to your specific research topic.”
“Make a clear argument. Keep in mind that this is a chance to present your take on a topic. Your literature review showcases your own informed interpretation of a specific area of research. If you have followed the advice given in this guide you will have been careful and selective in choosing your sources. You are in control of how you present them to your reader.”

For all of these points the citation information needs to be available in useful digital form for a visualisation system to be useful beyond a demo and this is why I developed the Visual-Meta system: http://wordpress.liquid.info/printed-meta/ and developed the Liquid | Reader to be able to experience and improve a full workflow: http://www.liquid.info/reader.html

The first bullet point hints about visualisations which can show visual lines between sources. For this aspect I have built the Dynamic View into Liquid | Author, as demonstrated in this video on YouTube viewed over 30,000 times: https://www.youtube.com/watch?v=bCpJTRd0hrE&t=16s and described on the Liquid | Author website, which also lists the (highly positive) user reviews: http://www.liquid.info/author.html

The other points reflect less on a potential visualisations and more on the student’s mental effort of critically engaging and structuring an (linear) argument focused on the research topic. These are aspects traditionally covered by plain text but which hint at visual augmentation.

The point of my PhD is not primarily to augment the student to do the work, but to demonstrate to the examiner that the work as indeed done.

Engage critically with your sources. To show that the student engaged critically with sources could be done by giving the sources link-types, which can be colour coded for example.
Signpost throughout to ensure your reader can follow your narrative. This can also suit the Dynamic View model by allowing the author student to present highlights and directions clearly.
Make a clear argument. An argument is essentially linear but is informed by non-linear sources. This can also be show in an expanding and collapsing Dynamic View model.

In view of this, I am still working on how to best show citation sources in the Dynamic View as blogged under:

Category: Updates