Making Information Self Aware

Published by Frode Hegland on July 14, 2019

We can fight fake news and find more useful information in the academic and scientific publishing tsunami if we make the information self aware–if the information knows what it is. This is not a suggestion of Harry Potter level magical fantasy but a concrete act we can start with today and lay for foundation for future massive improvement.

the intelligent environment

Many years ago I read an interview with one of the developers of the computer game Crysis where he was lauded with the quality of the AI of the opponents in the game. He said that making the AI was not really the hard part, making the different parts of the environment aware of their attributes was key. If a tree trunk is thick, then the enemy can hide behind it. If it is dense then it will also serve as a shield, up to a point.

the self aware document

This is what we can and must do to documents. We must encode the meaning in documents as clearly as possible so that the document may be read by software and human. The document must be aware of who authored it, when, what its title is and so on, to at least provide the minimal context for useful citations.

It should also know what citations it contains and what any charts and graphs means what glossary terms are used and how they connect. Of course, we call this ‘metadata’ – information about information and the term has been used in many ways for many years now, but the metadata has so far been hidden inside the document, away from direct human and system interaction. We should maybe instead call it ‘hiddendata’. For some media this is actively used, such as the EXIF data in photographs, but it is lost when the photograph changes format, is inserted into other media or is printed. For text-based documents this is certainly currently possible but seldom actually used and not usefully read by the reader software and lost on printing.

bibtex foundation

You may well feel that this is simply a call for yet another document format but it is not. This is simply a call for a new way to add academic ‘industry-standard’ BibTeX style formatting of metadata to any document, starting with PDFs, in a robust, useful and legacy friendly way, by simply adding a final appendix to the document which follows a visually human-readable (hence BibTeX) and therefore also machine parseable format.

As this will include who authored the information, which the reading software can ‘understand’ and make it possible for the user to simply copy text from the document and paste it as a full citation into a new document in one operation, making citations easier, quicker and more robust. Further information can be explained for reader-software parsing, such as how the headings are formatted (so that the reader software can re-format the document if required, to show academic citation styles in the preference of the reader if they are different from the presence of the author), what citations are used, what glossary terms are used and what the data in tables etc. contains and more.

more connected texts

This is making the document say what it is, where it comes from, how it’s connected, what it means, and what data it contains. This is, in effect, making the document self aware and able to communicate with the world. These are truly augmented documents.

This will power simple parsing today and enable more powerful AI in the future in order to much better ‘understand’ the ‘intention’ of the author producing the document, by making documents readable.

This explicitly applies to documents and has the added benefit that even if they are turned into different formats and even if they are printed and scanned they will still retain the metadata. The concept is extensible to other textual media, but that is beyond this proposal.

visual-meta

I call this approach Visual-Meta and it’s presented in more detail here liquid.info/visual-meta.html. I believe this is important and I have therefore started the process of hosting a dialog with industry and I have produced two proof-of-concept applications, one for authoring Visual-Meta documents and one for reading and parsing them: Liquid | Author and Liquid | Reader: www.liquid.info

paper

Digital capabilities run deeper than what previous substrates could, but even in the pursuit of more liquid information environments we should not ignore the power of the visual symbolic layer. We hide the meta at our peril – we reveal it and include it in the visual document and gain robustness through document format changes and even writing and scanning, gaining archival strength without any loss of deep digital interactivity, something which matters more and more as we live and discover how brittle our digital data is and how important rich interactivity is to enable the deeper literacy required to fight propaganda and to propagate academic discoveries often lost in the sheer volume of documents.

Furthermore, with the goal of more robust formats and supporting reading of printed books and documents, addressing information (as discussed in the Visual-Meta addressability post) can be printed on each page in the footer to allow for easy scanning of hand-annotated texts to be OCR’d and entered into the user’s digital workflow automatically. Digital is magic. Paper is also magic. One day they will merge, but until then there is value to be had to use both to their strengths.

As we make our information aware,
we increase the potential of our own awareness

Email from User Brad Stephenson

Published by Frode Hegland on July 5, 2019

Brad Stephenson is a Liquid | Author and Flow user who got in touch asking where Liquid | Reader is since he had seen it on the website. I apologised that it had been held back a bit and asked him why he was interested. His reply read like a manifesto for why we are building it so I asked him for permission to post his email and here it is:

Your mention of Reader as a helpful Literature Review tool caught my attention. My understanding, in simple terms: Reader would open PDF documents and allow copying of text within the app, then when pasted (assumedly in Author or another word processor) the bibliographic meta data would be embedded and automatically pasted with the text (I wasn’t exactly sure how it would be displayed and read _ code or text). Additionally the document could be read and highlighted in its entirety then Reader would include a feature which allowed only the highlights from the document to be displayed for review and assessment. This of course would make citing from PDF’s more streamlined and efficient.

A major frustration for me in completing my dissertation was in relation to citation software. My institution began with RefWorks as recommended by the library research assistant. The next year they dropped the contract with RefWorks, and recommended Mendeley, the following year dropping Mendeley and supporting Zotero. What I discovered was citations downloaded in compatible format from the library websites opened in Zotero were unreliably formatted. Instead of making the job easier it made it harder. If I were doing the project again I would probably build a bibliographic database using a spreadsheet, and manually insert footnotes. Your description of Reader indicating an ability to have the citation data embedded with the text which would be easily pasted and referenced for bibliographic or footnote usage.

Brad Stephenson

Stages for what to visualise for LR

Published by Frode Hegland on July 2, 2019

My PhD work is to “analyse current methods of performing a Literature Review and known hypertext opportunities and design specific visually augmented views to augment the student ability better deliver on the University PhD requirements assessment criteria and build and live in such a system to ascertain what is particularly useful for specific use-cases” as agreed with Les Carr and Wendy Hall.

University PhD Literature Review Requirements

The guide to conducting a literature review on the University of Southampton website is clear: “Conducting a literature review enables you to demonstrate your understanding and knowledge of the existing work within your field of research. Doing so allows you to identify any undeveloped areas or unexplored issues within a specific debate or field of study. This, in turn, helps you to clearly and persuasively demonstrate how your own research will address one or more of these gaps.” This guide is without an author itself so it is, a bit ironically, difficult to cite by anything more than it’s web address: http://library.soton.ac.uk/ld.php?content_id=31944998 It follows with specific recommendations on writing your literature review:

“Show the connections between your sources. Remember that your review should be more than merely a list of sources with brief descriptions under each one. You are constructing a narrative. Show clearly how each text has contributed to the current state of the literature, drawing connections between them.”
“Engage critically with your sources. This means not simply describing what they say. You should be evaluating their content: do they make sound arguments? Are there any flaws in the methodology? Are there any relevant themes or issues they have failed to address? You can also compare their relative strengths and weaknesses.”
“Signpost throughout to ensure your reader can follow your narrative. Each time you bring up a new source it should be made obvious to your reader why you are doing this and where the discussion is headed. Keep relating the discussion back to your specific research topic.”
“Make a clear argument. Keep in mind that this is a chance to present your take on a topic. Your literature review showcases your own informed interpretation of a specific area of research. If you have followed the advice given in this guide you will have been careful and selective in choosing your sources. You are in control of how you present them to your reader.”

For all of these points the citation information needs to be available in useful digital form for a visualisation system to be useful beyond a demo and this is why I developed the Visual-Meta system: http://wordpress.liquid.info/printed-meta/ and developed the Liquid | Reader to be able to experience and improve a full workflow: http://www.liquid.info/reader.html

The first bullet point hints about visualisations which can show visual lines between sources. For this aspect I have built the Dynamic View into Liquid | Author, as demonstrated in this video on YouTube viewed over 30,000 times: https://www.youtube.com/watch?v=bCpJTRd0hrE&t=16s and described on the Liquid | Author website, which also lists the (highly positive) user reviews: http://www.liquid.info/author.html

The other points reflect less on a potential visualisations and more on the student’s mental effort of critically engaging and structuring an (linear) argument focused on the research topic. These are aspects traditionally covered by plain text but which hint at visual augmentation.

The point of my PhD is not primarily to augment the student to do the work, but to demonstrate to the examiner that the work as indeed done.

Engage critically with your sources. To show that the student engaged critically with sources could be done by giving the sources link-types, which can be colour coded for example.
Signpost throughout to ensure your reader can follow your narrative. This can also suit the Dynamic View model by allowing the author student to present highlights and directions clearly.
Make a clear argument. An argument is essentially linear but is informed by non-linear sources. This can also be show in an expanding and collapsing Dynamic View model.

In view of this, I am still working on how to best show citation sources in the Dynamic View as blogged under:

Category: PhD