Transcript from 11:10
Gareth Mitchell: Well, advances in technology now. Virtual reality, perhaps? Telepresence holographic phone calls, maybe? Well, actually I’m talking about text. There’s so much going on in the future of text. There’s an annual symposium devoted to it. And recently, at this year’s meeting, there was talk of how to make text 3D. Well kind of. Now, to be honest, our vision system evolved for hunting and avoiding predators on the savannah. So, we’re not exactly that well designed for processing complex documents and books. But Frode Hegland, founder of the “Augmented Text Company” has a new authoring tool called “Author” and a PDF reader called, well, guess what? “Reader.” And, we’ve been speaking. Also, joining us was Vint Cerf, a pioneer of the early internet, and now “Google’s” chief internet evangelist. We began by discussing Frode’s simple, but actually elaborate innovation of putting metadata at the end of a document.
Vint Cerf: First observation is that the material that we put at the end of the PDF is, not only readable, but it is structured. Which means that a computer can understand what it is. And, this representation of metadata about the document is really important. The concept that the object knows about itself. And I want to extend this notion to, beyond simple text in the PDF form, but any other object, digital object could know about itself, in similar kinds of ways. So this is extremely powerful, but the other thing that’s important is that, even though it’s structured, it’s also readable by human beings. Which means that, in the far future, even if all the software that normally was used to represent or present a PDF has gone away, it’s possible to extract that kind of information fairly readily, because of its simplicity. So, I want to over emphasize how powerful this simple and extensible idea is. You can add anything that you want, really, to this metadata section. Or what Frode calls, “Visual-Meta.” And so, you can invent new meaning by adding new terms into this material that is at the end of the document.
Gareth Mitchell: Yes. Because I know that, both of you want to make this timeless, don’t you? So, a document that we create today could still be either human-readable, or machine-readable, or both within a thousand years. So, simplifying the way that that crucial metadata is incorporated, that’s a big deal.
Vint Cerf: It is a big deal. And if you imagine for a moment that this method is adopted widely, then you can imagine that, even in the far future, one could use this capability to reconstruct, not just a bibliography, but a kind of encyclopaedia of content, linking all these various things together. Literally reconstructing those interconnections if, for some reason, that data has gone away.
Gareth Mitchell: One innovation I found really clever is the way that text can be cited. And as an example, let’s say Frode’s PhD thesis when it’s ready. I might just copy a piece of that thesis from the PDF. And then I paste it onto another document. Now we might expect just the text to be pasted. But what also comes with it, automatically, Frode, is the metadata. So, tell me about that other data, that automatically comes with me, when I copy and then paste something from your thesis.
Frode Hegland: It’s really, quite exciting because PhD is really boring. A lot of the time there is a lot of fiddling with moving things about. Any academic work is the same. So, what happens there with the metadata is that it notes down all the stuff that you need to find the document, such as in the olden paper dates, you would have the title, the name of the author, and so on. It also includes a link, if there was a link, or a DOI, which is an academic way to search for a document. It can also include the original text and the location in the document. It also has a unique ID for that document. And that means, for the end-user, that when you’re reading a PDF, that’s been produced in this way, two interesting things can happen. First of all, you can see automatically what has been cited in a citation tree. That’s not actually possible outside of this system, which is really weird. The other thing you can do is, to click on that citation, in the document, and up comes this reference information. So you don’t have to jump to the references, and back and forth. And further or more, what we’re working on is, that if you already have the cited document on your computer, you should be able to click to open it, without having to go to a download site. And not just open the document, but open it to the page where it was cited, making this kind of a “Docuverse,” rather than something that relies on web servers that, as we all know, links can be brittle and servers can go down.
Vint Cerf: A way to think about what Frode is doing. We know that typical text is linear. It’s flat. It’s planar. And even books, although they are three-dimensional things and when we bind the pages together in a three-dimensional space, they still are typically consumed in a linear way, except that you can open the book to any page. And we have helpful mechanisms like the table of contents, and the index to help us to consume the text in other than purely linear ways. The interesting thing about this online environment, of course, is that you get to jump in a third dimension, from one place to another, literally with the click of a mouse. And is this extra dimensionality, I think that, makes what Frode has done so important because it gives us a different way of both producing, and consuming content.
Gareth Mitchell: Finally, then. Why does this matter to anybody who isn’t an author, or an academic?
Frode Hegland: This is for people who really care about their textual knowledge. Some people use advanced software like “Rome” and “Notion.” This is not that level of sophistication in terms of a learning curve, but there is a bit of a learning curve. We much rather you think of this as a racing bike, rather than a tricycle. There aren’t many buttons on the screen to help you click, click, and do simple things. It’s more about learning how you interact with it, in a richer way.
Gareth Mitchell: That was Brewster Kahle who says, “it’s about getting the hang of it,” Bill. And I’ll be honest, it took me a good while to get my head around all this. And I think to explain to the listeners, I would say, it gives you a way of holding a whole document in your head. I think a lot easier than maybe just by trying to scroll through a long PDF.
Bill Thompson: It does. You clearly had a lot to hold on your head, because that wasn’t Brewster Kahle, that was Frode Hegland talking. But I can agree with you that, what’s happening, what Frode and Vint are describing is, in fact, it’s one of those things where maybe, it doesn’t work as well on the radio. Very few things don’t work as well on the radio. Because it’s experiencing the documents in “Reader.” It’s seeing what it does to an annotated PDF that really makes the difference. Because you start to get a sense of, I would call it, the living document. I’m one of those people who use it, I write a lot, and I write a lot of structured text. And I’m very conscious that there is nothing to describe the complexity of the document I’m writing, other than the words. And I have no tools to allow me to navigate it efficiently. And there are some very sophisticated tools out there. And what I like about the way that Frode has thought about this over the years, because he’s been doing this a long time, is this attempt to engage with text, to make it work for people and, and to make it significant. And I think it’s something which it’s worth exploring more. And I know we’ll come back to.
Gareth Mitchell: To, I’m sure we will. And there’s more from, for Frode and Vint in the podcast version of this program.
Transcript from 11:10 (podcast addition)
Gareth Mitchell: Would you like to hear a bit more from Frode Hegland and Vint Cerf by any chance, Bill? Because we’ve got some anyway, but okay. Will you welcome it if we play?
Bill Thompson: Yeah. Yeah. I would welcome it, Gareth. In fact, I’ll even listen to it. How about that?
Gareth Mitchell: There we are. So yeah, we had a fascinating conversation with them. And Bill will know this feeling, when you get too super broad people together, the conversation can just flow. And then we just…
Bill Thompson: Like this podcast every week, Gareth. It is you, me, Glenn, Angelica. It’s just…
Gareth Mitchell: Flowing. High-level intelligent conversation. Conversations you don’t want to end. And it was like that with Vint and Frode. So, we recorded way more than we needed. And so, it clipped out about four and a half other minutes that didn’t make it into the program. A bit more about what else this idea of “Visual-Meta” can achieve. Bigger picture stuff. But also, there was a bit at the end of the broadcast interview where I said, “well, you know, why does this matter, anyway?” And actually, that was a retake of a previous version of me asking that question. They gave a longer answer that was just too long to use in the broadcast bit. But we thought it’d be nice for the podcast. So, in case you think I’m asking the same question twice. As we all know Bill, how it goes in podcasting and broadcasting, you often have a few stabs at the same question, don’t you? See what comes back. Yes, good. Just thought I’d explain. Right. Anyway, here is a bit more of that conversation
Frode Hegland: In terms of the really big picture, they say that the most fundamental thing in the universe is information. Not matter. I would take it further. I would say that the most fundamental thing in the universe is interaction because, without interaction, you can’t even have information. And if you then scale that up. That means that, exactly what you’re saying, Vint, we have large volumes of texts that are important. And they are presented in dead columns. So, we have to do everything we can in order to be able to extract meaning. And that means things, like using the headings we talked about earlier, it also means automatically seeing all the names on the document. Or only seeing the defined concepts. Because then, you can very easily, through these interactions, develop a better understanding of the intent of the author.
Vint Cerf: So, there is an interesting scaling problem here. Which we also see in the existing World Wide Web. Everyone, presumably, is well aware that the hyperlinking in that design goes one way. It goes from wherever you are, to a target. The target doesn’t necessarily know about you and where you came from. Actually, that’s technically not perfectly correct because, sometimes, the software knows where it came from, but it doesn’t remember that having two-way interconnections is pretty interesting. But, of course, it explodes in terms of the number of possibilities. Imagine a million people reading one book, and the book keeps track of the fact that a million people read it, and made comments about it, and so on. And now you encounter this decorated thing with comments from a million people and you’d be overwhelmed. So, we would need to look for ways of allowing you to filter the backlinks. For example, I would like to see what Frode thought about the book that he read. Or for that matter, if we had it available, what did Einstein think.Of course, it’s a bit late for that. So, the possibility of having these backlinks be part of the picture is an interesting one, from my point of view. But it has the problem of coping with scale.
Gareth Mitchell: It’s great to hear about all this innovation in the future of text. But bluntly, why does this matter to people who, for instance, are not authors or academics?
Frode Hegland: Two things to answer that question. The first one is, there is a lot of removal of what I would call clerical stuff. For instance, exporting with a table of contents, and a reference section, all of that is done automatically in this system. You just export your documents. When it comes to reading, to go through every word, every sentence in a PDF you’re reading for knowledge, takes a lot of time. With what we’re developing with these interactions, allow you to choose how to view the document. And we’re only going to keep experimenting with how we can make more ways for you to see your text. So, you can learn, and understand not superficially, but deeper.
Vint Cerf: I’m sitting here thinking about the editing of this discussion, and realizing that, because of the way that we are repeating, for example, ourselves or restating things. It’s going to require a two-pass edit because you actually have to be able to discover the repeat before you can figure out that there’s an edit to be done. I know, it has nothing to do with the conversation we’re having, I’m just thinking about this particular medium, and manipulating, and working with it has some fascinating properties. And it would be worth a discussion someday to expand our thinking about other media and how they can be approached. Some of you may have seen some of these 360 to degree video presentations, where you literally have multiple cameras, and you can view the film using either a virtual reality headset or even simply a mobile. But the mobile detects the fact that you’ve moved, and changed your orientation, and shows you something different. Because the video is actually a 360-degree thing. And as you move your mobile around, or as you move your VR headset around, you see something different. The editing of that is absolutely amazing. Seeing what Frode has done with simple, linear text, I just wonder whether we’re on the edge of new concepts of dealing with complex media.
Gareth Mitchell: All right, folks. So, there you go. That’s a little bit more of Frode and Vint.