Rajesh Jain: Constructing the memex

Can we leverage all recent tech developments to build the memory extender?

In the past decade, the internet has extended our world by making accessible a vast quantity of information that was unimaginable earlier in our lives.

It started in the early days with bulletin boards and newsgroups, pooling together a collection of documents on a single server. Then the ease of hyperlinking combined with directories and search engines made the physical location of information irrelevant.

If it was out there on the internet, it could, in theory, be found. For many of us, our first internet website memories are probably linked to Yahoo. Navigating through its hierarchy of categories or doing a search helped us get to what we were looking for.

Altavista and Excite started providing search within pages, allowing us to type a word or a phrase and know that there were tens of thousands of matching documents.

Google then came along and refined the process to perfection by using its PageRank algorithm, giving us results very relevant to what we were looking for. In effect, Google became our other memory.

In its efforts to provide uniformity and consistency, Google has become a mass-market search utility. But it is not good enough.

What is missing is the context that each of us have "� this is embedded in the web we browse, the documents we chose to save (or email to ourselves), and the subject-matter experts we know (or would like to know).

To begin thinking more deeply about this, we need to go back in time and learn more about a person called Vannevar Bush. According to Randall Packer and Ken Jordan [writing in their book

"Multimedia: From Wagner to Reality"], Vannevar Bush "rose to prominence during World War II as chief scientific advisor to Franklin Roosevelt and director of the government's Office of Scientific Research and Development, where he supervised the research that led to the creation of the atomic bomb and other military technologies. His contribution to the evolution of the computer ranges far and wide: starting from the invention in 1930 of the Differential Analyzer, one of the first automatic electronic computers."

In 1945, Bush published a paper "As We May Think", outlining a protypical hypermedia machine. He called this mythical machine the Memex "� meaning a Memory Extender.

Adam Brates wrote in his book "Technomanifestos: Visions from the Information Revolutionaries": "Bush's immense administrative burden "� the daily strain of sorting, allocating, researching, analysing, synthesizing, crosslinking, and filing "� spurred his idea for an invention that would perform this work for people.

Bush popularised the idea that machines could solve the problem of information overload. Bush wondered whether all the sprigs of scientific wisdom, if not somehow preserved, would fall from the tree of knowledge.

Information must somehow by connected to be relevant, lest it should become forgotten. Knowledge accumulated and stored in massive filing cabinets under lock and key would languish. An idea developed today might not be relevant until some point in the future.

What happens, though, if it is forgotten? Application of all new knowledge would require some means of keeping it available, accessible, and relevant."

Vannevar Bush wrote in his paper: "[The human mind] operates by association. With one item in its grasp, it snaps instantly to the next that is suggested by the association of thoughts, in accordance with some intricate web of trails carried by the cells of the brain. It has other characteristics, of course; trails that are not frequently followed are prone to fade, items are not fully permanent, memory is transitory. Yet the speed of action, the intricacy of trails, the detail of mental pictures, is awe-inspiring beyond all else in nature...[Associative indexing is] the basic idea of which is a provision whereby any item may be caused at will to select immediately and automatically another. This is the essential feature of the memex."

Vannevar Bush wrote his memex essay in 1945 "� before we had the computer, the internet, the web, Yahoo and Google. Even today, we struggle with information overload. The memex could be the solution, the silver bullet .

So, the challenge before us is: can we leverage all the recent developments in technology to construct the memex? Two interesting recent developments need to be connected together.

Search engines like Google allow us to search billions of documents in a fraction of a second, while weblogs have made publishing very simple with the result that there are millions of people writing regular journals on the web. Both are significant in their own right, as a combination they herald something much more profound in the information space.

(This will be continued in the next FutureTech column).

Rajesh Jain is managing director of Netcore Solutions Pvt Ltd. His weblog is http/www.emergic.org. He can be contacted at rajesh@netcore.co.in