As a consequence I just considered solutions that don’t make me decide which page I want to store and which not. Everything has to be stored automatically for me.

Guard your web entry points

Alternative 1: Guard all your web entry points

I guess everyone has about 5 web entry points where he starts 95% of his web browsing. My web entry points are:

• Bloglines
• Google
• programming.reddit.com
• links shared by colleagues over Skype
• links I get via mail

If all those web entry points are searchable I can track down 95% of any web site I ever visited – granted that I remember how I got to the web site I’m trying to recall (which seems no problem to me).

Of all my web entry points only bloglines has no such thing (I know that google reader has it, if only it could store password protected feeds). Google has google search history, reddit has a – although slow – search (Google’s always an alternative) and it allows me to list all my up/downmodded links (which is quite probably also something I remember). Skype has got a good search (if I remember in which channel the link was posted, which is often not the case) and finally there are enough online mail providers (such as Gmail) and server side mail programs (such as Zimbra) available that offer good search facilities.

Pros/Cons:
Searching for things where you found it on seems very natural to me
you miss the 5%: The link your colleague told you, the news site you check too irregularly to remember, the url you typed in from an advertisement
the refind process is often quite cumbersome

Google browser sync

Use Google browser sync to synchronize
firerox browser history between
your different computers

If your problem is that you use different computers – one at home, one at work – and therefore lose your history every time you switch computers Google browser sync might be a solution. This firefox plugin syncs your browser history, (that means when you start typing into the url field it behaves the same on all synced computers) your bookmarks and optionally your passwords. In the end it really feels like all firefox installations have the same state.
You might scream “privacy!” at this point which is solved quite well for my taste: You can encrypt your browser data right from when it leaves your browser and it is stored encrypted on Google’s servers. They promise they don’t decrypt your data using your password which seems quite fair to me.

Pros/Cons:
it solves recalling links within the last few days (which might be just what you want)
you can’t go back more than the number of days back you set your firefox history settings (which you might set to 999 days). This might not be a big disadvantage as the value of a bookmark vanishes over time: As soon as you forgot that you have bookmarked a certain page this bookmark is of no value to you anyway (I quickly skimmed through 100 of my delicious bookmarks from 2 years ago: I remember only 8% of the pages I bookmarked)
your search in firefox history is restricted to just the page titles – which might be not enough (and too many web sites still have meaningless titles)

Google history

When you mark ‘PageRank and Site Info’
then your browsing history is transmitted
to Google and allows you to use ‘Google history’

When you have google firefox toolbar installed and you have pagerank activated in your options your browser sends every loaded url to a Google server. Then on Google history you have a fulltext search over all the websites you ever visited.

Pros/Cons:
having a fulltext search over all visited pages is pretty neat.
it is a privacy nightmare. Google will use this data to improve their services.
your searches are restricted to the public web. Sites from your intranet or any service you need to login (such as your feed aggregator) are not searchable.
the web site you visited might have changed (such as the reddit frontpage or your local newspaper site) and the web site as you saw it is no longer in Googles indexes
the search is quite slow (compared to google web search)

Google desktop search

“I remember I saw it on programming.reddit.com…”
Google Desktop Search shows a history of all
my visits to programming.reddit.com

Another one from Google: Google Desktop is known for it’s capability to search files on your computer. Apart from this, Google Desktop also stores all the web pages you visit with your browser. That is the content of the page you really looked at, not the page that is in the Google cache. And this is quite an improvement to “Google history”:

1. You can search in an index that contains things you actually saw on that website. No matter how short something was on that website, this is now in your index. You remember you saw something on programming.reddit.com but you didn’t click on it? Search in your local google desktop cache and you have access to the html of all your visits to programming.reddit.com.
2. Your search doesn’t end at the “login screen”: Google desktop holds cache of your corporate wiki and your internal bug tracking system.
3. Once you have it, you’ll never lose it: Documents get deleted from the web or they get moved behind a pay-only wall. You won’t lose this valuable information as it is stored safely in your Google Desktop index.

Google desktop lets you access the data in index in many ways:

• In case you remember some exact words of the web page you’re looking for you can simpy search and order by relevance/date
• In case you remember the date you can browse through the history of that day
• In case you remember how you got to that web page you can search for the referring site and visit the browsing history to find the page this way

And it solves the problem with multiple browsers with the ability to sync you indexes via a Google Server (Windows only, Linux is still lacking that feature, you can ‘vote’ for that feature.)

Pros/Cons:
Gogole Desktop is “Google history” without the disadvantages
The software is proprietary, you don’t really know what’s going on, you need to trust Google (there are alternatives: Beagle (Linux only) and Slogger (just saves your browser history, lacks a search frontend)
Your history data is locked in. The cache lives in a binary file on your harddisk

That said I still find Google Desktop very neat. Although I just have tested it for two weeks it greatly served me for accessing web documents more quickly and without the saving effort needed on online bookmarking services.

Address side of the gift. It is remarkable
that this mail got delivered, as it wan’t
wrapped in an envelope or so!

When Julian Cash told me in an email that he was very gateful for the paper Frank, Grigory and I wrote, he asked me for my snail mail address. I wondered what he wanted my address for, then one or two weeks later I found a block of plexiglas in my mail.

Thanks a lot to Julian for showing such appreciation. It showed me that my labour has implications to real life too. Holding a physical gift in my hand, coming from work done in a virtual blogosphere feels special.

Btw: He also thanked Frank for his part by sending him an inflated ball.

What I’ve got a problem with is the idea that one way of organization is going to make everyone happy. This will never be the case, and, as with many other Web challenges a compromise will need to be reached. Unfortunately, the nature of the site map deliverable, and the process behind it, don’t really lend themselves to compromise or consensus. This creates a bloated process and quite a bit of effort that is, ultimately, wasted if it doesn’t help people find what they are looking for.

The comments are also very interesting and thought-provoking.
I so often was annoyed by hierarchical built sites where the keywords seemed to obscure to get the information I wanted. When this happens, I go first for a search and second for a sitemap.

Open Source Paradigm Shift by Tim O’Reilly — This article is based on a talk that I first gave at Warburg-Pincus’ annual technology conference in May of 2003. Since then, I have delivered versions of the talk more than twenty times, at locations ranging from the O’Reilly Open Source Convention, the UK Unix User’s Group, Microsoft Research in the UK, IBM Hursley, British Telecom, Red Hat’s internal “all-hands” meeting, and BEA’s eWorld conference. I finally wrote it down as an article for an upcoming book on open source,”Perspectives on Free and Open Source Software,” edited by J. Feller, B. Fitzgerald, S. Hissam, and K. R. Lakhani and to be published by MIT Press in 2005.