Clarification

According to comments here and on reddit, it was obvious that my intention of this post was somehow misunderstood – partly because of the original misleading title (was: “.. – 5 reasons why social bookmarking doesn’t work”). Maybe these adaptions from an xkcd comic does clarify:

Right tool: Use bookmarks to get things done

I think, derefr sums this up very nice:

I find a GTD approach works well: what next action are you going to apply to this bookmark? If it’s just “well, it was neat!” you have no reason to save it (perhaps share it, but not save it), and can throw it away.

The same goes for using the tag “mycomment” to follow up discussions you’ve partaken or “toread” to know what to read once you’ve got some free time. These bookmarks all serve a purpose that is clear to you while bookmarking. This also helps you picking an appropriate tag. No critique on that one.

Right tool: Sharing links

It is clear that bookmark sharing sites such as reddit, Digg, or Stumbleupon that all focus on link sharing have proven that this concept works. Delicious, Simpy, Magnolia et al. all have features to help you share your bookmarks. No critique on that one.

Wrong tool: Remembering potentially interesting links

This is what this article is dealing about: Saving bookmarks that are not useful to you now but – without yet knowing what you’ll use this bookmark for – you save it because it is potentially interesting in the future. I think that doesn’t work and the 5 points should prove that.

Reason 1: You can’t foresee the future

Deciding which web site will be valuable in the future is a very very hard task. I’m not too good at it. I pile up tons of bookmarks I never look at afterwards and on the other hand I decided to not bookmark sites which I needed afterwards. In fact I’m so unsure about my ability to bookmark the right pages I often don’t try searching for a link in my pile of bookmarks but instead google first because I expect being faster this way. Too often I searched my bookmarks altering tags and search terms and didn’t find the bookmark in the end.

Additionally: Even if I would know which links will be of interest in the future, I can’t decide how I should tag (categorize) my bookmarks. When I tag an article, I normally have skimmed it and while categorizing I look at its title. When I tag I’m in a completely different situation – information wise – from when I search for the link.

Your categories may change when you get
familiar with a product or topic

Your information level when looking at a document
differs from when trying to recall that document

Reason 2: You tear links out of its context

Bookmarking is like cutting passages
from books: you remove information
from the context you originally found it

The word “bookmark” relates to the pretty carton markers you use when reading books. Although the way it is used in the web is far far from what it means in books lets delve into that comparison a bit:
To go sure you will be able to find an important passage once you finished a book, you underline or write a few words into the margin to outline a paragraph. Then, when you recall that great sentence you most certainly know in which book it was written (unless that book is a conglomeration of quotes). Then, you often can remember the way that statement was used in the argumentation and in what topic it was embedded. And finally, amazingly, your brain often tells you where on a page (e.g. bottom left) the searched sentence is written. So you normally get quite a bunch of context information to guide you in your search and you will find the wanted sentence within a short amount of time, even if it wasn’t underlined. And even if you don’t find it, you often have a good time reading through the other amazing statements and end up quoting something you didn’t intend.

The way bookmarks are handled in the web would mean to books that you tear out that sentence out of the book, stick a few colored post-its to it and throw that snippet onto the pile with the 1325 other quotes. Bookmarking means taking information out of the context you originally found the information in. On the web context means how you found that link: Was it on Google or in your feed aggregator? Was it a blog post of one of your colleagues? Was it in an email? I often remember these things. Without being a psychologist or having an education in these things I guess our brain is pretty good in remembering context. So why don’t we use techniques that help our brain instead of trying to replace it?

Reason 3: It takes too much time

Bookmarking should save you time – and frustration. Leaving out the frustration bit: Does it really save you time?
Lets say it takes 10 seconds to categorize a bookmark and lets say you’ll use every 20th of your saved bookmarks (which are rather optimistic guesses). That means that when trying to recall an url from your bookmarking service you need to be 200 seconds faster than when you didn’t bookmark any pages at all (as it took you 200 seconds for bookmarking the 20 bookmarks out of which you used 1).

I’m pretty sure you won’t save over 3 minutes in average searching in your pile of bookmarks compared to thinking for halve a minute where you found that link and then going down that trail. So: Why the hassle?

Reason 4: It didn’t work for me

I tried it. I gathered 3444 bookmarks in 2 years using 3034 tags. I asked myself how I could change my tagging practices to improve the recall. I failed. I gave up. I cannot believe there’s no one out there feeling the same.

I stopped bookmarking nearly two months ago. First, when reading articles that felt so interesting it was hard to not bookmark them. Then, it was kind of liberating not having to think “is this page valuable in the future?” “what tags should I use?”.

I never missed it. I always found that link. I don’t regret.

Reason 5: Social bookmarking won’t improve that soon

You may argue that there soon will be techniques to overcome the problems I just mentioned. But my claim is that social bookmarking sites won’t improve that soon.

In my last post I asked: “Why is tagging stuck?”. Gene Smith argues correctly that tagging isn’t stuck. He continues:

Want to know what is stuck? Del.icio.us

The same is true for all the other social bookmarking sites. RawSugar did a brilliant next step (before it went offline) but the social bookmarking market is quiet ever since. I couldn’t find fresh ideas in delicious’ current redesign. It seems like they moved buttons from here to there. I hoped they wouldn’t just redesign the appearance but would also change the way users interact with their data.

So, I guess these services are just as good as it gets. No improvements to wait for. That means it’s our – the users – turn to change our habits, to find the right tool for the job.

Paolo Valdemarin:

4 years later I’m still wondering when will we get some truly advanced tagging tools.
Where are all these tools to manage all my tags (on Flickr, on del.icio.us, on technorati, in my RSS reader, on my blog, etc), to help me organizing them, to allow me to gain more advantages from tagging? (maybe they are somewhere and I simply have not found them yet…)

Matt Mower:

I have been surprised, that [...] the state of the art in tagging seems firmly wedged in 2003. Surprised because there seemed [...] to be a momentum building in the use of tagging

David Weinberger:

Tagging like it was 2002

Thomas Vander Wal:

In the consumer space thing have been stagnant for a while, but in the enterprise space there is some good forward movement and some innovation taking place
[...]
While there are examples that tagging services have moved forward, there is so much more room to advance and improve. As people’s own collection of tagged pages and objects have grown the tools are needed to better refind them.

Vander Wals post is very very insightful and worth a read: He sums up the tagging history and expresses a few brilliant ideas how to proceed.

The delicious lesson – revisited

The big question remains: Why is tagging stuck?

My suggestion is that we may rethink the delicious lesson: Not in terms of “is it true that personal value precedes network value?” but in terms of “what is the real benefit of the users?” or in other words: “How can we design the itch that causes users to generate valuable metadata?”

Recently I talked with Cédric Huesler, a coworker of mine about his use of del.icio.us: Instead of using delicious for storing his bookmarks for later retrieval he stores them to exchange links with strangers. Indeed he has 19 regular consumers of his bookmarks, 7 of these users he is consumer as well.

He doesn’t store his personal bookmarks at all. He can recall from memory where or how he found a certain website and goes back to his google history.

There are just a few entry points into new information on the web: there is Google, feed aggregators or frontpage sites. When there are good search utilities in those tools who needs bookmarks? I must confess that searching at those entry points feels more natural to me than remembering the exact tag I used.

Let’s put it straight: Using tags to find my bookmarks later just doesn’t work. I give up. And no, it’s not just the lack of good tools that help me going through my bookmarks to reorganize them. I won’t do that for all my 3444 bookmarks. And no, this won’t be solved with better tools to refind my items. What do you want to throw into the mix? Fulltext search and time based drill-down? This has nothing to do with tags.

So, we might have to rephrase the users motivation to tag, as I don’t think Joshua Porter was right when he wrote:

in order to gain more personal value, they use tags to be able to find their bookmarks later

I’m not yet at the point where I could correctly rephrase that statement, but I think Cédrics approach in using tags not for personal recall but for publishing is worth a thought. The value therein is close to the value of blogging: You get attention and you communicate. And that’s what the web is about, isn’t it?

That’s the feeling that arises when I think of all the bookmark services out there that ought to file urls I somehow find notworthy so that I can quickly recall them afterwards: There’s too much clutter and the services are just too slow. Therefore I again begin to save my bookmarks at other places: In firefox or in some text documents lying somewhere on my hard drive (probably I should tag them?).

«I want it now»

As for myself, if I don’t get my data within a fraction of a second, I don’t feel that I have good access to my data. I think one of the best things of Google Web search is that it is incredible fast. Tag services are not. That’s a pity because within tag services I search my lowly amount of bookmarks (around 2000 bookmarks at that moment) whereas with Google I search the whole net. I often find myself at the situation searching for a url on Google rather than looking it up on my bookmark service. Even if I have to do 3 Google searches – I feel at control. If I have to wait for 3 seconds for an answer, that’s simply too much.

An experiment: I ran for ((i=0;i<10;i++)) do wget -q --user-agent="Mozille Firefox 1.9" $url; done; for the three big players in tagging services: 181del.icio.us, 182Simpy and 183Rawsugar. For $url I chose a page displaying all bookmarks saved with a certain tag. I also ran a query on Google for that “tag”. The numbers are number of seconds per query. The queries were run on different hosts with different internet connection, all in Switzerland.

I’m happy to see that with Rawsugar I’m with the fastest bookmark service (2 times faster than Delicious, 3 times faster than Simpy). On the other hand: all the bookmark services are at least 4 times slower than Google. I know that Google has set a high mark – but emotionally that’s the response time I’d like to have when querying for my data.

«I just want my bookmarks»

Have a look at the “result areas” (highlighted in green) of the three bookmark services.

Delicious: Result area in green

Simpy: Result area in green

Rawsugar: Result area in green

To come back to the comparison with the closet: Even if there are 20 magic hands with documents sticking out of the closet, it is crucial that the most important document is the one closest to my face. That means, when using those bookmark services, my eye should first notice the most important link. My opinion: The service that solves this best is Delicious. The results appear on the leftmost part of my screen and the result area covers 64% of the screen. Regarding that matter Rawsugar is far worse: My eye has to search for the first result. I find it natural to start reading at the left of the screen. But at Rawsugar there are so many links in the head of the page and then the left column helps me to refine the result – a great feature but at the wrong place – furthermore: if I want just get a link I know I filed under “important”, I don’t need this at all. Even Delicious’ “saved by xxx other people” – shaped in different colorings – is too much clutter for me. Maybe I’m a puritan but – what the heck – I just want my 9 bookmarks!

Beyond the criticism

Sorry to only criticize. I like those services a lot. They are helping organizing links and they are for free. To turn this post into a constructive comment, two possible solutions to the problems mentioned.

Bookmark services offer a «Minimal mode»

Yes, tagging is about collaboration. But I fear the personal value of tagging is too small – people might leave tagging services because they don’t feel their bookmark problem solved. There’s too much network and too many features. I propose a “minimal mode”, a result page that just shows select * from bookmarks where user="phred" and one_of_its_tags="important". In firefox there are 184Smart Keywords also known as “quick search”. I set up “myr” to search for my rawsugar bookmarks. Typing “myr important” into my location bar I end up on www.rawsugar.com/links/phred/important. If there would be a site on rawsugar that displays the same results just in this minimal mode, I would bend my smart keyword to this result page and would be happy.

Someone codes a «Tag agent»

I imagine a “tag agent” that has incredible response time and no clutter. It could either get the results via rss/api from the bookmark service or it could hold all my tagged bookmarks in it’s own database. Such an agent could be installed on a password-protected part of my website so I can access it from wherever I am. The problem is: I don’t have time to write an application my own. I have thought about such an application, I’ve got plenty of ideas how it should look but I’m afraid I won’t find time to code such an application. I repeatedly get emails from people writing yet another thesis on collaborative categorisation. Instead of writing about insights on the mental actions taking place while tagging – do the tagging world a favour and write such an agent. If there’s such a tag agent already, please leave a comment.

185186187188

After the presentation at the tagging workshop at www2006 we felt the need to give our paper a more www-friendly, I-don’t-want-to-read-through-those-theoretical-equation-flooded-papers face.

So, here you go: Automated Tag Clustering: Improving search and exploration in the tag space. To read this document you should have a clue what tags are about, you should also know some tag services as delicious or flickr so you can understand the limitations these services currently have. If you don’t want to read through the whole papers, the numerous figures give you a good summary. Finally, to wet your appetite, here a few excerpts of the document:

Currently tagging services still provide a relatively marginal value for information discovery and we claim that with the use of clustering techniques this can be greatly improved [from introduction]

The whole promise of collaborative tagging is that by exploring the tag space you can discover a lot of useful information you would not find with traditional search engines. When your information need is not well defined, the idea that you can explore and see what other people tagged with certain tags is very attractive. We believe that tagging will be able to reach a very wide audience only when exploration techniques will be effective. [from limited exploration]

Although a great visualization paradigm, we believe that with today’s tagclouds it is hard to find more than one or two tags to click on. Tags are not grouped, there is too much information, so that you find lot of related tags scattered on the tag cloud. One or two popular topics and all their related tags tend to dominate the whole cloud. For example, looking at the del.icio.us tagcloud, one would mostly see tags related to web design and technologies. This is because these topics are overwhelmingly more frequent than anything else. [from limited exploration]

Tag web2.0 nowadays is so popular and is combined wildly with anything. In fact this tag is so overused that if you look at tag bookmarks in the del.icio.us dataset, the most used cotag is web2.0[...]. Basing tag similarity on these numbers often doesn’t make sense at all. The similarity measure should be chosen so the popularity of a tag doesn’t affect the set of a tags related tags. Don’t cut the long tail. The success of blogs is driven by the importance of the long tail. We all know that it is crucial to support the niches. Tagging applications should empower the long tail too. If you just sort by popularity, you’d loose all those niches. [from choosing a similarity measure]

We’d be happy to get any kind of feedback on the article. Just post a comment to this blog post.

Edit (4 years later!): A few guys asked me about the source code: Source code with syntax highlighting, download.
You need kmetis to make this run, see usage() to see how it should be used.

Locked in?

I recently tried out Simpy and RawSugar and thought about switching to one of them.
It is true that you are not really locked in to del.icio.us. You can export all of your bookmarks by just typing
curl --user username:password -o backup/delicous_backup.xml -O 'http://del.icio.us/api/posts/all'
But in a way you just feel like you’re locked in, isn’t it?
Why so?

• del.icio.us has the most folk using it, it feels good to be with the masses (the mass can’t be wrong, can it?)
• there’s more feedback while tagging
• could I switch back to delicious when this other service turns out not to meet my needs? Well, I would have to find out…
• heck, I think I am addicted to del.icio.us. Sure I could change but will my life go on? Will my “last bookmark list” still work? And ah, I’ve got references to del.icio.us at 13 points of my blog and then I would have to tell my friends that I switched and then my to:mybestfriend wouldn’t work any more and.. ah, why should I?

Well, guess what? You are locked in! :-)
I think the point is that the advantages and disadvantages of the bookmark services aren’t known. I haven’t read much about it, have you?
I recently switched to RawSugar, after using del.icio.us for about 1 year. I have a little bit of experience using Simpy while using it for a little project. I will therefore confine myself to describe just this three services, if you have experiences with other services I would appreciate your comment.

del.icio.us

Let’s start with the one you’re most probably using.
Advantages:

• Clean, simple user interface
• Most data, therefore
  • best popular page
  • you see how popular an url is while tagging
  • much feedback from other users while tagging

Disadvantages:

• slow roll out of new features
• bad performance
• spam
• I guess my dad couldn’t use it

Joshua and his team are trying hard to limit del.icio.us to what it should do: Saving bookmarks and finding them:

“delicious is a tool, not a community. One reason i donut want it to be a community. some community behaviors are not good.” the “you suck” effect, and flame wars [Joshua Schachter at Carson Summit]

In fact del.icio.us has got the cleanest ui of all bookmark managers I’ve seen. On the other hand, I can’t imagine my dad using it. The whole terminology and the design are not the type of web page my dad would use. But then again this is an advantage. The fact that del.icio.us doesn’t want to please all users makes it simple for the ones it is made for.
The other disadvantages heavily depend on each other: When Joshua started his service he most probably took a few mysql tables and optimized his queries, when the demand started to grow he’s put in some cache here and there but then the big wave hit: in the last half year, the number of bookmark posts per day tripled. I guess he now is working on improving the performance rather than rolling out new features, as

“tags doesn’t map to sql at all. so use partial indexing.”[Joshua Schachter at Carson Summit]

The succeeding bookmarking service programmers already knew the problems of tagging applications before they started, Joshua and his team have to improve a running service. And then they switched to Yahoo’s location, that probably also helps slowing down the roll out of new features. The “private bookmark feature” is planned for about a year already, yet it is still to be released.

RawSugar

RawSugar is a bookmarking service run by a small company with a handful of programmers. It became beta in September of 2005.
Advantages

• Hierarchic tags
• multi-word tags separated by comma
• fast service
• fast roll out of new features
• private bookmarks

Disadvantages:

• not that much data
• different terminology than what you’re probably used to

The fact that RawSugar started their service after the success of del.icio.us helped them to design their engine for big amount of data. They definitively have a good engine, As they never put public information about their engine, I’m not sure if I’m allowed to speak about details here, but they don’t just have a 3-table-mysql-schema :-) In fact I believe their engine would handle a wave of new users without stopping roll out of new features.

They have got hierarchic tags, and this is a very clever idea. First I wasn’t totally convinced about this feature but it has itself proved to be very helpful. I don’t like del.icio.us’ tag bundles. Tag bundles may be a good thing if you have 100 to 200 tags, but if you have 500 or 1000 tags (I currently have over 2000), it isn’t manageable any more. With RawSugar you can put a hierarchy in your tags while tagging an item. if you type “article>rant” you say that “rant” is a child tag to “article”. If you later tag a site with “rant”, the tag “article” will automatically be added.

A word about their terminology: At first it was difficult to map between the “delicious terms” and “rawsugar terms”. Sure, tags are still tags but somehow I was (too much) used to the whole terminology of delicious and took it as granted. On the other hand it’s good to see new viewpoints and that’s why I think new players in the tag market are a good thing. RawSugar aims to be not only a service for bloggers and geeks. Their UI looks more “traditional”  like the one of Amazon, for instance: After you have searched your bookmarks after a certain tag you see a search field where you can type in another name of a tag and then hit “refine”. I think this is more “natural” than clicking the “+” signs at del.icio.us.

Finally they have a bunch of features:

• Nice integration to your blog (tag your articles and then simply add their automatic generated navigation to your blog)
• you can switch users while tagging (means: you can have different “directories”: one for your blog, one for your company, etc..)
• they are playing with automated clustering techniques (similar tags and clusters), I’m looking forward into the integration of this algorithms!

Simpy

Simpy is a one man project of Otis Gospodnetic. As he is/was a developer of Lucene he has a good knowledge of database systems and scaling. He has got a good engine too so I guess his service would still stand after a big wave of new users. Simpy seems to be online since early 2004 (according to the date of Otis’ earliest links).
Advantages:

• group tagging
• search results can be sorted by date, popularity or relevance
• it’s possible to search by site (i.e. www.nytimes.com) and/or extension (i.e. mp3) and/or combined with tags
• multi-word tags separated by comma
• not commercial (free time project)
• private bookmarks

Disadvantages:

• one man project, therefore not many resources to put in
• UI is a bit cluttered

What I particularly like about Simpy is it’s spirit: Otis is “one of us” giving away his spare time for this service and demanding nothing in return. In return, on Simpy’s developer mailing list he gets a lot of feedback from people that want to contribute, that is: write little applications, that use Simpy’s API. I am in contact with Otis: he is always very friendly and open for feedback. That said, he knows that his UI lacks some polish :-)
I’d say the killer-feature of Simpy is group tagging. While tagging you can click the checkbox of one of the groups you’re in. There’s an RSS-Feed of each group so this is a very useful feature. Ever tried to do that with del.icio.us? You have to log out and log in into the group account..

Bottom line

There are many many more bookmark manager services that I never actually used (I probably have signed up to about 50% to them just to get my favorite username in case they turn to get “teh r0×0r”), there’s a good list at listible.

In the end, there’s no real winner. And I like it that way. I suppose in long term each bookmark service will have it’s community.
The popular page on del.icio.us is ruled by entries about design and web2.0. I guess other services will attract other communities.
That said, I envisage two things:

1. a service that asks you about your hobbies and topics of interest or let’s you submit a attention.xml file and then tells you which service fits best for your needs.
2. an “interchange format” that helps you switch from one bookmark service to another, ore more generally: more collaboration between the bookmark manager services. Update: Faber of smarking just started a project to design a interchange format based on microformats, he’s set up a mailing list. Great effort!

Well, I highly encourage you to look beyond your own nose because there’s a big world beyond delicious. :-)

Further reading

• social bookmarks review: most complete social bookmark manager comparison chart I’ve seen
• all social that can bookmark: pretty complete list of bookmark services with their features explained at a glance. Pretty neat.
• Why simpy over delicious
• blinklist vs delicious
• Magnolia: delicious killer?
• poll: what is your preferred bookmark manager?

Statistics is a broad mathematical discipline which studies ways to collect, summarize and draw conclusions from data. [Wikipedia]

Statistics help us to draw conclusions from data. In a way this whole tagging thing just popped up and now we are trying to figure out what really is happening. I think statistics can help us to understand tags.

When I did set up my performance test system I wanted to know the metrics of delicious so I did try to extrapolate some hand collected data but it didn’t turn out that well.

After that I started collecting post data from del.icio.us and am happy to announce that I’ve set up a site with delicious statistics that is fully automated (my hands can rest now..). There are trends about number of posts per day as well as numbers of tags per post.

Overall bookmark post trend

The stats are based on data I extract from the most recent posts feed, which I’m grabbing 6 times an hour (I’m trying to not be evil: No screen scraping, no grabbing each minute.) I miss a big portion of the posts (actually I record just about 10% of the data) but I guess the stats are precice enough to draw some conclusions.

Why statistics?

I’m fond of del.icio.us (as you may know) and when I’m fond of a website I urge to know how many people are using it, if the service is attracting or scaring away folk, I feel a need to know what’s up. Especially after delicious has been acquired by Yahoo, you may ask “do people stay?”.

Anyway, that’s not the only cause for stats. When I set up the performance tests I wanted to have real numbers. I also asked on the delicious mailing list. That same question was asked a few times, but no answers..
Now my stats don’t answer all question. If you’re asking yourself “how many inserts has my tag system to scope with if it gets really big” these will help you. But I cannot do any query-stats, maybe alexa may give you some query trends (maybe you subtract my number from alexas and will get the query stats?).

First impressions

From the stats you can see the two downtimes of delicious since August.

del.icio.us downtime august

del.icio.us downtime december

You also see that the recent growth of del.icio.us merely started in december. I think it has got to do with the more elaborated look and feel (changed in the middle of november) as well as with the new firefox plugin that does give a more professional touch to the service. This grow is a thank you to Joshua and this team.

Then, take a look at the “tag hump” at 10 tags per posts:

del.icio.us tag distribution december

My first quick investigations show that this is caused by – you guess it – tag spammers.
I found two spammers that constantly post bookmarks with 10 tags (look out, the first link has got chinese characters in it, my firefox slowed down big time). This shows that stats can help finding anomalies such as spam.

I also thought that maybe the lazy sheep bookmarklet can cause such humps but, by default, lazy sheep’s posts have a maximum of 6 tags. There’s no irregularity at “6″ so I guess lazy sheep doen’t have a big influence (which is a fact I’m quite happy with).

I think it will be interesting to observe these tag graphs when the bookmark post user interface changes. I believe the interface plays a big role in how people tag and this sort of graphs could prove that.

Further improvements

I may give statistics about the number of estimated users (currently tracked: 100k) and number of bookmarks (currently tracked: 500k) but I’m not yet sure how I can compute numbers that seem accurate.
I plan to come up with a few other del.icio.us services such as tag clusters but I’m not yet sure if that project comes to an end so I’ve decided to put up the stats so you’ll have at least this.. :-)

Hold on, that’s too much del.icio.us for me

Uh, all this talk about del.icio.us is too much [Otis]

Yeah, you are right. The point is that this stats can be computed from all tagging-powered webservices that serve a “most recent posts” feed. If you’re interested to have a stas on a different service or you want to do del.icio.us stats by your own just leave a comment. If there is enough request, I’ll comment&refactor the code and will publish it as LGPL.

Comparing to other services

Del.icio.us vs. Yahoo MyWeb 2.0

Dorrian Porter has tracked the number of posts of Yahoo’s MyWeb2.0:

posts per week on Yahoo’s MyWeb2.0 (graphic by Dorrian Porter)

Newly saved pages have averaged between 10,000 to 20,000 per week

These numbers are per week. Del.icio.us has got an average of about 55′000 posts per day! This means that right now the data base at del.icio.us grows about 20 times as fast as the one of Yahoo’s MyWeb2.0. That leaves no question as to why they have aquired delicious.

When I first heard about del.icio.us (and after that few days when I didn’t get it..) I thought: “This is revolutionary”. There were many things tags made possible that were just not possible until that day.

Joshua Schachter was the guy that invented tags (or at least that’s how the story is being told). Originally thought as a way to organize ones own bookmarks the social effect became obvious:

If everyone tags, the “community” profits.

Now, we have del.icio.us. Now we organize our bookmarks with tags. And our photos.
And our books, our games, our software, our tagging sites, and also your bulldogs, if you have any.

However, as we have tagged our whole life, what do we do with it? What is it good for?
I fear the tagging-revolution is about to calm. And I believe that’s because many people don’t see the advantages in tagging. I believe that many many things can be made possible by using tag-based systems. If we realized this, tagging would get some fresh air and eventually tagging gets mainstream.

Is it just me, or is the tagging revolution really stuck? I desperately miss new, visionary, inventive articles on tags.

• To all smart people, where are your ideas?
• To all programming geeks: Where are your algorithms, your “proof of concept” web services?

I could stop here with my article, but, hey, I don’t want to be the grumbling guy that sits and waits for new things coming up, so here I am, trying to expose my brain to you.
In this article I want to take a look at what areas tags are already strong in and how tagging could gain ground in these areas.

Tags help you to organize

When Joshua came up with the idea of tags, it was purely meant for organizing. It was only when also other people started organizing by tags, when the whole idea of “folksonomy” came up.
What does organizing mean? It is like tidying ones room: You put every paper and pencil you have at a place you can remember and seems logical to you so you can easily remember where you have put that thing. Now as we are not limited into physical means when we organize data we have many new possibilities. There are already good articles about this so I won’t discuss this in detail here.

At the end of the day, the question arises: Is organizing your bookmarks by tags really that good?

Just to make the point I come up with another way to remember things: While browsing, the browser could save all pages in a cache and when you are searching for a page you have visited (which is why you originally bookmark the page anyway), you make a fulltext search through all your cached pages. It’s a kind of “Google search” over pages you have already visited. I know this would have some downsides but it would have some advantages too. I often searched for a page I bookmarked and couldn’t remember the tag I used. This problem wouldn’t occur in the “searching through cache” system.

What I am trying to say is: If tagging solely would be for organizing your stuff, it wouldn’t be worth the trouble.

Folksonomy – Classification of the masses

Folksonomy is – as I understand it – the distributed classification of data by the big mass of people who tag stuff. Folksonomy often is said as a new system to build a taxonomy. It’s like building the Open Directory by thousands of people tagging stuff.

What is folksonomy good for? Why do we want to put bookmarks into categories?

Folksonomy enables to explore

Where do you go to start building a new expertise? Is it del.icio.us? Is it Google?
Let’s say your boss tells you that the data your software saves in the database should be encoded. You didn’t think much about cryptography, it merely was a topic that you “should know about” but you were never really interested in cryptography (I’m speaking for myself here.. :-) ). You don’t really know where to start. You know you want to know something about cryptography, but you don’t know exactly what.
A good list of articles or even starting points could shorten your learning curve.
Thereafter, you may want to “travel through the cryptography universe”. And to travel means knowing which articles are related to the one you just read and are so enthusiastic about. You need a map of the cryptography universe, you want to know what is left and right, top and bottom, you want to know everything and everyone related to “cryptography”.
Now then: What would you do?

Do tag systems help you to explore?

Delicious on cryptography

Delicious results on «cryptography»

I would go on del.icio.us/tag/cryptography+introduction. There I find a nice article titled “An Overview of Cryptography“. I guess I’m lucky! If I’d read the article, I’d probably find out which subtopics exist, how cryptography is related to similar issues and so on. You kind of get this “map of the cryptography” universe. But, this is done by only one author. Probably I don’t trust him (probably I should do so, after reading his cv), or you simply do not have time and/or energy to read through 44 pages, although the article looks good. I’ll probably go back to delicious and find out, that the related tags of “cryptography” are:

• security
• reference
• encryption
• crypto
• algorithms
• computing
• software
• nsa
• tutorial
• kids
• education

Now this is not very convincing, is it? You argue:

Yeah, but this is far better that what I get on Google

.

Google on cryptography

Google results on «cryptography»

It is. When looking at this Google results I remember that Google is meant for searching when I already know what I search for. But now I am at a different stage. I don’t know exactly what to search for. I don’t know, because I don’t have any expertise in cryptography. BTW: Google does come up with an article that looks like a good introduction into cryptography as well..

Open directory on security

What about open directory? Let’s give it a try: After typing in “cryptography” I find out that this topic is classified in Science > Math > Applications > Communication_Theory > Communication Theory > Cryptography. Clicking this link you get what you were probably looking for.
You get a nice overview:

Google open directory on «cryptography»

• Algorithms
• Books
• Events
• Historical
• Journals
• People
• Programming Libraries
• Research Groups
• Theory

Now you stand at a guidepost. You see the “cryptography universe”. You probably don’t see what is left and right to cryptography, but here you have a “cryptography at a glance”.
Now it’s up to you: Do you want to explore “algorithm land”, take the shortcut and download the programming library of the language of your choice? Or do you even want to get advice from people that are experts on that matter?
Even if the links provided here don’t give you what you are looking for, here you get a clue what you should look for.

Comparing the three

Let’s compare browsing to a reallife quest: Finding out where your next conference will take place. Say you want to go to the next etech conference, you don’t know where it is and you are not an American citizen.

Ouch, nearly bumped my head into horton plaza!

On the conference websites they often put a map showing the conference place like 10 meters above surface. This map is helpful. But only at the point when you are quite next to the conference.

Help, I cannot breathe out there!

Then, when you desperately search for a more general map, you’ll possibly find a map of how it looks from outer space. Yeah, I know that San Diego is in the US, but I’d like to know which airport is next to the conference.

Distances between observer and data

That’s quite similar to the views we have with del.icio.us and open directory.
Delicious would tell you: “the roads nearby are ‘union street’, ‘Broadway circle’ and ‘Broadway’…”, open directory proclaims: “we have five continents in the world: ‘America’, ‘Asia’, ‘Africa’, ‘Australia’ and ‘Europe’…”. Now, I’m exaggerating a bit but you get the point: Sometimes you need a map that lays between the too detailed and the too general map.
Looking for this type of view is like saying: “I want a bit more ontology than tags but not that much taxonomy as open directory”. That’s where I’ve put the question mark. It’s not that you always want to see the data at that distance but sometimes you desperately want to have that viewpoint.

Now, what has this to do with tagging? I believe that this missing in-between view can be won by analyzing tags.
Have you noticed how flickr does this in-between view?
When you search for love, flickr cluster asks you: “What do you mean by ‘love’?”:

flickr cluster results on «love»

• a couple kissing?
• a mother holding it’s baby?
• a red heart?

“Wait: Flickr is a bit different from del.icio.us”, you say. Yup. Flickr uses a narrow, del.icio.us a broad folksonomy system.
But I believe that the data clusters, flickr creates with it’s narrow folksonomy data, can also be generated with delicious’ broad folksonomy data. I am programming an algorithm that computes del.icio.us clusters. I’m still at an early stage but I get clusters like this “shopping cluster”:

«shopping» cluster

I realize that even if the cluster data is available, there’s the question how to navigate through the data. The “zooming in” and “zooming out” won’t be as easy as with Google maps.
But anyway, here is the land no one has explored before. I think this is the area we should talk about. Here is room for improvement.

Folksonomy helps you to stay informed about a certain topic

Back to what folksonomies are good for: If you have built an expertise in cryptography, you want to stay informed. If RSA is hacked, you certainly want to be informed.
Delicious has got an “Inbox” where you can subscribe to a tag, e.g. “cryptography”.
Each bookmark that is tagged “cryptography” gets in your inbox. That’s a great way to stay informed. Alternatively you have a list of of recent popular sites tagged “cryptography”. You can subscribe to this lists using RSS and hopefully you get informed timely if RSA is hacked..

Do tag systems keep you informed?

I think the comparison with the distance to the data applies here too:
If I’d subscribe to cryptography, I’d probably miss some important items, just because the guy who bookmarked it used the tag “crypto”. On the other hand, I do not want to be informed about another Rijndael algorithm, I want to narrow the incoming links to articles or essays that deal with cryptography.
Delicious already offers to narrow results: I could subscribe to «cryptography» and «essay», and, when delicious will support union (and it will, as Joshua promises), I also could have subscribe to (cryptography or crypto) and (essay or article) but you see that it doesn’t really solve the problem.
I imagine that one day you can say:

I want to keep being informed about cryptography

and the service asks you:

Should I keep you informed about

• new implementations
• new articles/essays
• security issues

And I believe this is possible. Flickr already asks you this when you are searching for love pictures. I guess it will be based on clusters again.

Tags help you sharing Lists

Back to what tags are good for: They help you building lists. Let’s name a few examples:

• Wish lists: I know that numerous online shops enable you building whishlists. But I’d like to have a whishlist that’s not bound to a company, that I can arrange and rearrange. Many are already using del.icio.us as a storage of their wish list.
• Share your bookmarks: A friend asked me for some links to javascript WYSIWYG editors. I gave him a list of all my bookmarks tagged javascript and editor
• Offer viewpoints of your data: Let’s say your favourite CMS features tagging (featured in many of those new fancy ruby on rails applications), I’m not speaking about blogs here: To allow “normal” visitors to view your data, you’ll add a navigation providing starting points to your entries; specific locations a visitor can jump in to so he could take bathe in your articles. Probably you would add a link to all items tagged “references” and “networking” to achieve that.

How can tag lists be improved?

I’m often annoyed that I cannot put my del.icio.us links in a specific order. I did a little script that puts my newest bootkmark at the bottom but it doesn’t fully solve the problem.
Actually I’d like being able to compose a view of tagged bookmarks, i.e. I want to offer a list of all firms our company has built the network for:

Networking references

Big firms

• UBS
• Migros
• ABB

Medium-sized firms

• Stadlerrail
• Räber Information Management GmbH

Small firms

• Citrin Informatik GmbH
• Goldschmiedeatelier Thildy Keller
• Mini Fruits Trading

Nowadays, such a list can’t be automatically generated from my bookmarks, but it could be, by letting me configure my view as myView = (references+networking, "Networking References", (big_firms, medium-sized_firms, small_firms)).
I know it’s not a big challenge to program such a thing, but nonetheless it doesn’t exist, as far as I know?

Bottom line

It appears to me that there’s not been much progress being done related to tagging systems lately. What rather became better is the embedding of tagging systems into already existing technologies such as search. It gives the impression that core issues are done and that there’s no much room for improvement. In this article I wanted to disprove this.
I think that there’s much much more than I have written in here, I even believe that todays tagging applications cover just about 5% of all the possible features tagging makes possible. Thus, let’s gain ground.

I am a big fan of delicious. I’ve got 1.5K bookmarks there, I like it’s spirit and how open everything is. This article isn’t meant to criticize, but I think delicious is facing some problems.

Performance scale

You might have read my article about Tag system performance. To summarize my tests: MySQL is just not built for large tag-systems. It just doesn’t scale. It does scale up to 1 Million items but delicious does have far more posts.
I am pretty sure delicious is still on the MySQL train, this strong believe comes from my performance tests: The mysql-schemas I tested really have the same characteristics as delicious has.
I fear delicious faces a performance dead end: They have put more servers in the mix, they cache quite a bit, it still is slow. I strongly believe that for delicious to have a future it must become much faster. For me this is the number one downside of delicious. I dream of a bookmark service that has billions of bookmark-posts yet it still will perform nicely. I think it is time for new tag-systems to come up. On tagdb mailing list, there are very good ideas how large scaled tagging systems should work (e.g. systems powered by Lucene).

Popular link scale

I think one of the coolest feature of delicious is the popular page. When you read this page regularly you are up to date.. wait: you are up to date concerning CSS tips and firefox and live hacks. You all know that if delicious would get mainstream that page wouldn’t be that interesting any more. It already got boring a bit. As someone put it:

I particularly cannot look at that CSS link lists anymore

I think this page doesn’t scale. It is stuck. And moreover it’s a pity that the coolest page on delicious is not about tags. At first glance you don’t even see what tags a popular link has.
IMHO what is needed here are clusters. Bookmarks go into categories: “browsers”, “programming”, “design” but also “health”, “politics”. When delicious gets mainstream there most certainly will be “sports” or “stars”.
One should then have the possibility to subscribe to certain clusters or better make this subscription automatically out of tags in a users bookmarks.

Bottom line

I think there are some fundamental things that must be rearranged at delicious, otherwise there will be

• a) a big competitor (Google? Yahoo? Microsoft?) coming up or
• b) people will spread to different bookmark services that concentrate on certain clusters. Probably some meta-sites will arise where you can have an overview over all the different sites

I think this problems will arise for every bigger tagsystem. I hope that people will not sniff at tagging systems thinking that they don’t perform well enough..

“writing”, “toread”, “productivity”, “language”

Now what instantially pops into my mind is, that the tag “toread” is quite different from the other tags. In fact it is something I want to do with this bookmark further on. I name this type of tag “adjective” (I will come back to that name later on..). The other tags I consider as “categories“.
Now you’ll probably say “ah, this is a rare exception”. This is not true. I often tag items with “blog” because it happens that the interesting page I found about my favourite hobby happens to be a blog. Therefore I named this type of tag as “adjective” as it is rather a description to the item than it is a category to it.
Other tags used often as adjectives are “reference”, “tutorial”, “fun”, “cool”, “news”, “free”..

Now this categorization is not very correct. Sometimes, I use “blog” not as a adjective. This is, if I want to bookmark a blog that has no content that interests me but it just looks good. Then, I’ll probably blog it as “design blog”. In that day when I redesign my blog, I want to search for all design blogs I tagged..
You see: it lays all in the connection between those tags, not in the tags itself. This is IMO pretty important.

What is that for?

Clusters

You probably tried to cluster your bookmarks by using clusty. What this service does: It tries to put your tags into separate clouds. You know the “tag-bundles” of delicious? This is something like a “auto-tag-bundle” feature. Try it out, if you not already did so and see the problems that arise..
I think the key problem in this cluster-service lies in the fact that this service considers all connections (also the adjectives). But it shouldn’t do so! Adjectives aren’t tags I want in my clusters. Adjectives are spread all over my tags, so they should first be cut away from my “tag-tree” (the tree that is built out of your tag-connections you built by tagging bookmarks).

Similar items

This categorization is also important when you search for “similar” items of a bookmark. When I want to search for similar items of that “how to write more clearly”-article, I’ll search for “writing+productivity+language” and will leave out the “toread” tag (adjective).
Probably this made you realize that categorizing tag-connections is an important task.

Tag clouds

Now there are those tag clouds. When I look at my taggloud then the “biggest” tag is “resource”. Now tag clouds are here to easily find bookmarks (I never search my bookmarks for solely “resource”) or to have a map of your main interests (“what is your hobby?” “ah, I am a big fan of resources”.. :-) I am sure you were also annoyed by that. I want those adjective-tags cut away..!

Synonyms

Now back to some therory: There is a third type of tag-connections: Synonyms. “delicious” and “del.icio.us” are classic synonyms. But I consider “ruby” and “rails” as synonyms too (no, they aren’t synonyms but up to now they are used as synonyms). You type in the second tag just to be sure that you won’t search for the second and find nothing.. I don’t think this category is too important for the cluster-task but I just name it here because I’ll use it further on.

Example

Let’s go for an example.
Lets consider tags that are connected to the tag “ajax”. I gathered some tag-connection-data from delicious (via its rss-feed). And I run a query on my statistical data. This is data gathered during the period of one week. It is not complete. But our experiment will work anyway:

Column “tag-connection” is the tag connected to “ajax” (i.e. javascript), column “weight” depicts the number of times this connection occurred in a bookmark-post on delicious. The tags are ordered by weight. In column “type” you see the result of my computations for this tag-connection. Just to make it clear: These are all tags connected to tag “ajax” ordered number by occurrence of the connection. If a bookmark-post somebody did on delicious is tagged with “ajax” and “javascript” that gives one point for the “weight”-column for “ajax-javascript”.
The outcome is quite good, I think (I must admit that I have taken the example that worked out best :-))
There are some errors, sure: xml-ajax should be a “category”-type as well. But we are looking at the usage of these tags not their “real” meaning (whatever that is).

Computation

Synonyms

To compute these categorization I first went for the “synonyms”. The connection “ajax-javascript” is considered as synonym because “ajax-javascript” is “number one connection” of all connections where ajax is a part of. And when considering the connections of “javascript” (the “vice-versa-connection”), ajax is number two.
I consider two tags as synonyms if “in one direction” the other tag is number one and in the other “direction” the other tag is in the top 10. I made up this rule because I think that in most cases there is one “stronger” synonym that is used most of the time when the “weaker” one is used. The fact that the tag “ajax” is mostly used with tag “javascript” could also mean that “javascript” is a supercategory of ajax (which it somehow is). To avoid that this sub-super-categogy-connections are considered as synonyms, we go sure that “ajax” is also important for “javascript” so ajax is not so sub to javascript.. I hope you can follow :-)

Category/Adjective

Then I compute the “category”. Lets put the values of the above table into a graph.

On the x-axis you see the tags: The tick 1 stands for “web”, 2 for “programming”, 3 for “css”, 4=”design”, 5=”php” and so on. You see I removed the synonym-connections “ajax-javascript” and “ajax=xmlhttprequest” as I think they “disturb” the distribution.
The y-axis depicts the weight of the connection: ajax-web has weight “105″, ajax-programming has weight “100″ and so on.
The black line is the “weight”-column of the table above, the red one is the first derivative, the blue one the second derivative of the weight function.
This graph makes it clear that “web” and “programming” are used quite often in combination with “ajax”, then, there is quite a “gap” followed by the “adjective tail”. I consider the “adjective tail” as connections to be categorized as “adjective”. The tags in this tail are used “out of context”: They don’t really belong to the “ajax-cluster”. They sometimes occur together with ajax, but just sometimes. Mostly not. Therefore they are considered as “adjectives”.
Now the task is to find this “gap”. In my experiments I tried to find the last gap. To find the last gap I started at the end of the tail and searched for the first peak of the first derivative (that is when the second derivative goes from positive to negative) and checked if the peak was high enough. If these to conditions were fulfilled, I snipped the connections into two parts the “pre-gap” connections (category) and the “post-gap” connections (adjective).
The same computation has to be made for the “vice-versa” connection. I considered connections as “category” if one of both computations told that it is a “category”.

Further processing: Ambiguous tags

To achieve good clustering results, I think there is a need of checking if the tag is used in different ways. The prominent example hereof is “apple”. Now, when delicious is still restricted to the blogworld, it is clear that apple means Mac-apple. But in future this may change. To recognize if a tag is used in different environments, the algorithm would have to check the “neighbours of neighbours” (as suggested by Pietro Speroni). That is for ajax: check if the neighbours of “javascript” are more or less the same as the neighbours of “web”. You see that it all lays in the connections between tags. The tag per se is not well-defined but the tag in connection with another tag defines it quite well. Therefore for clustering I’m proposing splitting up amiguous tags. That would add much more simplicity to the resulting clusters.

We are onto something

I’m pretty sure we are onto something. I think this is direction it should go. Computations over tag-connection-distributions are cool. Users shouldn’t insert these infos when posting the bookmarks. Posting should stay easy. I’m not that sure about this “synonym”-computation but I think the “category”-computation turned out pretty good. I tried to build some clusters by hand just by considering the category and synonym-connections and I found a completely detached cluster consisting of the tags “cooking”, “health”, “recipes”, “diet” and “food”. As I said, I think we are onto something..

Further reading

• Building tag clusters by hand
• Pietro Speronis different approach to clustering tags (with java-mindmap-visualisation!)

I tested the following schemas (I keep the naming from the previous article):

• mysqlicious: One table. Tags are space separated in column “tags”; as introduced
• mysqlicious fulltext: Same schema but with mysql fulltext on the tag column; as introduced
• scuttle: Two tables: One for bookmarks, one for tags. Tag-table has foreign key to bookmark table; as introduced
• toxi: Three tables: One for bookmarks, one for tags, one for junction; as introduced

You may want to have a close watch at the details of the schemas when having a look at the sql-create-table-queries.

But let’s go directly to the results. The details about the setup of this tests are mentioned at the end of this article. The x-axis depicts the number of bookmarks in the corresponding database, on the y-axis you see how much time each query took to execute.

Results

Intersection: 250 tag set

The first two tests are done with 250 tags in the small dataset (see below for explanation). I think the queries in the “1 million bookmarks database” are the only size we should pay attention to. I mean if you have a small number of bookmarks, performance isn’t really a thing to bother..

We run intersection queries, like

I want to search for bookmarks tagged with “design” and “html”

You see that, not surprisingly, mysqlicious with its WHERE tag LIKE "% tag %" is very slow. That is, MySQL has to go through the whole dataset and test each bookmark against the query.
What actually is surprising me, is that the fulltext search of mysql is not that high-performance. In fact it is not faster then the LIKE-query in the MySQLicious DB. This really disappointed me. I tried to do any quirks possible to make this faster as I think, a tag-database-system with mysql fulltext would be very easy and like the only thing you should head to...
What is surprising me too, is that the queries on the 3 table schema are about double as fast the the ones on the two-table ones(take a look at the queries if you think you could give me a hint on this). Noticeable is, that in the scuttle and toxi-variant, the more queries were run, the faster they were. I didn’t do any tests with queries and inserts mixed so this may be coming from just plain good caching and this effect possible doesn’t show up on live bookmark management systems.

Intersection: 999 tag set

Now have a look to what happens if we broaden our small tag set: MySQLicious with fulltext suddenly gets the performance leader. That means, if you have a bookmark management system with diverse tags (this most probably comes from the fact that there are many users), the fulltext solution is possibly the way to go.
So now, as you see, choosing the right schema is all about tag distribution. In my previous post about guessing the overall tag distribution on del.icio.us, I came to the conclusion, that delicious’ most popular tag “design” is showing up in 3.2% of all bookmarks on del.icio.us. So then, what is the mean tag distribution?

• If we say 1% (a tag shows up in 1/100 of all bookmarks on an average) that makes our small tag set 250 tags big
• If we say 0.25%, the small tag set grows to a size of 1000
• If we say 0.1%, the small set will contain 2500 tags

So I’d suggest that if your average distribution is 1%, take “toxi”, if the distribution is broader, take “MySQLicious fulltext”.
If you take a closer look, you can see that the fulltext schema stayed as fast as when queried in the 250 tag set. That means, if you want to go sure your tag system responds ok in every situation, you should go with the “mysql fulltext” schema.
Hannes has done some further investigation on mysql fulltext running on MySQL 4.1 (my tests were on MySQL 4.0.21)

Union

When doing a union query we say

I want to search for all the bookmarks that are tagged either with “delicious” or “del.icio.us”

This queries, you guessed, are handled the fastest by “MySQLicious schema” with its LIKE-queries: MySQL seeks through the bookmarks, harvesting all bookmarks with one of the given tags and says “I’m finished!” when it was at bookmark number #968, because it found 50 bookmarks. Whereas in the other schemas, MySQL has to join the tags with the bookmarks first and only then could search though it..

Insert

When comparing the different schemas on the time of the insert-”statements” of one bookmark, the result isn’t very surprising (notice that I’ve changed the scale of the y-axis).
Mysqlicious with it’s 1 table is very fast indeed, its variation with fulltext had to create the fulltext index and therefore is a bit slower. Scuttle, with its 2 tables and toxi with its 3 tables are at least two respectively three times as slow. I have to remark, that I used quite a bit of caching for the toxi schema, as I didn’t want hours to have the data ready..

I guess it doesn’t really make sense to base your decision, which schema to take on the time for an insert: Bookmark inserts are about 100 times as fast as the intersection queries..

«What? That slow??»

You said it. You don’t want your intersection queries take 0.2 seconds each. That would bring your system to its knees.

There are some recipes to avoid that:

Caching

I think, you don’t come around good old caching. I think that you could cache results to a query like “mysql+tagging” for about an hour or so. If a user queries his own items, I would lower the cache time (as up-to-dateness is more important with his own items).
Then, I expect if you for instance cache items per tag and intersection them with a decent algorithm, that could be faster..

The Best Of Both Worlds

I think you could have “mysqlicous fulltext” and “toxi” running at the same time. That means you have to update/insert in both schemas but when you have to query, you could take the one you think is faster: For simple union the mysqlicious without fulltext search, for intersection queries with common tags the toxi, and for those with uncommon tags the mysqlicious fulltext variant.

Slicing and dicing

You could “slice and dice” data (as Nitin proposed it in two of his posts): That is: you slice your user/tag/item-room and build fact tables. You “prebuild” your results in a way. This way, inserts take long but queries themself should be much faster. In our examples, you would for instance first query the tag-intersections on “toxi” and then get the facts about each bookmark out of the “mysqlicious”-fact-table. But you really should read Nitins posts, as they give a lot of insight.

Using a non RDBMS system

Update: It’s been about a year since I wrote that article, and during that year I came to the conclusion that RDBMS systems don’t scale good in systems that have more than 1 million items. Yes, this is a warning: If you are planning to build a large scale system then look for alternatives to RDBMS systems. To quote Joshua Schachter, founder of delicious:

«tags doesn’t map to sql at all. so use partial indexing.»[Joshua Schachter at Carson Summit]

I didn’t try any of the non-RDBMS system but it looks like Apache Lucene and Hadoop. There has been a discussion on the Tagdb Mailing list about these solutions.

«I don’t believe you! I want to try it at home»

Download the source code (PHP) I used to run the queries and test yourself, extend them as you like. The source is published as LGPL.

Performance Tests Setup

Now, if you have read that far, you probably want to know some background information: As you noticed, for each schema, I set up 4 databases, one database holding 1000 bookmarks, the next 10′000, then 100′000 and the fourth 1 million bookmarks. The inserted tags (as well as urls) are random English words taken from two sets of tags:

• the large set containing about 44′000 tags (that are simple English words)
• the small set is varying in size (the results shown here are taken from 250 and 999 tag sets)

Every bookmark gets one to ten tags attached. Every odd tag is from the large set, alternately taken from small and large set. Every schema got exactly the same bookmarks and tag data.

Then every schema got queried with an alternately 1-3 tag query. So the first query is for instance just “blog”, the second “design+css”, the third “webdesign+music+software”, the fourth again with just one tag an so forth..
All the tags for the queries are taken from the small set so that the queries don’t all end in empty results..
All the queries are tested and work. The outcome of each query on the three schemas is exactly the same.

Mysql Setup

I used mysql 4.0.21.
An excerpt from /etc/my.cnf (I think these are the relevant settings to this performance test)

key_buffer=300M
query_cache_size=30M
query_cache_limit=30M
table_cache = 64
ft_min_word_len = 2
ft_stopword_file = ''

System

CPU: 3GHz Dual Xeon
Cache: 1MB
Harddisk: SCSI Ultra 320 Atlas 10K, no RAID
RAM: 3GB

Assumptions

• Queries select just the id of a bookmark. I assume that you have to do a second query to get all the wished data to display. I know that this is not fair towards the mysqlicious schema.
• I left out user data, as I assume, user data columns wouldn’t change the outcome of this tests. I wanted to keep the schemas as simple as possible.
• Each query is done with LIMIT 50 as I assume that a normal application doesn’t want to get all bookmarks. I assume nobody wants to order bookmarks by any dimension, because this would be very expensive (ever wondered why you cannot sort bookmarks on del.icio.us by date or similar? You get it..)

Acknowledgements

Thanks to Citrin, the company I work, to let me use our new server to run the queries. The server didn’t have much anything else to do so the results should be accurate.
The graphs are done using JpGraph. Very easy to use and produces beautiful images.

Further reading

• Flickr architecture
• Lab notes: Fulltext not so fast: Fulltext performance issues
• WebmasterWorld forum: mysql fulltext performance issues
• Mysql Supersmack: Mysql performance tool
• Mysql Benchmark
• Powerpoint article of jeremy zawodnyon Mysql optimisation
• Pete Freitag did a sort of review of this article