previous post: «Analyzing tag-connections»
Lately it became very silent around del.icio.us. There are some new features but nothing groundbreaking. Either people are used to it and use it as a daily tool and there’s no need for new things or otherwise folks just don’t have faith in the future of del.icio.us.
I am a big fan of delicious. I’ve got 1.5K bookmarks there, I like it’s spirit and how open everything is. This article isn’t meant to criticize, but I think delicious is facing some problems.
You might have read my article about Tag system performance. To summarize my tests: MySQL is just not built for large tag-systems. It just doesn’t scale. It does scale up to 1 Million items but delicious does have far more posts.
I am pretty sure delicious is still on the MySQL train, this strong believe comes from my performance tests: The mysql-schemas I tested really have the same characteristics as delicious has.
I fear delicious faces a performance dead end: They have put more servers in the mix, they cache quite a bit, it still is slow. I strongly believe that for delicious to have a future it must become much faster. For me this is the number one downside of delicious. I dream of a bookmark service that has billions of bookmark-posts yet it still will perform nicely. I think it is time for new tag-systems to come up. On tagdb mailing list, there are very good ideas how large scaled tagging systems should work (e.g. systems powered by Lucene).
I think one of the coolest feature of delicious is the popular page. When you read this page regularly you are up to date.. wait: you are up to date concerning CSS tips and firefox and live hacks. You all know that if delicious would get mainstream that page wouldn’t be that interesting any more. It already got boring a bit. As someone put it:
I particularly cannot look at that CSS link lists anymore
I think this page doesn’t scale. It is stuck. And moreover it’s a pity that the coolest page on delicious is not about tags. At first glance you don’t even see what tags a popular link has.
IMHO what is needed here are clusters. Bookmarks go into categories: “browsers”, “programming”, “design” but also “health”, “politics”. When delicious gets mainstream there most certainly will be “sports” or “stars”.
One should then have the possibility to subscribe to certain clusters or better make this subscription automatically out of tags in a users bookmarks.
I think there are some fundamental things that must be rearranged at delicious, otherwise there will be
I think this problems will arise for every bigger tagsystem. I hope that people will not sniff at tagging systems thinking that they don’t perform well enough..
RSS feed for comments on this post. TrackBack URI
Have you checked Guten Tag –> http://creative-mobs.com/portal ?
Comment by Stephane Lee — September 1, 2005 11:03 am #comment-1205
I just checked it. I already stumbled upon it a few times..
Looks interesting, though I’d wish that the results are more “mingled” together, so the bookmark results aren’t separated per site: Delicious and Blogmarks should be together. I’m still waiting for a site that clusters this thingies.
So that is if I’m interested in what’s happening in the browser world, then I have a subsite that shows me what’s going on there. The “per tag search” is too low level for me to actually browse bookmarks/photos/whatever.
It that you that runs Guten Tag (cool name!)?
Comment by phred — September 1, 2005 11:57 am #comment-1206
Yes, I run this thing.
I still have many things to work on before it becomes something great ! No much time to give though.
Expect something new soon.
Too low-level ? Hum, how could it be different ?
Related tag aggregation ?
Please feel free to drop me any advices that’d be useful.
Comment by Stephane Lee — September 1, 2005 4:16 pm #comment-1208
Re: use of Lucene - please see Simpy.
Comment by Otis Gospodnetic — September 1, 2005 6:56 pm #comment-1211
ever seen flickr? even before it got bought by yahoo, it had far more users than del.icio.us, had tags, and had great performance. del.icio.us/MySQL will scale fine performance-wise once Joshua has staffed up.
Comment by getaclue — September 2, 2005 2:20 pm #comment-1218
Getaclue: I fear it won’t. Have a look at my the performance tests. When reaching the 1 million marks in number of posts the performance breaks. There’s no way to avoid this, except taking an additional index/whatever into the mix.
Flickr is not based on tags. Tags are an addon there. On flickr there’s no possibility to make tag-intersecions IMHO. Tag intersections are the expensive queries. The “one-tag-queries” are quite cheap.
What do you mean with “staffed up”?
He already optimized indexing. He put in new servers. That didn’t help too much. Perhaps he needs even more servers.. but however: If you have much data then one query takes it’s time and you cannot limit this time in adding more servers.
Then, finally, on delicious you work with your bookmarks. I suppose, most users use delicious during their time working. When you work, you want fast tools. I suppose people are using flickr in a different way, namely in free time: It is not that important to have a fast site when you are browsing photos..
Comment by phred — September 2, 2005 3:08 pm #comment-1219
Ah by the way, when flickr has already been mentioned: Flickr has a very neat cluster engine. Have a look at their love cluster. Ah I’d love to see such clusters on delicious..
Comment by phred — September 2, 2005 3:14 pm #comment-1220
Otis: Looks very interesting! Maybe we all (45K users) should switch to Simpy and then watch how it performs? :-)
Comment by phred — September 4, 2005 4:11 pm #comment-1228
I think an excellent fix for the popular would be adding two new popular pages: 1 for each tag (or each tag and its related tags) instead of rigid “categories” and 1 for each person. The second one would find popular links from your inbox, and maybe go one more step and find all the popular links from the inboxes of the people in your inbox. Social clustering for social bookmarks.
As for competition, Yahoo! has their version up, called My Web 2.0.
Comment by George Hotelling — September 10, 2005 8:32 pm #comment-1285
[...] Most tellingly of all, the question of scalability is answered with a prediction that meta-sites will start to crop up, providing social bookmarking to specific areas of interest, like Millionsofgames.com does for casual games - http://www.pui.ch/phred/archives/2005/08/does-delicious-scale.html [...]
Pingback by Suttree » Elixir for Immortal Baboon — October 11, 2005 3:33 pm #comment-1505
FYI, flickr does tag intersections just fine:
http://flickr.com/photos/fncll/tags/nome+whitealice
I see no good reason why MySQL couldn’t continue to work on del.icio.us…
Comment by Chris L — December 11, 2005 11:54 pm #comment-1904
Chris: But flickr is a different case: With delicious, everyone can add tags (broad folksonomy). On flickr, just the photo uploader can (narrow folksonomy). I guess the data on delicious is much “broader” than that on flickr.
You are right that it works. But I doubt that it is efficient. Just have a look at the performance measurement. I don’t see a way to improve the performance.. The barrier is MySQL itself I suppose.
Comment by phred — December 12, 2005 9:20 am #comment-1908
[...] Im August hat Philip Keller einen interessanten Artikel veröffentlicht, der der Frage nachgeht, wie skalierbar del.icio.us ist. Auch diese Frage wiederum in zweierlei Hinsicht - sowohl technisch, als auch inhaltlich. [...]
Pingback by Themenmonat Tagging: 2. Aktuelle Trends, Probleme und offene Fragen | agenturblog.de — June 24, 2006 10:18 pm #comment-4335