The graph was done with GraphViz. You can laben the edges and even influence how long they should be, e.g. look at this example: http://www.graphviz.org/Gallery/undirected/ER.html

The coloring I did by hand with Inkscape

I was exploring the tagging world and really like ur work. I have question about the Cluster Graph. how you generate the graph (i mean visually) with weighted edges.

Thanks in advance
—
Noor
Aachen, Germany

I tried using kmetis from metis-4.0 suit and discovered it entertains only integer edge weights.

In case one opt for dice similarity the edge weights ought to have floating point values.

I was told by authors that hmetis2.0 supports floating weights. But unfortunately there are no windows binaries for the same yet and hence I could not give it a shot.

I plan to switch to igraph now. I will report the results when available.

Mean while if any one has any suggestion or comment please respond.

By the way I am trying to use the tag clustering in my image search system. I am quite optimistic about the results.

Best,
Shrijeet Paliwal
CS@Stony Brook University

as already pointed out METIS is minimizing the edges between different clusters while balancing the size of the clusters. Thus it isn’t very useful for community detection.

Most of the community clustering algorithms implemented in igraph optimize the same quality measure “modularity” of Newman. It describes mathematically what we may expect from a good clustering: That the vertices (tags) have stronger connections (edge weight) to the other vertices of their cluster than in a graph where all edges are equally distributed (no structure, called null-model).

In a perfect world all algorithms should produce the same result. But it’s a NP-complete problem and thus they have to guess and differ in how well they achieve the optimization aim. Beware that some implementations in igraph ignore the edge weight which produces additional structure where none is (i.e. edge weight 0.01 vs. 1.0).

The good news is, that it isn’t necessary to use eigenvectors to find good modularity clusterings. Some “simpler” algorithms exist with comparable or better results and without manually choosing the number of clusters.

Randolf

igraph implements a different spectral clustering based on the algorithm of Newman (see http://arxiv.org/abs/physics/0605087). It also has implementations for a couple of other clustering algorithms (e.g., the fast greedy modularity optimization of Clauset et al, the walktrap community detection of Latapy & Pons, the spinglass clustering of Reichardt & Bornholdt). If you have similarity measures, then the walktrap community detection or the spectral algorithm of Newman may be of some use to you.

I was looking around to find a library/tool that could perform tag clustering. As I understand I need something that deals with graphs.

I had a look at METIS (pmetis), but it partitions the graph into k equal size parts, and it doesn’t seem to perform the task of finding communities in graphs.

I just found that the algorithm you mention (described in the paper “Spectral Clustering Approach to Finding Communities in Graphs”) was implemented in igraph library http://cneurocvs.rmki.kfki.hu/igraph/ (at least they say that in their docs).

So my question is - do you know this lib? If yes, what is your recommendation?

Although this way is tedious I guess all public services offering user generated content and/or public APIs have to go down that road.

I was able to contact Grigory and he’s helping me. Sorry for the late response as I working out Eigenvectors and other mathematics concept, and implementing clustering algorithms.

Reg: Tag Spam- You said: “But the counter attack for spammers is very easy: Don’t use the same tag combinations too often.”
So, what you essentially mean is: the spammers would tag resources with unrelated tags to an extent when it is not so large. So that one can’t raise any suspicion.

Do you think that thinking of all these factors, isn’t there an opportunity for semantics of tags to come into the picture? Or can this in some way handled by some intelligent similarity measure?

Thanks in advance
Niket.

I don’t know another paper or article that deals exactly with that problem.

About your questions:
[1] What I meant was: When I looked to bookmark spam, the spammers often used the same tag combination, e.g. health-education. They added hundreds of bookmarks with tag “health” and “education”. This tag spam caused those two tags to have a very high similarity number. This number was so high that it was over the normal range of similarity. That’s how I spotted the “tag-connection-spams”. But the counter attack for spammers is very easy: Don’t use the same tag combinations too often.

[2] No

[3] I don’t know any of these programs. Even metis I never tried out. I’d need to find a library yourself. Maybe this list of Open Source Clustering Algorithms can help you?

[4] This part of the paper were findings of Grigory Begelmann. You have to ask him about these peculiarities..

Is there any paper/article which tells us comparatively, which clustering algorithm is best suited for tag clustering?

Thanks,
Niket