Poblish has always provided a “more articles like this” facility for every article on the system – not just related articles from that blog or Twitter feed, but related articles from all blogs and Twitter feeds. This list used to appear next to each article, crammed into a column that was always just a little too narrow to make the list truly usable, so I’ve moved it to a new screen which you can pop up using the big “Explore…” button.

We’ve also restored the “Similar Bloggers” facility and put it alongside the list of articles, to help you explore other bloggers who deal with similar themes. Finally, if you’re logged-in, you’ll find your own individual list of recommended articles. This uses the latest collaborative filtering techniques to suggest a list of articles based on your own ratings, flags, favourites, as well as those of people with tastes similar to your own.

Above the Explore button, you’ll see what looks like a “tag cloud” for the article. However, what you’re seeing is much cleverer than what 99% of other applications offer. We use semantic analysis to determine the article’s key themes, or “Zones“, rather than simply relying on the categories the blogger chose; we rank them according to how often they have been mentioned during the past 24 hours; and we provide a link to the Zone’s home page, where you can see – and follow – a feed of matching articles.

The point of all this is to seamless weave articles – whether blog posts or Twitter posts – into the greater and wider world of political content, using state-of-the-art techniques, and to make it easier than ever for people to explore and to learn.

I mentioned last month that Poblish has been using OpenAmplify‘s semantic/sentiment analysis service to give technology a shot at making sense of the vast sea of content that is the political blogosphere, in such a way as to help policymakers make better informed decisions. As I’ve said before:

Billions of individual thoughts and personal experiences have been written about, from all conceivable perspectives. No policy process will come up with ideas that have never been thought of before; so existing content represents a knowledge base that should not be ignored

In my piece at Left Foot Forward, earlier this week, I imagined a future in which such tools could take a source article and use this content to automatically, dynamically identify counter-arguments, hopefully before bad policy is made. Well, we have the content, we know that counter-arguments are out there, some of which may very well not yet have crossed the mainstream media’s horizon, and we hope – and believe – that technology can help us find them.

Only a very small percentage of Poblish’s articles have so far been semantically analysed (OpenAmplify are very kindly letting us evaluate their software for free, so the number of articles we process is limited), but all new articles are – and for those articles that have them, Poblish is now displaying the results in the page’s sidebar. Here are the results for the following article.

The way we display the results is simplistic at best, but essentially what we’re showing are the main topics from the article, divided into their relevant category, and coloured as follows:

• Blue: favourable references (or “polarity”). Dark blue for wholly positive (never negative), light blue for generally positive (but occasionally negative).
• Red: unfavourable references. Red for wholly positive (never negative), pink for generally positive (but occasionally negative).
• Grey: neutral references, or a mixture of positive and negative ones.

Clearly there are successes and failures in the above list. Sunny Hundal‘s name appears as a mere noun, rather than a human name (though I wonder if the fact that his surname was misspelled in the original article is relevant here) and some of the polarities seem a little random.

Bear in mind, though, that each set of results you see was the result of an analysis of one, single article, without any context. Give the tool 200,000, however, and we can be certain that insights will start to massively outweigh mistakes. Context is critical, and – just as we don’t judge people or texts on the basis of what we objectively see – semantic applications should not be regarded in isolation, but as part of a vast network of humans and machines, using different techniques to identify and weave links between pieces of information, gradually improving our understanding of them.

All in all, the questions I’m interested in are:

1. Do we believe semantic analysis can work?
2. Do we believe that it can reveal insights that it would be impractical for human beings to find?
3. Do we believe that those insights might be just the ones we need?
4. Is it worth us investing more in such solutions?

I’d offer a yes to each of those questions, and have had a lot of fun evaluating OpenAmplify, but: what do you think?

Many WordPress bloggers use plugins like these to help people who read their posts find other, related posts.

That’s all well and good if you only want to help readers find your own articles, but perhaps other political bloggers have made your own point better than you have? Let’s now turn that round: perhaps you’ve made another blogger’s point better than he has? Wouldn’t it be great if there was a Related Posts service that let people follow links to similar content from one blog to another, irrespective of who wrote it and where you started reading?

Poblish offers just this. Simply open a blog post from one of the 1500 feeds we monitor, and you’ll see a list of similar posts, ranked by similarity, from across all of those feeds.

If that wasn’t cool enough, the list of related posts updates automatically. So, if you create a new, collaborative article with us, you can watch the list update as your work progresses – literally as you type. That’s very useful – perhaps you make a particular point, then some articles appear that strongly refute that point. You might then reconsider, delete your last paragraph, and move on in a different vein.

I believe that tools like this are an essential part of making the political blogosphere a knowledge base, that can not only improve political blogging, but also improve policy-making.

…

(I should add, as an aside, that all these services essentially use Inverse Document Frequency algorithms. Here’s a worked example. They can work very well – Poblish’s especially, I hope, as our algorithm uses stemming and stopwords – but there’s no attempt by the computer to understand the text, or the context, so there will inevitably be howlers. These are not semantic solutions of the type I mentioned yesterday, but don’t worry: we have big plans in that area.)