Crowd-sourcing the evaluation of Linked Open Data with TripleCheckMate

On November 16th, 2012 the DBpedia Data Quality group started a campaign for assessing the quality of DBpedia. To get the best of this effort we developed TripleCheckMate, a tool designed for crowd-sourcing the evaluation of Linked Data. Today, we are happy to announce that TripleCheckMate has been released as an open-source software (under Apache 2.0 Licence) and the code can be accessed from Github. Although the DBpedia quality evaluation campaign is now over, you can still watch a screencast of the tool here.

Some highlights of TripleCheckMate are:

  • The Google Oauth2 provider for authentication: this not only prevents spam but also helps keeping track of the users’ evaluations.
  • Multiple types of resource selection: completely random, random resource that belongs to a specific class, or manual (with SPARQL-Enabled auto-complete)
  • An easy-to-use user interface
  • An incorporated (but configurable) error classification scheme
  • Inter-rater agreements: new or already evaluated resources are assigned to users with a probability of 50%, which allows the measurement of inter-rater agreements

The tool was developed in Java, using the Google Web Toolkit framework as a front-end and a minimal back-end with Tomcat and MySQL. Future version of this tool will allow further configurations and include an embedded H2 database inside the war file for easier deployment.

Please send us your feedback either through the DBpedia mailing list or our google-group.

This entry was posted in Announcements, dbpedia, Software Releases. Bookmark the permalink.

Leave a Reply