Virtuoso and LODRefine – more vicious than virtuous pair (part 2)


there was a big black hole (BBH). And then it got bigger. Seriously. At this point in my life I realized that more I learn more is there to learn.

Failure is success if we live to learn from it.
(Malcolm Forbes and me)


Chewing through the virtuoso documentation made me realize that using /sparql endpoint for updating graphs (i.e. inserting triples into it) is not  a particularly good idea. One has to have some security, right? I said to the SPARQL user that it has the right to return error on any non-SELECT demand. I found out that there are three possibilities of securing your virtuoso endpoint:

There is good news…

Implementing  digest authentication into LODRefine was easy peasy (after I found the right library that is). I used Apache HTTP Client for this:

import org.apache.http.client.params.AuthPolicy;
import org.apache.http.NameValuePair;
import org.apache.http.auth.AuthScope;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.auth.UsernamePasswordCredentials;
  DefaultHttpClient client = new DefaultHttpClient();
	new AuthScope(endpoint.getHost(),endpoint.getPort()),
		      new UsernamePasswordCredentials(properties.get("username"), properties.get("password"))
  client.getParams().setParameter(AuthPNames.PROXY_AUTH_PREF, AuthPolicy.DIGEST);
  HttpPost httpPost = new HttpPost(endpoint.toURI());  try {
        List  nvps = new ArrayList ();
        nvps.add(new BasicNameValuePair("query", query));
        httpPost.setEntity(new UrlEncodedFormEntity(nvps));
        HttpResponse response = client.execute(httpPost);

Nice and simple, if only the virtuoso endpoint you are trying to use supports this kind of authentication – if it does, it is accessible at /sparql-auth. I had to enable it on my local virtuoso instance. I also created a new user with SPARQL UPDATE rights. For more details see Virtuoso guide.

… and there is oh-not-so-good news …

In LOD2 we decided to use WebID authentication. This will definitely not be a walk in the park. You’ll have to wait for part 3 to see, whether I survived this dangerous endeavor.

Avoiding danger is no safer in the long run than outright exposure. The fearful are caught as often as the bold. Especially if they don’t run fast enough.
(Hellen Keller and me)

Virtuoso and LODRefine – viciously virtuous pair (part 1)


I’m assuming that you :

  • are familiar with the notion of linked data (triples, SPARQL endpoints)
  • have heard about Virtuoso Universal Server,
  • want to give LODRefine a chance and,
  • don’t mind waiting for part 2 of this post.

The beginning…

They: “We want to upload triples directly from LODRefine to Virtuoso.
Me: “A wha?

… and the (almost) joke

A Virtuoso, SPARQL, and SPARUL enter the bar.
Bartender: “Hey guys, what can I get you?
They: “Select ?drink where {?drink < “Martini (Coctail)” }
Bartender (puzzled): “Gesundheit!? 

But seriously!

If you are familiar with LODRefine, then you know you can transform your data to triples using RDF extension (thanks to Fadi Richard from DERI!). Now what? You can export triples to file and then upload this file to some triple store.

Imagine you could insert triples directly from LODRefine. Guess what? This is exactly what I’m working on right now. Yay me! :)

I started with installing Virtuoso Open-Source Edition (OS X 10.7) and (fortunately) had no problems building, configuring and installing it according to the instructions. After adding SPARQL_UPDATE rights to the SPARQL user I was ready to insert some triples. And I did… insert some triples via /sparql endpoint. Lovely!

At this point (not endpoint) I learned about SPARUL (could be a name of a villain in the next Batman movie) and figured out how to delete all named graphs containing a specific substring in their name. Why I wanted to do this? Because I installed demo, sparql demo and what-not … and shortly after realized I would be much happier person without all these graphs in my virtuoso instance and that I don’t want to click on each and every one of them to remove them.

[In Vonnegut’s  Breakfast of Champions style…]

This is how you can remove all the graphs containing specific substring in IRI:

DELETE {GRAPH ?g {?s ?p ?o}} 
   GRAPH ?g {?s ?p ?o}  
   FILTER( contains(str(?g), "your substring here"))

Level 1: install instance of Virtuoso locally finished.

Next level: add “Upload data to triplestore” to LODRefine…



Enhanced by Zemanta

GitHub Pull Request frenzy or… How to reference an existing issue almost without breaking a sweat

I’m on a rampage for the last two days… sort of. Why? I’m preparing for a free webinar on using LOD-enabled Open Refine (LODRefine) and I had to fix some issues (bugs) due to migration of LODGrefine from Google Refine to OpenRefine… OpenRefine used to be called Google Refine, endorsed by Google and hosted on Google Code… but it is not anymore. Now it’s on GitHub. Yay!

Image representing GitHub as depicted in Crunc...

Image via CrunchBase

svn is dead, long live git?

On GitHub you can fork a repo with a single click. It’s like magic. But… with fork comes great responsibility… if you’re into forking and contributing and committing and pushing that is.

Another thing you can do with one click is to create a pull request. Usually you want to do this when you implemented a super duper feature or fixed a bug.  Unfortunately, what you can’t do with one click is to link your pull request with an existing issue created on the upstream repo (i.e. repo you forked from), especially not if you’re working on a branch of your fork. The thing is: each time you create a pull request from GitHub website with a click, a new issue is created. Nasty. Of course I googled for solutions. People suggested using hub. I tried it, really, I did!, but referencing issue number in pull-request from the branch just didn’t work. All I got was this lousy

 Error creating pull request: Unprocessable Entity (HTTP 422)

I tried referencing issue with its number, with full URL, using base (-b) and head (-h) parameter… nothing worked.

Wild guess: To be honest, what I didn’t try was merging my branch into forked master beforehand and then create the pull request with issue number.  Will try next time I fix something, might take a while. :)

Long story short

I had to create pull-requests to  GitHub repositories I’ve forked for some existing issues. I ended up creating pull requests with clicking and creating new issues. At least I figured out how to automatically reference an existing issue (from the new one).

The easiest way was to reference the issue in a commit message, e.g.

git commit -m "Fixed #xxx"

where xxx is issue number, obviously.

Words of wisdom after the git-battle: At the end I had  two references to the issue, so be careful what you do in your branches before you push them to remote repository… It can get ugly! :) I just spent two hours figuring out how to remove one of the commits from remote repository history so it wouldn’t show up as a second reference to the issue, but no luck, just time lost.

Another wild guess: if you add some additional text to the commit message instead of just “Fixed #xxx”, new issue is created. Will definitely test it next time. Or you do it and tell me. Please… :)


Enhanced by Zemanta

Charming Python and the mysterious case of libmysqlclient.18.dylib

I’m quite new to Python, virtualenv and other related animals on Mac, but so far I was able to create new virtual environments without much fuss… until yesterday. Two weeks ago I had to prepare an environment with MySQLdb. I installed it:

pip install MySQLdb-python

and thought “This is it”. It was definitely not. When I tried to import MySQLdb into Python, I got this error:

Error loading MySQLdb module:
MySQL_python-1.2.3-py2.7-macosx-10.7-intel.egg-tmp/, 2):
Library not loaded: libmysqlclient.18.dylib
Referenced from: /Users/mateja/.python-eggs/
Reason: image not found

What the heck? After half an hour of googling I found a solution…
I edited my .bash_profile and added this line in it (note: I do have mysql installed in /usr/local/mysql/):

export DYLD_LIBRARY_PATH=/usr/local/mysql/lib

This worked like a charm… in a console, but not in PyCharm IDE. Fortunately, this answer also provided a very helpful hint…
I had to add DYLD_LIBRARY_PATH variable to launching module configuration (Run->Edit configurations). Guess what? This time it worked like a charm in PyCharm as well :)
Note: PyCharm requires configuration per script… I have to check if there is a “global” configuration…

Anyway, go me! Viva la Stackoverflow!

Enhanced by Zemanta

Public Transport Hell: Inconsiderate People

I’m a daily commuter. I spend almost 4 hours every day on a train and believe you me… there are times when I would use a cell phone jammer myself if I had one…

Situation #1: Woman was talking to her (girl)friend on a cell through speaker! about how she cheated on her husband for almost an hour. Like we wanted to know every dirty detail…

Situation #2: A girl considered a hole train set as her private office. She made like ten phone calls. She was using a headset (earplugs) with the phone and obviously she lost a feeling how loud she’s talking. She was screaming.

Situation #3: Listening to really loud music via phone so we all have to listen to it, like it or not.

Situation #4: Eating really smelly food (like McShittyStuff or any other fast food) in a small train unit 20 minutes before departure. We had to bare with this smell till the end of the ride. The worst thing is… it doesn’t smell bad to the people eating this shit.

Situation #5: There is a bar on the train and sometimes three or four guys talk really loud and laugh like it is their last day on earth while drinking beers. And this is early 6am train, when some people would like to take a nap…

Get my point?

It doesn’t matter if you ask them politely, because… they don’t care. It doesn’t even matter if you grumble… they don’t care. I can understand why the guy decided to jam cell phone signal, someone must have stepped on his nerve. Being Batman is not the right thing to do, but hey, do we have another option?

Maybe one day I’ll throw up on someone eating bad smelling stuff. I wonder if they’ll ever eat again on a train.

Philadelphia Bus Troll Jams Cell Phone Signals When People Talk Too Loudly [Wireless]

I can understand the need for enforced cell phone silence in a variety of venues – libraries, funerals, theaters. But on a public bus in the middle of Philadelphia? Call me old fashioned, but that’s kind of a dick move.

We need this to understand how you use our service - you can take it out if you like. Cheers, your Blogspire team.



Enhanced by Zemanta

Pushing it step by step… this time ACTA

If you want to introduce a really huge change into the system, you do it slowly, gradually, step by step, so that people don’t see it coming. It’s not the musicians who will benefit from this all, but all the record houses and all the others who refuse to change themselves, their policies, their way of doing business. It’s much easier to enforce more rules and laws. It would be much better if they would instead try to introduce changes into their mental models.

We used to say “Only in America”… but we don’t anymore. It’s happening everywhere.

Amid ACTA Outcy, Politicians Don Anonymous Guy Fawkes Masks

Members of Poland’s Parliament hold up Guy Fawkes masks, symbolizing protests by Anonymous, to protest ACTA. A new photo is doing the rounds on the Internet. It shows members of the Polish parliament holding print-outs of the notorious Guy Fawkes mask in front of their faces, homage to an unofficial symbol of the Anonymous cyber collective.



Enhanced by Zemanta

Silence creeps on you

How many types of silence do you know? I know two. Awkward and non-awkward. Guess which type is in effect on on this blog… The third one, doh!
My English is a bit rusty these days… a lot of humidity in the air, you know. Also… I’m taking some classes in How-to-be-silent-with-a-style. How am I doing so far? :D

Don’t worry, next classes include How-to-write-a-blog-post-and-bore-everyone-to-death lessons! Yeeehaaaw! :mrgreen:

Need a friend? Breed one.