The business case for Open Data

I attended a great seminar yesterday organised by the Irish Software Association on the business possibilities for open data. The room was packed and we got some very good presentations followed by an interesting panel discussion. As a result of this discussion I got a pointer to an excellent EU funded research report that there was almost always a new benefit to a government from either eliminating charges for access to public sector data or reducing the price to a cost recovery level. Similarly, a recent report from Deloitte suggests that open data will be one of the main mechanisms that will enable a recovery from our recent economic woes.

I think this is very timely that this seminar was held in the week leading up to the Dublin Science Hackday, because I am sure many of the Hacks delivered at this event will probably use some of the open public data.

Here are two of the three presentations delivered (I don't have soft copy of the slides delivered by Maurice Lynch of Nathean Technologies)

This first presentation was by Dominic Byrne and it describes some of the great initiatives being done by Fingal county council to promote open government data. I must admit I was initially skeptical when I saw the relatively limited number of feeds cataloged on the Fingal Open Data web site when it launched, but time has proven that it was the correct policy to launch with what was available and then subsequently concentrate on improving it. This is a wonderful local initiative, the only concern I have is that we must not let local initiatives like this blind us to the fact that the real power of open data is its global reach. For example, when I looked for a mobile phone application for DublinBikes, I found that there were no applications written exclusively for DublinBikes but lots of applications written for city bike rental schemes which could be configured to work with bike rental schemes in many different cities including Dublin. From looking at the web sites associated with these applications I noticed that many of them were initially developed for a specific city that the developer was familiar with, but then they soon realized that they could easily broaden the appeal of their app by making it configurable to work with similar bike rental systems elsewhere.

Open Data Business Opportunities - Fingal Open Data & Dublinked

View more PowerPoint from Fingal Open Data

This second presentation was by Jonathan Raper, whose company Placr has made a great business out of exploiting Open Data in the UK. Both during his presentation and during the subsequent panel discussion, he shared some of the lessons he learned from the struggle to promote open access to public data in the UK. One interesting story he told was about what happened when the London Transport authority realized that many applications were being developed that relied upon a feed that they had accidentally made public. Their IT department initially wanted to shutdown the feed because the popularity of the application meant that too much load was being generated on their infrastructure. However, the mayor was aghast at this suggestion because he saw that there would be a huge political backlash if these popular applications suddenly stopped working. Someone pointed out that this was really an online equivalent of the ancient legal principle of "right of way".

Irish Software Association/ Open Data Bisiness Models

View more PowerPoint from Jonathan Raper

Less than a week to go until Dublin Science Hackday

There is only a few more days left until the start of Dublin Science Hackday that its happening in Dublin City University this weekend (3rd/4th of March). This is one of the first in a series of exciting events planned for Dublin City of Science 2012 celebration. I have very good experiences of being involved in Hackday events inside IBM and so I expect that getting involved with an even wider public audience should provide even more innovation and excitement.

Initially I was planning to organise for a team comprised of IBM employees to compete in this event, but the reaction to this suggestion was not very positive. On the one hand, employees who had spend a busy week from Monday to Friday working hard on their technology projects, did not relish the prospect of spending a 36 hour stretch of their weekend working on another demanding project. Also some of the organisers felt that it might not set the right tone to have a team of experienced professionals competing against teams that are mainly comprised of enthusiastic students with little practical experience.

Instead I think I will be arranging for some IBM employees to attend the event as advisers either giving training talks on technology topics that would be useful to hackers and/or informally offering practical help to the project teams as they complete their projects.

In any case I am looking forward to an exciting weekend of technology. I saw the great projects completed at previous Hackdays held in cities such as London, San Francisco and cape Town and I know that the Dublin hackers will be just as good (if not even a little better).

Are you going to tell me or do I have to ask?

IBM is trying to encourage employees to use social media for many forms of communication which were previously done by email. As you can imagine this is getting a mixed reaction from different employees. For example Luis Suarez is leading the way in trying to eliminate use of email almost completely, but many other employees rarely use public social media sites or even the IBM internal implementation of Lotus Connections.
Recently we had a training session where the speaker was explaining how to use the various tools and enthusiastically extolling the benefits of using them. Some of the more reluctant participants raised a few issues that they had encountered and we had a lively discussion on whether these new tools actually helped solve the problem of information overload or if they actually made the problem worse because now people are expected to read a selection or blogs and keep up with activity in various Lotus Connections communities as well as reading their email.
I did some thinking about this issue afterwards and I came to the conclusion that social media tools do not inherently solve the problem of information overload, but they do transfer control of the information flow from the sender to the receiver. This is a very significant change, but in order to take advantage of this it is  necessary for the receiver to put some thought into what information they actually want to receive and then they also need to put a little bit of work into implementing a process of receiving information which works for them.
Prior to the invention of the internet, people used to communicate via physical letters and newspapers. With letters the sender used to decide whom they would send the letter to and with newspapers the reader would decide what newspapers they wanted to read. There was a cost associated with sending a letter and  also a cost to purchase a newspaper so this acted as a natural limiting factor on the amount of information that anyone received. Once the internet was invented, email replaced letters and web sites replaced newspapers.
Since the cost of sending an email is virtually zero people have a tendency to send copies of their emails to more people than just the people who really need/want to receive the information in the email and this leads to information overload. Automated SPAM filters can deal with the really obvious abusers of the email service, but it still is not feasible for most people to read all of the emails they receive each day. If you really want to be in control of your information inflow you need to establish some procedure for deciding what emails you will read (personally I like the GTD system, but I am sure there are many alternatives).
Nobody should limit their information inflows to just the information that other people decide to send them. Therefore you ought to regularly spend some time seeking out new information. Each person will find a different system that works best for them e.g. reading the daily newspaper, reading a series of favourite blogs, logging into Facebook etc. Since everyone has different preferences there is no one system that will work for everyone, but you ought to make a conscious decision about how to go about seeking "news" because the sources of information you rely upon will have an important effect upon how successful you will be in life.
Needless to say another important factor in reducing information overload is that people need to feel empowered and confident enough to tell people "no I didn't read your email". If you do not feel personally empowered to take control of your own information inflow, no tool can help you.

Security for stand-alone Java programs that use the Notes API

Since its launch in the early 1990s, Lotus Notes has had a wonderful security architecture which implemented a public/private key architecture which has only recently been widely adopted by other platforms. Unfortunately this meant that many people are not really familiar with how it works.

I recently blogged about how to write stand-alone Java programs which can manipulate Notes/Domino databases. A few people commented upon the fact that when they run their stand-alone programs on a machine on which they have a Notes client installed they are normally prompted to enter a password each time the program launches. Since they don't get a similar password prompt when they run their program on a machine which has a Domino server installed on it, they seem tro think that the Domino server code gives them some form of privelege to bypass Notes security. This is not true, but to explain why I will first need to explain some important facts about Notes/Domino secuirty model.

1. Notes implements what security experts call "two factor authentication". This means that you need to prove your identity by two different mechanisms. Firstly you need to have an ID file containing your private key and secondly you need to know the password used to secure the ID file.
2. Domino servers also have ID files to prove their identity. However, most administrators insist that their real human users choose a complex password to secure their ID file and change it frequently, but most administrators don't use any password at all on their servers' ID files. This is because otherwise they would need a human to be present to type in the password every time the server is restarted and in any case sharing the ID file's password with all of the potential administrators would negate any security benefit from having a password.

When you run a Java stand-alone program that used the Notes API on a machine which has the Notes client installed you will run with the identity of currently installed ID file (which typically requires you to type a password). When you run this same program on a machine which has the Domino server installed you will run with the identity of server's ID file (which probably doesn't require you to type a password). Because of this you will need to ensure that the server is granted the appropriate access rights to the databases your program needs to use.

Domino has the concept of "scheduled agents" which can run on a server in the background and do various maintenance tasks. If these tasks executed with the server's ID it would be necessary to give the server access rights to a lot of databases which would not be very secure. Instead Domino implements a mechanism that these scheduled agents run with the access rights of the user who signed the agent this means that each of the users can run their own version of the agent running with access rights to just their own databases.

World Radio Day highlights how a technology with a long past can also have a bright future

Monday of  this week was the officially declared by UNESCO to be  World Radio day. I was listening to the latest episode of the BBC click podcast which was devoted to this event while out jogging and I was struck by how the audio broadcasting technology has both a long glorious history and a bright future.

Here are a few facts worth thinking about:

• Radio was effectively the first world wide web. Modern radio stations tend to broadcast at frequencies that have a relatively short range so that neighbouring radio stations won't interfere with each other, but in the early days of radio stations tended to use Long Wave transmission which had a much longer range. In fact the BBC World Service has been broadcasting globally since 1932 by using a network of transmission stations strategically placed throughout the globe that would re-transmit the programs originally transmitted from London (an architecture which is strikingly similar to that of the Internet). Of course they were assisted in the early days by the existence of the British Empire, but this network is still apparently reaching and audience of almost 200 million listeners every day.
• Radio stations were probably the first  users of social media. As soon as telephones became widespread, many radio programs adapted the phone-in model whereby listeners could phone the radio station to contribute to the discussion happening in the studio. In recent years the technology has been updated to use Twitter and similar tools, but the basic idea has been popular for many years.
• Radio technology is cheap and truly ubiquitous. While many people listen to radio programs on very sophisticated and hence expensive devices, cheap radio receivers are affordable for even the poorest of people. In fact their power consumption is also so low that batter powered models can be used in remote areas where no mains electricity supply is available. In fact it is even feasible to have devices whose battery can be recharged by manually winding a handle.
• Audio broadcasts can reach people even when they are busy. For example, many people listen to the radio while preparing breakfast in the morning and almost all cars come equipped with a radio that you can use to stop yourself getting bored on long journeys. I know that some people might read blogs while driving, but this is definitely not to be recommended for safety reasons. However, listening to the radio while driving is perfectly safe.
• Modern distribution techniques like podcasting complement rather than compete with radio. I follow may different podcasts and I notice many (but not all) of the best shows are radio programs that are simply recorded and turned into a podcast with minimal effort. The skills that radio broadcasters have learned over the years enable them to produce a very high quality product and for minimal extra effort they can transform their existing radio content into podcasts that can reach a much wider audience that are outside the reach of their transmitters. In fact I know that many colleagues who are not natives of Ireland really enjoy the fact that the Internet allows them to easily keep in touch with home by listening to the local radio station from their home town.
• Audio broadcasting is a field that is open to both professionals and amateurs at the same time. While I was in secondary school, I had great fun working as a part-time DJ on our local pirate radio station. The technology we used was amazingly cheap and low-tech even by the standards of the time. The production standards were not very high and were not really capable of competing with the real professionals, but we did nevertheless manage to build up a loyal group of listeners. It is no surprise that there never was a pirate television station in the west of Ireland despite the fact that there was a clear demand for an alternative to the single station that was available at the time - the costs of setting up even a very basic television station would be several orders of magnitude higher.

When you consider all of these factors it is clear that audio broadcasting is a technology which will flourish in the years ahead even if the tools and techniques we use to produce the program and/or listen to the content will continue to evolve.

[xpost] Running stand-alone java programs that read and write Notes/Domino databases

[This post was originally posted to my IBM internal blog]

Most code which interfaces with Notes/Domino databases will run in the context of a running client or server e.g. a notes agent which runs on a schedule or in response to an event. However, there are times when you might want to run a stand-alone programs that will read and write Notes/Domino databases.

Bob Balaban (who wrote the definitive book on the Notes Java API) recommends that programmers should write agents in such a way that they can be run either as a stand-alone program or as a Notes agent because of the fact that it is so much easier to debug a Java program than to debug a Notes/Domino agent (he calls this the two headed beast). However, it is also often handy to distribute a program which modifies a Notes database in the form of a stand alone program e.g. I recently had to  write a program which tweaked the settings of a Sametime Community server - since many Sametime administrators don't even realize that they have a Domino server underneath I thought it would be easier to distribute the code in the form of a stand-alone program rather than as an agent and asking administrators to go through the complexity of installing and enabling a Notes agent.

One of the things that is complex about running a piece of Notes/Domino code is that you need a lot of context e.g what Notes ID etc. If you are running java agent on a machine which has Notes installed upon it then the environment variables will be already set up for you, but if your machine does not have Notes installed it can be tricky to figure out all of the paths and environmental variables that are needed. In particular I have often wasted a lot of time trying to get Java programs to run on Linux Domino servers.

What many people don't realize is that Domino Linux servers come with a very handy startup script in /opt/ibm/lotus/notes/latest/linux. that can do all of the hard work for you. To use this program you should invoke your program with the script, like:

/opt/ibm/lotus/notes/latest/linux/startup /LOCATION/MYTOOL

where LOCATION is the path to your tool and MYTOOL is the name of your tool. Then your tool will get invoked with all of the necessary environmental variables defined properly.

This startup script  is really handy, I am not sure why it is not more widely documented.

Remmina is the best Linux application for working with remote computers

As many people know, I have Ubuntu installed on my work laptop. However, this does not mean the bulk of my development and testing is done on this platform. The software that IBM develops runs on many different platforms and so I need to do most of my work on remote servers which are running various different operating systems.

Ubuntu comes with Vinagre pre-installed - it appears as "Remote Desktop Connect" in the system menu. This tool can connect via the RDP protocol to Windows type systems and it also speaks VNC to most other platforms. While it works, it is not exactly a joy to use so I sometimes use Tsclient to connect to Windows servers and TightVNC to connect to VNC servers. Neither of these tools is perfect either so I decided to look to to see what alternatives are available. After a bit of experimentation I hit upon Remmina as the best overall tool that can be used to connect over either VNC or RDP to remote desktops.

The main features that I like about Remmina are

• It has a lot of flexibility about resolution. As well as allowing you type in any arbitrary screen resolution you can also ask Remmina to pick a resolution that is optimized to the screen you are using.
• There are a number of Hot keys that can be used to quickly access common features e.g. change the window size to match the current resolution or visa-versa.
• It integrates the local and remote clipboard so you can easily copy text from a local application to a remote one.
• It is easy to get files to or from a Linux machine, but configuring shared drives on windows can be much more tricky. Therefore Remmina helps you by configuring your local home folder (or any other directory you specify) as a shared folder via Samba and then automatically connects to that shared folder from the remote session it establishes on a windows machine.
• It has a handy applet which integrates with the system tray and allows you to quickly connect to any of your configured servers.

However, Remmina is not perfect. The main problem is the almost total lack of documentation. The project website has the FAQ about where to find documentation "Remmina was designed to be hopefully work for most people without the aid of any documentation". Although this is not really a grammatical sentence it is close to being true, but here are a few things that it would be useful to know before you start using Remmina:

• Most UI elements in Remmina can also be accessed by a combination of keys on the keyboard. When you hover over a control in the UI with the mouse the equivalent key combination will pop-up.
• All of the hot key combinations are accessed by first pressing the ctrl key on the right of your keyboard  For example when you hover over the "Full Screen button" in the UI you are told that the keyboard equivalent is "Control_R+F". I initially thought that this meant that I should press the ctrl R and F keys at the same time, but it actually means that you need to press the right hand ctrl button and the F key at the same time.
• Because the right hand ctrl key is used by Remmina, you will often need to use the left hand ctrl key to use the applications on the remote machine. For example, most editors use the ctrl key combined with the right or left arrows to navigate through the document a word at a time. If you are editing a document in a remote window that you opened via Remmina, you need to use the left ctrl key for this type of navigation because if you use the right crtl key Remmina will interpret your ctrl arrow key presses as instructions to switch to a different active remote connection.

Fighting for our musical freedom

I listen to a lot of radio shows as podcasts on my phone. One thing that I notice is that most of the talks shows that include occasional music when broadcast live have the music cut out when posted as a podcast. I assume that the reason for this is because the radio station is worried that they don't have the rights to post the music on line.

One of the shows that I like to listen to is the Miriam Meets series on RTE. It is ironic that even when she is interviewing the performer and composer of a song on the program they still cut out the music because they don't have the rights to their own music - presumably the reason they agreed to the interview is t promote their music, so cutting the music form the show doesn't serve anyone well. On the other hand, I was particularly pleased to hear a recent episode when she interviewed singer/songwriters Paddy Casey and Declan O Rourke. They were proud of the fact that they are not signed up with a record label because of the freedom it offers them. They were even wiling to perform some of their compositions live on the radio. I think more musicians should value their freedom in this way.

I was recently experimenting with some music technology and since I am not a big Apple fan I was looking for a Linux alternative to GarageBand. A friend pointed me at the Linux Multimedia Studio. Not only is this excellent software that is easy to use, but there is also an excellent collection of sample projects available. Viewing these sample projects I am reminded of when I first encountered open source software. In much the same way that having access to the source code allows you to study how a program works and even tweak it to better meet your needs, having access to the LMMS project file allows you to study the individual  instruments used in a recording and tweak the recording so that it is more suited to your personal taste.

Imagine how cool it would be to have the "source file" for some of your favourite songs. Not only could you sing Karaoke, but you could even hear exactly how you would sound like if you played some of the drum solos along with your favourite celebrity hard rock band!

My friend Speedie, recently posted about the need to encourage coding skills among young people  ensure that they don't lose their creative skills in the digital age. While I agree with him, I also think that we have the same issue with music. Many young people think that the only way to create music is to download a track that someone else created and they never even think of creating their own musical compositions.

Modern music creation tools have become so easy to use  that there is no excuse not to express your musical creativity. Perhaps we ought to launch a MusicDojo series of events to complement the CoderDojos. I don't have much musical skills, but if any musical types want to start such a series I am more than happy to help with the technology part.

[xpost] Scribefire and Lotus Connections

[This article was originally posted to my IBM internal blog]

A while ago I wrote up a description of how to configure the Scribefire blog publishing tool to work with Lotus Connections. At that time it was necessary to manually specify the service URL and the API type, but in the meantime Lotus Connections has fixed the way that it identifies itself to tools like Scribefire so that now you simply need to provide the URL of your blog (if you are using the IBM internal deployment be sure to use the https:// version of the blog URL) and your username/password - all other parameters will automatically default to the correct values.

However, there is another issue that people will find when using Scribefire with Lotus Connections. It is not really a bug as such, but it is very annoying nevertheless.

When I configure Scribefire to publish to a blog hosted on wordpress.com, the tool automatically configures all of the blogs that I have write access to on this platform. This is actually quite a convenient feature and in any case it only takes me a few moments to delete the small number of blogs that I don't actively contribute to. However, in Lotus Connections, newly created communities get a blog by default and by default all members of the community have write access to the blog. I am a member of a huge number of communities on the IBM internal deployment of Lotus Connections (I am sure this is quite typical for most employees, especially since you can be added to the membership of a community without having to even give your proposal). As a result when I tried to add my personal blog to my scribefire configuration, I ended up with several pages of blogs added to my list of blogs on Scribefire and I had to scroll down 6 times to find the one blog that I was likely to want to publish to.

I have tried removing the blogs that I don't use from my Scribefire configuration, but this is a tedious task since I need to remove them one by one and Scribefire asks me for confirmation each time. I wonder if anyone knows an easy way to remove several blogs from my configuration?