The journey towards creating a Tableau Web Data Connector

I recently watched a video by Tableau/Alteryx/general guru Chris Love in which he emphasised the importance for people to not only retrospectively present their mindblowing successes at cajoling Tableau into running the universe – but also the journey leading up to that point, including any outright failures. After all, there is no shame in failing to do what one sets out to achieve; instead, what would actually be detrimental would be to refuse to acknowledge and learn from failures.

With that in mind, I considered what features of Tableau I was not at all familiar with, might be of future benefit to me, and are likely complex enough to require a “journey” rather than a double-click.

I settled on web data connectors. This a feature I know of, but never encountered either as an author or a user. I don’t have a desperate need for them right now in my day-to-day activities, but I can imagine many useful scenarios and have at least some intellectual curiosity about how they work. So here goes with post #1 on “how do I learn how to make one?”.

What do I think a Tableau web data connector is?

I was lucky enough to go to the Tableau Conference in 2014, where (I think) the concept was announced. From that, my current belief is that a web data connector is a feature of Tableau whereby you can have it directly access structured data from other websites that have a usable API.

Fitness is a common example usecase. The likes of Fitbit.com record and analyse your step count for instance, and show you a few nice charts – but they also offer an API through which you can export your data out of their website and into your analytical software to do your own custom analysis, perhaps using different techniques or adding more external data to get further insight from all the juicy information they store on ones’ activity (or lack of).

A web data connector for a site then would let you access whatever data it makes available to you in a way suitable for immediate consumption in Tableau, avoiding the need to manually export and rebuild it into a local database and so on before analysis. Hopefully it will also allow your Tableau analysis to update when new info is in the datasource.

What do I think I probably need to create a Tableau web data connector?

Web data connectors were a feature of Tableau 9.1, which I already have installed.

From my memory of the Tableau conference, I believe that web data connectors are written in Javascript, so a good working knowledge of that language is probably useful. I can’t say I have this…many years ago I did dabble in a little Javascript as it happens, as I wanted to make a matched betting calculator for a website I had to enable people to make lots of free money from bookies (a topic for another day, but yes, there is a genuine opportunity to make lots of $$$ sitting in front of the internet every day, although that sentence couldn’t sound any scammier I know!).

However that was a long time ago and I can’t really remember anything about it. Except that it’s basically a case of writing code in a text file, and I already have Notepad++ installed which I know supports syntax highlighting of Javascript – so that’s probably an acceptable tool to help me write it in.

I also know Tableau has famously good help pages on their website, so am pretty confident I can look there to learn how to do it.

I also know that many people in the Tableau community, most notably probably the Information Lab, have kindly released examples of web data connectors that one can perhaps dissect to see what it’s all about.

What API shall I try and connect to?

My criteria in picking an API that may or may not work with this were that it should:

• support XML or JSON, which I guess are the formats that Tableau web connectors most likely use.
• be free to access; I’m feeling no urge to invest financially for a first attempt.
• not require logging into with a username/password – as a) I suspect logging into an API needing authentication adds complexity, and b) should I develop something that vaguely works, I don’t really want to be responsible for anything “security-wise” for my first attempt.
• be of a subject matter I find interesting, as I have no immediate need in my career or life to do this, so it’ll have to be vaguely mentally stimulating in order for me to bother having a go at it for any length of time.

So, the winner, for now anyway, is the BoardGamesGeek API.

For the non-initiated, BoardGameGeek is a website visited by, well, the clue is in the name, and I’m afraid I am rather the target audience – although I do my level best these days to keep my habit under control. Gone are the days when the box collection was so big that it had to be stored in a cardboard mountain on the living room floor as no shelving storage system could realistically hold it. I remain, however, extraordinarily interested in the topic, even having got rid of most of them (at least as long as you don’t poke around my phone too much…believe it or not you can have a passable game of Agricola et al on an iPhone).

Back to the subject at hand: one feature of BoardGameGeek is that users can record a list of which boardgames they own. This is publicly accessible info on the normal web; see my now-minuscule collection now here.

They also offer this information via an API, which is documented here. Here for instance is the API call needed to return the info in the above boardgamegeek collection in XML: http://boardgamegeek.com/xmlapi2/collection?username=Adam121

Sometimes it takes a couple of tries to return actual data, but when it does, you get something starting like this:

Quite clearly there’s some potentially interesting data available here, but as it stands, it’s not going to slot into Tableau nicely for analysis at all without some preprocessing by other tools first. But who wants to have to load TWO analytics programs when one will do?

Therefore, I would like to build a web data connector to allow Tableau to connect to it directly nice and live.

Next steps I have in mind

1. Try using someone else’s web data connector so I can see what they look like in practice.
2. Go on Tableau’s site and see what instructions are available about building them.
3. ????
4. PROFIT! (yes, just like in South Park).