While Glyn and I have been discussing the new MSc course between ourselves and with others, we have repeatedly come up with the same issues and themes, again and again. As a planning exercise earlier in the summer, we gathered some of these together into a ‘manifesto’.
The manifesto is online on our main ‘Computational Journalism’ website with a bit of extra commentary.
I’ve just put an archive online containing GeoJSON and topoJSON for UK boundary data. It’s all stored on Github, with a viewer and download site hosted on Github pages.
The data is all created from shapefiles released by the Office of National Statistics, Ordnance Survey and National Records Scotland, all under the Open Government and OS OpenData licences.
In later posts I’ll detail how I created the files, and how to use them to create interactive choropleth maps.
As I was eating lunch this afternoon, I spotted a conversation between @JoeReddington and @MySociety whizz past in Tweetdeck. I traced the conversation back to the beginning and found this request for data:
I'm looking for a dataset that shows which UK constituencies are in which NHS trust areas. Anyone?
— retired account (@retired10072015) September 14, 2014
I’ve been doing a lot of playing with geographic data recently while preparing to release a site making it easier to get GeoJSON boundaries of various areas in the UK. As a result, I’ve become pretty familiar with the Office of National Statistics Geography portal, and the data available there. I figured it must be pretty simple to hack something together to provide the data Joseph was looking for, so I took a few minutes out of lunch to see if I could help.
Checking the lookup tables at the ONS, it was clear that unfortunately there was no simple ‘NHS Trust to Parliamentary Constituency’ lookup table. However, there were two separate lookups involving Output Areas (OAs). One allows you to lookup which Parliamentary Constituency (WPC) an OA belongs to. The other allows you to lookup which NHS Clinical Commissioning Group (CCG) an OA belongs to. Clearly, all that’s required to link the two together is a bit of quick scripting to tie them both together via the Output Areas.
First, let’s create a dictionary with an entry for each CCG. For each CCG we’ll store it’s ID, name, and a set of OAs contained within. We’ll also add an empty set for the WPCs contained within the CCG:
import csv
from collections import defaultdict
data = {}
# extract information about clinical commissioning groups
with open('OA11_CCG13_NHSAT_NHSCR_EN_LU.csv', 'r') as oa_to_cgc_file:
reader = csv.DictReader(oa_to_cgc_file)
for row in reader:
if not data.get(row['CCG13CD']):
data[row['CCG13CD']] = {'CCG13CD': row['CCG13CD'], 'CCG13NM': row['CCG13NM'], 'PCON11CD list': set(), 'PCON11NM list': set(), 'OA11CD list': set(),}
data[row['CCG13CD']]['OA11CD list'].add(row['OA11CD'])
Next we create a lookup table that allows us to convert from OA to WPC:
# extract information for output area to constituency lookup
oas = {}
pcon_nm = {}
with open('OA11_PCON11_EER11_EW_LU.csv', 'r') as oa_to_pcon_file:
reader = csv.DictReader(oa_to_pcon_file)
for row in reader:
oas[row['OA11CD']] = row['PCON11CD']
pcon_nm[row['PCON11CD']] = row['PCON11NM']As the almost last step we go through the CCGs, and for each one we go through the list of OAs it covers, and lookup the WPC each OA belongs to:
# go through all the ccgs and lookup pcons from oas
for ccg, d in data.iteritems():
for oa in d['OA11CD list']:
d['PCON11CD list'].add(oas[oa])
d['PCON11NM list'].add(pcon_nm[oas[oa]])
del d['OA11CD list']Finally we just need to output the data:
for d in data.values():
d['PCON11CD list'] = ';'.join(d['PCON11CD list'])
d['PCON11NM list'] = ';'.join(d['PCON11NM list'])
with open('output.csv', 'w') as out_file:
writer = csv.DictWriter(out_file, ['CCG13CD', 'CCG13NM', 'PCON11CD list', 'PCON11NM list'])
writer.writeheader()
writer.writerows(data.values())Run the script, and we get a nice CSV with one row for each CCG, each row containing a list of the WPC ids and names the CCG covers.
Of course, this data only covers England (as CCGs are a division in NHS England). Although there don’t seem to be lookups for OAs to Health Boards in Scotland, or from OAs to Local Health Boards in Wales, it should still be possible to do something similar for these countries using Parliamentary Wards as the intermediate geography, as lookups for Wards to Health Boards and Local Health Boards are available. It’s also not immediately clear how well the boundaries for CCGs and WPCs match up, that would require further investigation, depending on what the lookup is to be used for.
All the code, input and output for this task is available on my github page.
While we negotiate the transition from the old house which we’ve sold to the new house we’ve just bought we’ve been renting a lovely flat up on Penarth head. One of the main benefits of this flat is the glorious view over Cardiff Bay and to the city centre beyond. No matter what time it is, whenever I pass by the living room window I end up staring out across the city. During the day, there’s boats coming and going through the barrage locks, or into the docks proper. At night the city is lit up with a terrible orange urban glow that somehow looks both peaceful and exciting. I’ve spent a lot of time just stood on the balcony watching, and it’s been quite relaxing. Not only that, but I’ve had the opportunity to see some fairly interesting occurrences; especially when there’s been an unusual visitor to Cardiff docks, such as this tall ship we had visiting earlier in the year:
This was the case again this evening, when we were able to stand and watch the warships of various flags and types leaving Cardiff docks after the conclusion of the NATO summit in Newport. Leaving aside any particular feelings about militarisation, it is still genuinely interesting to see these things in your home city, even more so when you’ve got a good view.
Unfortunately despite still being up on the hill overlooking the city, the new house does not have such a commanding view of the docks, bay, or Cardiff. Losing that is one of the worst things about having to move. I guess I’ll just have to get used to putting my shoes on and leaving the house whenever I want to stare out over the bay…
Last week I had a lovely time down in Winchester with m’colleague, attending The Graphical Web 2014. This year the theme was ‘Visual Storytelling’, so I’d gone along to see what new things we could learn about visualisation to include in the MSc in Computational Journalism. We’d also already had a few conversations about the course with people who were going to be at the conference, so we were planning to take the opportunity to chat in person about their involvement.
There were many excellent informative and entertaining talks, ranging from the process behind the redesign of Google Maps, through how Twitter does data visualisation, and on to what happens when your data visualisation becomes immensely popular. I’d highly recommend anyone with an interest in any of this to take some time to look through the schedule and watch the videos of some of the talks - I’ll certainly be forcing the MScCompJ students to watch a few.
There were some interesting messages from people at the conference that I’ll be taking forward with my own work and trying to impart to the students. One that is key, I think, is to strike the right balance between detail and simplicity when presenting data. This was mentioned several times throughout the conference, but it really is important. Too much information in your visualisation and you can alienate the reader and confuse or hide your message. Not enough information and the context is lost, and the use of the design to the more advanced reader is reduced. It’s one of those balancing acts that we find so often when trying to mix both people and computers. Attempting to solve this problem and find this balance is challenging and interesting, and I look forward to seeing how the students next year cope with it.
Overall, it was a really good conference. I met a number of interesting people, found a whole set of new people to follow on Twitter, and returned to Cardiff excited about the year ahead.
Previously I’ve written about a little script to download all the category icons from Foursquare and to create many different coloured versions of them. I’ve recently had to do this again for a project, and found my previous script did not work with a recent API version. I’ve updated the script to fix it and put it up on github.
Last week I gave a quick lightning talk at UnifiedDiff - a local tech meetup here in Cardiff. The main point of the talk was to try and recruit more industry involvement for our new MSc in Computational Journalism - preferably by getting some web developers and software engineers in to give guest lectures on the tools, languages and processes they use.
The talk went well, and I’ve had several offers from people wanting to get involved and add some real value to the course, which is brilliant. Of course, there’s always room to add more, so if you’re interested in coming and talking to our students, get in touch!
If you’re interested, the slides from the talk are here
On the 25th May 2014, after many years of procrastination and denials that we would ever get married, Lisa and I finally tied the knot. It was a superb day, plenty of fun was had by all, and I thought it was worth writing a little bit about some of the companies, suppliers and individuals who helped make it so good. If you’re planning a wedding in Shrewsbury or Shropshire, you can do far worse than to ask these guys for their help:
The ceremony venue was in what is almost the ‘default’ venue for civil ceremonies in Shrewsbury: Shrewsbury Castle. You can’t use the word ‘fantastic’ enough when you’re talking about the castle - it really is a great venue. We got married there, my mother got married there recently, and our friends got married there before her. It’s limited in terms of the numbers you can have in the ceremony itself, but that really didn’t bother us, and so the castle was the obvious choice. Ian (the castle custodian) was really helpful, from our first contact onwards. He was always available to answer questions, and on the wedding day he worked really hard to help everything go smoothly. Such a lovely, friendly chap, we knew we could relax about the ceremony as he had everything under control.
The reception venue was Drapers Hall, a great restaurant just around the corner from Shrewsbury Castle. When we visited home in 2013 to look for venues, I thought this was kind of a strange option - it wasn’t anything like the more traditional hotels, country houses and renovated barns that we were looking at. In the end, I think it was that fact that made it perfect. We spoke to Nigel, who runs the restaurant, about our wedding, and from the moment he started talking about food, describing the kind of thing he could make for the wedding breakfast, and then for the evening reception, we were sold. Because he wasn’t a hotel trying to sell us a ‘package’ deal, we were able to tailor everything to our own tastes and needs. The venue itself is amazing; a great old Shrewsbury building, full of history, and decorated with a mix of old and new that works so well.
The food was glorious - all our guests got a choice of starters and main courses, and they all tasted and looked amazing. The staff were great, very helpful, always on hand without being overbearing, and nothing was ever too much trouble, all evening. We had a dedicated contact all day, who introduced himself to Lisa as she was wandering the halls the evening before the wedding (unaccustomed to having nothing to do) by saying “Hi, I’m Tim. I’ll be your guy for the wedding day”. He totally was our guy; whenever we needed something, he was there to help. He kept us up to date with plans and timing, and managed the whole event to help it go off smoothly. The rest of the staff were also consistently brilliant, doing everything from re-making a playlist on spotify at short notice, to dealing with some idiot (me) knocking over the celebration cake in the middle of the evening party. The rooms in the hotel are glorious, and there are only six of them, so you can restrict who you invite to stay over! I was so pleased with the whole event, it was really good fun.
Again, not really my domain, but after visiting every wedding dress shop in the world in Cardiff and Shrewsbury, and conducting 3 months research into what did and didn’t suit, Lisa went with Hayley J. Obviously I can’t really comment on the process, having not been involved, and I haven’t even met Hayley herself, but Lisa assures me that the whole thing was done extremely well. The story she tells is that she had sort of decided what she wanted, but couldn’t find anything that ticked all of the boxes, or anything that fit properly. She was only visiting Hayley on the off chance, expecting that a custom made dress would be far too expensive. Within about thirty seconds of meeting her, Hayley had described exactly what Lisa wanted without even asking what she was looking for in a dress, and had quoted a more reasonable price than any of the ‘off the peg’ dresses Lisa had been considering. A couple of fitting sessions later, and Lisa had the most lovely wedding dress ever. I think she looked amazing in it, and I know she was delighted.
Originally, I was going to hire suits for myself and the groomsmen. However, I wanted to wear an everyday three-piece suit for the wedding, I wasn’t really interested in going with a morning suit as might be more traditional. Then we actually looked into hiring normal style suits, and they were pretty awful. Plus, hiring suits is expensive. So, with only a couple of months to go before the wedding, I told my groomsmen and family they were on their own, and I wasn’t hiring them suits. Instead, I was going to take the suit budget and blow it all on me, getting a suit tailor made. I checked a couple of places online, but then went with Martin David. I turned up there expecting to be told there was no way that they could make a suit in time, but actually they said they could do it in seven weeks. Again, the guys there were friendly and helpful, they explained all the choices and decisions well, and I felt totally happy all the way through the process. Plus, it was nowhere near as expensive as I thought it would be. I ended up with a really classic looking three piece suit, that will last me well for years.
Again, a Lisa thing. I think that, as with her own dress, she tried every shop in the world for Bridesmaids dresses, before deciding on ordering some through Wedding World in Shrewsbury. As with all the people we met while organising our wedding, they were extremely friendly, knowledgeable and helpful. The dresses arrived on time, and they even dealt with a couple of small manufacturing problems quickly and efficiently.
I’m pretty sure that Frank Painter & Sons is probably the best choice for hiring a wedding car in Shrewsbury, but we hired them because of a pun. I mean, their cars are ace, and their prices are reasonable, but it was the pun that sold it to us. While out touring wedding venues, we bumped into a couple of their drivers who were sat outside a hotel while the ceremony happened inside. We mentioned we were getting married, and asked if they had a card we could take away so we could call them later. At some point, we mentioned that we were only looking for one car, for Lisa, and one of the drivers said “well, you could always have one each. You know, his and hearse!” It was hilarious. As long as you know that Frank Painter & Sons is also a funeral directors. Which you didn’t. But you can see why we hired them, eh?
Oh the music. We agonised for ages about the music. Then at almost the last minute, we booked the Hot Jazz Biscuits, and I am so glad we did. They’re actually run by an old school friend, although I didn’t realise it at the time. We’d heard some good things about them, and their videos online looked pretty good, so not really knowing what we wanted, we went with them. They were awesome. They turned up on time, were set up and ready to go when the party needed to start, and they played wonderful music all evening. We didn’t really have a plan on how many sets we wanted, or when they should play, so they just took care of it themselves. We didn’t even have a first dance planned (we just couldn’t decide) so we told them to play something and we’d dance to it! That’s how we ended up with Van Morrison’s Moondance as our first dance. The music was good, and we danced all night, which it turns out is exactly what I was looking for. They even did a few encores for us, even though they’d played for longer than we’d paid them for. A superb band that kept the party going all night.
This is a bit irrelevant in a post about a Shrewsbury wedding, as we didn’t use photographers from Shrewsbury, or even Shropshire, but they were ace so they deserve a mention too. We used the lovely Caroline and Ian from weheartwedding, based in Cardiff. Lisa met them at a wedding fair, they were nice, and they were happy to travel as long as we paid for fuel. Professional and lovely all the way through the day, I can’t recommend them enough. We got the photos through just a few days ago, and they’re all brilliant (most of the photos in this post came from them). Choosing which ones we want to print out and display is going to be a really hard task indeed…
