Building the City of Regina Crime Map

Recently, a small web app I made for the Regina Police Service has gone live along with their new website (props to Les Parker for the design and implementation!). It’s got a lot of positive feedback, so I thought I’d elaborate a bit on how I threw it together.

Humble Beginnings

It started as a hackathon project at our last HackREGINA event – I was fortunate to have a bit of time to myself, and wanted to use the City of Regina data to map population data on a google map. Google had recently come out with the heatmap layer for the maps API v3, so it seemed like a great fit. Within a few hours, I had a working dataset and code hobbled together and ended up doing a quick demo of it at HackREGINA.

The City of Regina folk seemed to love the idea, and they also pointed me in the direction of the crime statistics report from the Regina Police Service. The report was delivered in a PDF format every month, which makes it a bit difficult to read using a computer. What was worse, it was a scanned document – so even if I wanted to try to get the text from the page it wasn’t there. I did attempt to use a OCR program to lift text from teh scanned images, but it didn’t prove to be very useful.

Cleaning the Data

Luckily, one of my partners in crime Les (who I worked with on Gofor) was just starting a new position in the RPS. He was able to talk to the right people and soon got me an XML file. It wasn’t the prettiest XML I’ve ever seen, but it’s computer-readable; I was able to write a small utility app that scraped the data I needed from the XML, organized it in a certain format, and output it to JSON so that my web page could use the data. Soon I had a working version of the 2011 crime stats on a heat map.

From there, we started looking at other possibilities with the data: a month-by-month breakdown, reorganizing the crime types into more usable categories, and adding some sort of date-range slider to the controls to allow users to look at specific months of data. Les and I worked together to get a more usable form of monthly crime data (the excel crime data sheets), and I changed my utility to create JSON data from CSV files. I also removed the initial City of Regina population data from the crime map version, as it didn’t really seem to fit both datasets together.

The app is setup in such a way that the data sources could easily be replaced, as long as they followed a specific format. So if someone were to supply a different “crimes.json” file, it would totally work. This unlocks potential for other police services (or other neighbourhood and time-based heatmap data) to use the app with little extra development.

Web App Design

I also improved the web app a bit – initially I was pinging Google’s geocoding service every single time we loaded the page, and for each neighbourhood we wanted to display. Fortunately there’s not a lot of movement in neighbourhoods so instead I just created a static JSON file with neighbourhood names and latitude, longitude cooridnates, avoiding the need to bug Google at all for that information and dramatically increasing the page load time.

I use a few specific javascript libraries, mostly for convenience:

Since it’s been released, we’ve got lots of great feedback and suggestions, most of which we’ve implemented: a “top 5 neighbourhoods” list, shortcut buttons for specific date ranges, and a total count & legend. The application itself isn’t the cleanest javascript code I’ve written – all the code is inline, it’s all hobbled together jquery & underscore calls, and I’m still using tables. But it gets the job done.

If you have any feedback, suggestions, or would like to use the map utility for your own data, let me know!