Recently, a small web app I made for the Regina Police Service has gone live along with their new website (props to Les Parker for the design and implementation!). It’s got a lot of positive feedback, so I thought I’d elaborate a bit on how I threw it together.
It started as a hackathon project at our last HackREGINA event – I was fortunate to have a bit of time to myself, and wanted to use the City of Regina data to map population data on a google map. Google had recently come out with the heatmap layer for the maps API v3, so it seemed like a great fit. Within a few hours, I had a working dataset and code hobbled together and ended up doing a quick demo of it at HackREGINA.
The City of Regina folk seemed to love the idea, and they also pointed me in the direction of the crime statistics report from the Regina Police Service. The report was delivered in a PDF format every month, which makes it a bit difficult to read using a computer. What was worse, it was a scanned document – so even if I wanted to try to get the text from the page it wasn’t there. I did attempt to use a OCR program to lift text from teh scanned images, but it didn’t prove to be very useful.
Cleaning the Data
Luckily, one of my partners in crime Les (who I worked with on Gofor) was just starting a new position in the RPS. He was able to talk to the right people and soon got me an XML file. It wasn’t the prettiest XML I’ve ever seen, but it’s computer-readable; I was able to write a small utility app that scraped the data I needed from the XML, organized it in a certain format, and output it to JSON so that my web page could use the data. Soon I had a working version of the 2011 crime stats on a heat map.
From there, we started looking at other possibilities with the data: a month-by-month breakdown, reorganizing the crime types into more usable categories, and adding some sort of date-range slider to the controls to allow users to look at specific months of data. Les and I worked together to get a more usable form of monthly crime data (the excel crime data sheets), and I changed my utility to create JSON data from CSV files. I also removed the initial City of Regina population data from the crime map version, as it didn’t really seem to fit both datasets together.
The app is setup in such a way that the data sources could easily be replaced, as long as they followed a specific format. So if someone were to supply a different “crimes.json” file, it would totally work. This unlocks potential for other police services (or other neighbourhood and time-based heatmap data) to use the app with little extra development.
Web App Design
I also improved the web app a bit – initially I was pinging Google’s geocoding service every single time we loaded the page, and for each neighbourhood we wanted to display. Fortunately there’s not a lot of movement in neighbourhoods so instead I just created a static JSON file with neighbourhood names and latitude, longitude cooridnates, avoiding the need to bug Google at all for that information and dramatically increasing the page load time.
- The Google Maps API for the map and heatmap controls,
- jQuery UI and jQRangeSlider for the date range picker,
- Twitter Bootstrap for the button and select input element styling and a bit of layout,
- Underscore for working with collections of data
- jQuery for loading ajax resources and working with elements & events on the page
If you have any feedback, suggestions, or would like to use the map utility for your own data, let me know!