Sometimes an interactive isn’t the right approach. I’ve recently been very interested in motion graphic videos. They can communicate complex ideas in a very engaging way. Plus they can look great too. I made this in Motion 5.

Watch the full video

One of the greatest developments for me lately was the introduction of Politics Insider, the Globe’s new subscriber-only blog on national politics run by online politics editor Chris Hannay. I was invited to contribute every Wednesday, using data to give a new perspective on national affairs.

I was surprised to see my name in an ad for the project. It’s a real honour appear beside the likes of André Picard and Konrad Yakabuski. I’m an ant among giants.

I made an interactive for this last year, which was nominated for a Data Journalism Award. That one took a few weeks of dabbling. I made this one in a single night after putting it off for far too long. The end product used the Fusion Tables API to dynamically query the data from the dataset. I really wish we could easily create our own database installs with PHP, but the FTAPI worked quite well.

It definitely has the highest  traffic and most engagement of any interactive I’ve made. This has less to do with the content and everything to do with it leading the home page for several hours that day. Promotion = traffic. No promotion = no traffic. Simple as that.

View the entire interactive

What a fun project this was. I contacted MLB to see if they had any data on R.A. Dickey, the Jays’ new knuckleball pitcher. Did they? Ha! They have reams of data. In a spreadsheet. And they’ll email it to me right away. And answer questions promptly. The exact opposite of almost every government source I’ve ever reached in Canada. The beauty of working with professionals.

The data had a multitude of columns, from batter name to pitch type to result and speed. But the most interesting potential came from the coordinates: it tracks where the ball leaves the pitcher’s hand and where it crosses the plate. Enough to reconstruct every pitch.

I started working on a Raphael interactive (naturally — Raphael is the very best interactive engine in the market today in my opinion… yes, better than D3!). I designed an entire top-down version that tracked the ball’s movement from pitcher to plate… but it wasn’t quite right. The arc algorithm was too complicated and the data didn’t show the knuckleball’s supposed “flutter” in flight. So I tried a front view instead, from the perspective of the batter.

People always ask how I make these things. It’s usually a lot of math, really, and a good understanding of JavaScript. I took the X and Z position of the ball in the pitcher’s hand and animated it to the position on the plate. I used several arrays to track all the balls in play so they could be filtered later. And I built a multipurpose search function that could take any variables and spit out the appropriate pitches. It’s actually one of my simplest interactives, technically speaking, but the end product is visually among my very best, I think.

And man, was it ever fun to make! My first sports data project. And it got a great reaction too. I’m so not used to this — most of my work is about politics and national news, which rarely garners excitement from the masses. If it gets attention, it’s usually criticism (but after being EIC of a student newspaper, you get very used to criticism).

For posterity’s sake, I just want to save some of these very nice comments from Twitter.

“What an insanely ambitious interactive”

“Lovely interactive for baseball fans”

“One of the coolest things I’ve ever seen”

“Unbelievably cool”

Thanks! Means a lot. And for what it’s worth, it got an hour or so on the homepage at some point. So that was nice too.

View the full interactive

This project took about a year. Really. Because we had to collect the data from a dozen sources in between other projects that kept demanding our time. But we finally got it out and it was a huge success. The incredible Rob Carrick wrote a great column to accompany the interactive. It looks at how young people today have it much worse than a generation ago. This is old hat to some people, but it’s never been dealt with so objectively before. We finally dug deep across several metrics to identify the trends.

View the full interactive

The Globe investigative team uncovered these connections between the OLG and other major casino players like MGM. We used this flowchart online to illustrate the connections.

View the full interactive

We scraped data on Canada’s Main Estimates to create this interactive chart, showing how much things changed year-to-year.

Explore the full interactive

I adapted this print graphic into an online interactive flow chart. Choose what type of investor you are and find custom advice.

View the full interactive

As part of The Globe’s new series “Reinventing Parliament,” we decided to investigate how often Members of Parliament voted against the majority of their party. The undertaking took weeks of number crunching, programming and visualizing to put together. The end product shows a clear trend: MPs rarely break ranks, with most voting along party lines more than 97 per cent of the time. But Conservative MPs are overwhelmingly more likely to break rank than members from other parties.

The project began by scraping 600 pages of parliamentary voting data. We designed a scraper in Python that trudged through these pages, scraping the MP’s name, party, riding and vote. All told, we had 162,280 votes between June 2, 2011 and Jan. 28, 2013.

The next step was determining whether the MP voted with or against their party. We first believed we could check this against the whip’s vote. But we ran into trouble when we realized the party whip was sometimes (unsurprisingly) absent. We decided to measure each MP’s vote against the majority of their party instead — a measure of 50 per cent plus one.

This was a two-step process. First, we ran another Python script on our scraped data to count the number of yeas and nays on each vote. We ran the resulting spreadsheet through another Python script to compare each MP’s vote. The script added a column to the original dataset indicating whether they voted the “same” as or “different” from the party.

This spreadsheet gave us the first usable data to measure party unity. We used Microsoft Access to run queries, starting by calculating the number of times an MP voted differently from their party. We ran another query to find the total number of votes for each MP. (While there are 600 votes total, each MP has a different attendance record.) Combined, we had a percentage score for party unity.

At each step of the process, we spot-checked and confirmed our results against the original raw data to ensure there were no irregularities. We found a few errors along the way and re-designed our queries or adjusted the Python scripts until we were confident the final data set had no errors.

Then we began the analysis. We started with a few questions. What was the most contentious vote? (It was No. 466, a motion to study the beginnings of life.) Who broke ranks most often? (By percentage, it was former NDP MP Bruce Hyer, who later became an independent MP. By raw total, it was Conservative MP James Bezan.) Did votes change with party leadership? (A few NDP MPs voted against the party before Thomas Mulcair became leader. Afterward, there were no dissenting votes.)

We supplemented this analysis with more parliamentary data. We scraped the information on each motion, which gave us more context like the purpose of the motion or who sponsored it. This made it easier to cross-reference party unity scores and explain the outliers. We also scraped information on all sitting MPs to add information like riding and title.

Finally, we created a short Python script for scraping MP photos from the Parliament website for use in the interactive. An arduous element of this step was correcting the file names for each MP, since they had to match MP names stored in the dataset so they could be dynamically added to the page when needed.

Then the process of visualizing the data began. We had three main goals with the visualization: 1) showing how many Conservatives voted against the majority of their party compared to MPs from other parties; 2) showing the high levels of party unity, despite the occasional dissenting vote; 3) allowing readers to see more detail about each motion or bill that the MP dissented on.

We settled on using D3, a JavaScript library that uses new web technologies to animate individual nodes in unique and interesting ways. The library also allowed us to re-arrange the data across several views. By colour-coding each node and separating MPs with some dissenting votes from those with none, we could immediately show the number of Conservative MPs who registered lower party unity scores.

D3 is a powerful library but it comes with one major caveat: its more advanced animations don’t display in Internet Explorer, which remains a pervasive browser among our readers. Other news organizations like the New York Times have previously not supported IE for these visualizations.

As a fallback, we decided to try Raphael, a fantastic JavaScript library that works in every browser but doesn’t have the more complex “force collision” capabilities of D3. We were able to export X and Y co-ordinates for each node and each view in the D3 visualization, giving us three sets of points from which we could build an entirely separate visualization. If a user visits the page in Internet Explorer, the code directs them through a Rapahel version of the script with animations and interactivity intact.

The end product is hopefully an engaging example of data journalism, which lets readers explore the data and become more informed about the inner workings of Parliament.

Read the full story

Explore with the full interactive

An open letter to all interested parties:

It has come to my attention that the University Students’ Council is planning to move the Gazette and its 24 editors from its 40-year location to a much smaller office currently occupied by four people. As I understand, this plan was made surreptitiously without due consultation with the Gazette, its staff, or even multi-faith stakeholders at the centre of the issue. As former editor-in-chief of the Gazette, I have many grave concerns about this plan, not least of which is the Gazette’s ability to function as a key part of campus democracy, should this plan proceed.

The publisher-newspaper relationship is a tenuous one, to be sure. As such, it’s always prudent on the publisher to be as open, transparent and accountable as possible so political influences and personal vendettas are not allowed to interfere with the newspaper’s autonomy and freedom. The decision to move from one office to another is not a direct threat on press freedoms. But when the decision is made without due consultation and fair review of the consequences, the motivations become suspect. Is the publisher acting in the best interest of the newspaper? Is the student council acting in the best interest of its constituents? A review of the available information leads to me to conclude it is not.

The decision to move offices to a significantly smaller space has several consequences likely not considered in the publisher’s calculations. As a journalist now employed at The Globe and Mail, I can only credit The Gazette, its training, its welcoming environment and its community for my success. Many young volunteers use the Gazette as a launching pad for their careers, based in large part on the community fostered inside its space. The walls emanate tradition from dozens of previous editors who are now employed in the industry around the world. A newspaper is a collaborative effort made possible by an open space that fosters creativity, sparks imagination and enables collaboration. This is necessary for several aspects of its operations, from group story meetings, to cross-section collaboration, to daily editorial board meetings, which can involve 20 or 30 editors and volunteers gathered to discuss important issues of the day.

All of this and more has been threatened by the publisher’s decision. And I suspect none of it has been duly considered. The proposed space – which contains one receiving area, three small offices and one storage space – would quell collaboration. Editorial board meetings could not function. Volunteers would not have space to work, forcing them home or elsewhere, disrupting the editor-volunteer dynamic that was so crucial to my success.

The publisher must be exceedingly careful here; decisions against the wishes of the newspaper can be easily (mis)understood as sidelong political punishments. The recent turbulence between the student government and the campus newspaper is well documented. These same politicians are now in a position to affect The Gazette. And they appear to be doing so.

This controversy requires open, productive conversations between all interested groups. The furtive decision-making process by the publisher cannot continue.

I implore Western to review these decisions now affecting its campus newspaper, which would have grave consequences on its role in campus democracy. I ask the University Students’ Council to suspend its plan until a more thorough review of consequences and alternatives can be evaluated. I ask the Gazette’s editor to work in consult with the publisher to ensure transparency and accountability on all sides.

Please proceed with much caution. The decisions you make now will affect students, journalism and press freedoms for decades to come.

Sincerely,

Stuart Thompson

Gazette Editor-in-Chief – 2010-11

Tell the USC and Western they need to conduct a full review of all alternatives and consequences before pushing through this plan. Ask them to involve all parties in a collaborative discussion.  Contact usc.gm@uwo.ca, achakma@uwo.ca and USC.president@uwo.ca

Update (Jan. 17, 2013): USC President Adam Fearnall’s responds to my letter

Support from Twitter

Sad and angry to see what the students’ council is doing to my beloved student newspaper. bit.ly/W0fceZ @uwogazette @western_usc — Ian Denomme (@IanDenomme) January 16, 2013

Scott Feschuk, career maker. MT @stephenfgordon: My first publication was in the UWO Gazette, edited by @scottfeschuk : bit.ly/W7iaLS — scottfeschuk (@scottfeschuk) January 16, 2013

Adding my name to the list of Gazette alum who endorse this open letter by former EIC @stuartathompson: stuartathompson.com/2013/01/wester… #cdnpse

— Kaleigh Rogers (@KaleighRogers) January 16, 2013

I strongly endorse @stuartathompson‘s open letter to Western U & its students’ council re @uwogazette & campus press bit.ly/13E6xBg

— Arden Zwelling (@ArdenZwelling) January 16, 2013

As a former EIC of @uwogazette, I wholeheartedly agree w/ @stuartathompson‘s open letter to @westernu and @western_usc: stuartathompson.com/2013/01/wester… — Jay Menard (@JayCMenard) January 16, 2013

The UWO Gazette deserves to be treated with respect & given space on Western U’s sprawling campus that befits its long & storied history. — City Lights Bookshop (@CityLightsLondn) January 16, 2013

So it seems the situation at my old student newspaper is getting a bit dicey: o.canada.com/2013/01/16/uni… — aaronwherry (@aaronwherry) January 16, 2013

Rebuffed from sitting in on campus paper’s editorial meetings, student council weasels try banishment. Uber-classy! bit.ly/V8lYBG — Paul Wells (@InklessPW) January 16, 2013

+1, minus “best student newspaper” guff @sproudfoot: Western’s SC shoving Gazette, best student newspaper in Cda, into a closet: angering — Andrew Coyne (@acoyne) January 16, 2013

I hope for the #westernu student council’s sake that they’re not bullying the campus newspaper @uwogazette #ldnont tiny.cc/7mn0qw

— Pat Maloney (@PatatLFPress) January 16, 2013

@ardenzwelling Everything I did at the Gazette was more important and fulfilling than anything I ever did in j-school.

— Shannon Proudfoot (@sproudfoot) January 16, 2013

I’m a @uwogazette alum who has equally loved and been critical of the paper. But supporting this is a no-brainer: stuartathompson.com/2013/01/wester…

— Mallory Daley (@mallorydaley) January 16, 2013

In total solidarity with @uwogazette bit.ly/10cErOK Thanks @stuartathompson. Don’t mess with the press.

— Ron Nurwisah (@boyreporter) January 16, 2013

Disturbing. What’s going on with @westernu student council’s desire tocontrol/move @uwogazette campus newspaper? bit.ly/10cpvAb

— Jim Rankin (@Jleerankin) January 16, 2013

The @uwogazette is one of the best student run papers in the country. @western_usc Leave one of your best services alone!!! @stuartathompson

— Fiona Simpson (@Blunt_Logic) January 16, 2013

Much of my success so far has been attributed to the @uwogazette. @westernu- please leave it alone stuartathompson.com/2013/01/wester… via @stuartathompson

— Jessica Grossman (@JessGrossman) January 16, 2013