A colleague and I presented a 50 min talk at my first weekend event yesterday.

I counted approx. 70 people in attendance before we got to talking. The talk was well received and we even spent another 20 min or so chatting to individuals that wanted some more information.

The talk was about how CBC is taking web performance seriously, the tools we use to improve our websites performance from both the front and back ends. Slides are available in pdf format.

Barbara and I will be giving this talk again, this time at the O’Reilly Conference in June.

Julian and I had the honour of having our paper accepted by USENIX. In fact, Julian is at this years LISA conference presenting it (I’m unable to attend due to schedule conflict).

Our years together working at CBC has taught us a lot about running a News website. Specifically around dealing with our CDN (Akamai).

How do you manage that fine line between having fresh content appear on the site quickly (what News wants) and protecting the origin from the load of a breaking news event (what the SysAdmins want)?

This paper answers that question and gives you a glimpse at how we do things at CBC.

You can read the paper here.

I’m visiting my Grandparents in Montreal this Christmas. They live in a very small town so the hotel room options are quite limited.

There is only one hotel in this town and the published rate was $94 a night. I decided to hit up hotwire.ca to see if they could get me a deal.

Indeed I could! Hotwire was advertising $88.50/night. Great! So I decided to book. The final total was a little higher than I expected: $215.43. Hmm..

I went to expedia.ca and calculated the price they were offering at $94/night. Total with taxes: $215.26. Looks like expedia have them beat by $0.17.

Price from Expedia

Price from Hotwire

Price from Holiday Inn

Just for fun, I decided to look up the price with Holiday Inn directly: $215.24, a $0.02 savings over Expedia!

It all comes down to the “tax recovery charges and fees” that the individual websites charge.

The Tax recovery charge is a charge that the Hotel charges Expedia or Hotwire that are imposed by them from the government, such as: room tax, excise tax, tourists taxes, etc.. If this is the case, then why are the taxes and fees different for the same hotel room?

To me, it sounds like hotwire is moving some of the cost of the room into the “fees” category in order to display cheaper per night rates.

I’ll stick with booking with the hotel directly, thank you very much. I’m sure I can use the $0.02 that I saved over booking with expedia. Oh wait, I just did… and that’s my two cents on using websites for hotel “deals”

FYI: The hotel tax in Quebec is 14.98%. Of which it only seems that Holiday Inn is accurately representing in the “Taxes” section.

The code that runs the photo galleries on cbc.ca is powered by PHP.

This is the first time we’ve really used “dynamic delivery” to serve news content on our website since ~2004. Our stance has always been to serve static content to our users (we use the term “baked out”).

In addition to moving to a dynamic backend to deliver our Photo Galleries, we’ve also changed the way internal clients are able to include these galleries in their pages. We’ve exposed an API for internal users to use. This means that every photo gallery on cbc.ca is now powered by our “Photo Gallery API”. Previously users would have to use server side includes (SSIs) to include static content.

On average, the API get’s accessed ~1.8 million times per day. More so during peak times (for example, during Hurricane Sandy the API was getting hit as much as 2.6M times per day). Each access resulted in an average of 27KB in payload being returned to the user. So on any day, we would be moving ~460GB of xml data.

On the intial rollout of the code, the application had no notion of how to handle If-Modified-Since (IMS) requests from browsers. This meant that even if the client had a copy of the XML data that the API returned, it would always get a copy of the data from the server when they revistied the photo gallery page (or other pages that included the gallery).

This posed a few problems:

• Users are downloading data for which they already have.
• The back end application is spending time returning data that the client already has.
• Page rendering time is a little longer as the browser has to wait to finish downloading the xml and processing it.

By default PHP doesn’t know how to handle these IMS requests. It is up to the individual developer to code logic to handle these types of requests and to handle them accordingly.

Once this logic was put in place we saw some drastic changes in the volume of data being sent to our users. Now, on average we’re only delivering 12KB per hit. Resulting in  ~205GB of traffic per day, a savings of 44%.

Click for Larger

Take a look at the above graph.

The green line represents the number of hits. On November 6, you see it split into a blue and red line. This is when the new code went live. Before 100% of hits were 200 OK, now about 45% of hits are 304 Not Modified vs. 200 OK. As such, the size per hit has been reduced because returning a 304 response is much more lightweight than 200s.

We’ve eliminited all of the problems identified earlier. Users now load XML data from their local browser cache (if they have it). The backend application does not need to select ALL the data from the DB on each hit. Page rendering time is reduced as the browser does not need to wait to download content it already has.

Over a year later I’m finally able to calculate the true cost of purchasing my house in Toronto. Why did it take so long? By the time all the paperwork was done, laywer stuff complete, and the city finally sent me transfer papers for the front pad parking, it was 14 months after the closing date.

So without further delay, here is the break down:

Purchase Price: $520,000
Mortgage Insurance : $9,300
Land Transfer Tax (Ontario and Toronto): $13,000
Lawyer Fees: $1,516.68
Account Set Up (Internet): $20.00
Account Set Up (Gas): $25.00
Account Set Up (Toronto Hydro): $30.00
Front Pad Parking Transfer: $117.80
Bank Fees (For deposit checks): $13.00

These costs are what is usually bundled into what people call “closing fees”. This is because they are all due on or around the date you legally own the property (closing date). Experts say you should budget 1-2% of the purchase price towards these fees.

So on a $520,000 home, I had to pay an extra $24,022.48. Some things to keep in mind when you buy real estate in Toronto.

I was riding the bus on my way to a party when I turned to Sara and exclaimed: “Wouldn’t it be cool to ride every single TTC bus route in one day?”.

So I took it upon my self to start planning just such a route.

Some ground rules.

1. The trip must be continuous and if possible with no backtracking.
2. The following routes will be excluded:
1. 4xx Series Routes (Community Routes)
2. 14x Series Routes (Downtown Express)
3. 3xx Series Routes (All Night/Blue Line Routes)
4. The 1S Yonge Subway Shuttle
5. Additional services on the route (route numbers ending in a letter, like “A”,’B”, etc..) are treated the same as its parent route (route number with no letter suffix).
3. Entire trip must be completed in 24 hours starting at 00:00 Saturday morning to 23:59 the same day.
4. Utilization of streetcar and subway to transfer to another bus route is optional.

There are approx. 181 routes, which includes the three subway lines and 11 streetcar lines. Doing this challenge on a Saturday brings the total down to 167. The following routes will not be operating on Saturday:

• 117 – ALNESS
• 139 – FINCH – DON MILLS
• 141 – DOWNTOWN/MT PLEASANT EXPRESS
• 142 – DOWNTOWN/AVENUE RD EXPRESS
• 143 – DOWNTOWN/BEACH EXPRESS
• 144 – DOWNTOWN/DON VALLEY EXPRESS
• 145 – DOWNTOWN/HUMBER BAY EXPRESS
• 162 – LAWRENCE-DONWAY
• 1S – YONGE SUBWAY SHUTTLE
• 224 – VICTORIA PARK NORTH
• 33 – FOREST HILL
• 502 – DOWNTOWNER
• 503 – KINGSTON RD
• 508 – LAKE SHORE RD

So I’m really only going to be missing out on 5 routes by doing this challenge on a Saturday (117,139,162,224,33) as the remainder of the routes are already excluded by my rules.

Green: Saturday routes. Red: Additional weekday routes

So I currently have two problems to solve. First: how can I prove that I have travelled on all the routes? Second: Coming up with a route and schedule to follow will be difficult considering all the routing engines out there are meant to find the most optimal route.

Figuring out the solution to my second problem is easy: Just write the worlds most inefficient routing engine to ensure that I get from point “a” to point “b” using all possible bus routes. I’ll save that for Part Two of my post.

If anyone has any options for my first problem, I’m all ears. Leave a comment below with your suggestions.

The slides from my PostGIS mini-tutorial talk that I gave at PGWest 2011 are now available here.

I will be speaking on two occasions at PGWest next week.

My first talk will be a mini-tutorial about how to install and use PostGIS. This is great for anyone who is interested in learning more about GIS in general as well.

My second talk will be the same one I gave at PGCon 2011 this year in Ottawa, Canada. I’ll discuss how World Flight Planner uses PostgreSQL and PostGIS for its flight planning back end.

A copy of both presentations will be available online.

Toronto recently released water usage data for the past 10 years. Below is my analysis of the data.

Some interesting facts:

• If we were to take all of the water used in 2007, it would fill a perfect sphere with a radius of 6.3Km.
• The amount of water used in the past 10 years by the city is enough to fill lake Superior over 85% full.
• Residential consumption has dropped 11% since 2007.
• In 2010 a Toronto citizen used on average 369,200,000L (using population data from 2006).
• This doesn’t seem right. Is my math off? 924105963750 m³/2503280 people

The volume of water a city uses is hard to put into perspective. The source data uses m³ unit which is equal to 1,000L. In my charts I use the TL (Teralitre) unit, which is: 1,000,000,000,000L.

Does the amount of precipitation correlate with the amount of water used?

The answer: Yes! A good example can be found in 2007. A spike in water usage was a result of the lowest amount of precipitation in the past ten years. The inverse is true. The spike in precipitation in 2006 resulted in a decrease in water usage in the same year.

What part of the city used the most water for the past ten years?

Ward 5 (Etobicoke-Lakeshore) takes the cake.

GO Transit has a bunch of fare options for you to chose from.

GO publishes a fare schedule online, however it’s approx. 10 pages of charts and fare schedules.. They don’t mention any specific formula they use to calculate the different fare classes from the base fare.

I spent some time last night figuring it out based on the tables, and I present to you my findings:

• Half Fare: Base Fare divided by two and rounded to the nearest $0.05.
• Adult Ten-Ride: Base fare multiplied by 9.25 and rounded to the nearest $0.25.
• Student Ten-Ride: Base fare multiplied by 8.5 and rounded to the nearest $0.25.
• Child Ten-Ride: Base fare multiplied by 2.5 and rounded to the nearest $0.25.
• Group Pass: Base fare multiplied by 4 and rounded to the nearest $0.10.
• Adult Monthly Pass: Base fare multiplied by 33 and rounded to the nearest $1.00.
• Student Monthly Pass: Student Ten-Ride fare multiplied by 3.06 and rounded to the nearest $1.00.
• Child Monthly Pass: Child Ten-Ride fare multiplied by 4 and rounded to the nearest $1.00.