The Coffin Block is the three-story building in the middle of the photo. You can't tell, of course, but it was yellow. It was built in the 1830s, when the city was only a few decades old, by the same architect who designed Osgoode Hall. Back then, Front and Wellington and Church was easily one of the most important intersections in Toronto, with the harbour just a few steps to the south and the St. Lawrence Market a stone's throw away. The Coffin Block would spend 50 years at the heart of city life, home to a variety of businesses, most notably some of our earliest telegraphs companies and William Weller's stagecoach company. The stage would pull up right out front—with the Wellington Hotel conveniently located right next door—and you'd buy your tickets from the small room at the from the building (where the stripey bit is in the photo). Before you knew it you'd be in Montreal; William Weller's stagecoach held the record for the fastest trip between the two cities in the days before trains: a blistering 35 hours and 40 minutes.
The photo was taken by F.W. Micklethwaite, who was one of Toronto's most important early photographers. I'll definitely be posting more of his stuff. Just a few years after he snapped this shot, the Coffin Block was torn down so that George Gooderham, owner of the massive Gooderhan & Worts distillery, could put the Flatiron Building in its place. You can find out more about him in the post I wrote about his company, here.
You can find out more about the Coffin Block here and here. And there's a drawing of it here.