The building at 132 Main Street has been a farmhouse, tavern, stagecoach stop, post office, hotel for travelers, and the home of noted architect Cass Gilbert. Today it is a museum. Built as a house in the early 1700’s, it is named for tavernkeeper Timothy Keeler, who turned it into an inn in 1772.

It also came to be known as Cannonball House because in the Battle of Ridgefield in 1777, the British army fired a small cannonball, which became embedded in the tavern’s north wall. The tavern today is listed on the National Register of Historic Places.