I've always had people say it's a must to visit him when you're in London.
Little piece of trivia here.
Most people think that Big Ben is the clock tower joined on to the Houses of Parliament. Big Ben is actually the largest bell inside the clock tower. The tower was called the Westminster Clock Tower although following the Queen's jubilee (I think it was her 60th in 2012, although it may have been the 50th in 2002) it was renamed the Queen Elizabeth Tower.
If you're in London it would seem very rude not to swing by and have a look. You can walk over Westminster Bridge (right beside the tower) to the south bank, then look across the river at the Houses of Parliament and the clock tower. I'd highly recommend it at around twilight. The views along the Thames after dark are really quite something - wherever you are in central London the Thames views are pretty good. Don't forget to go and see Tower Bridge as well.