I don't usually buy anything, but I enjoy having a wander round. I just like the novelty, and whilst the museum exhibits also provide novelty, the gift shop offers a different kind of novelty.
Occasionally I buy souvenirs for friends. I have one friend who collects fridge magnets, so I often find myself passively on the lookout for one that they'd like. I've also found that sending postcards to friends is a nice way to stay in contact with them (I often find myself too overwhelmed by life to stay in contact more meaningfully, so postcards are how I remind them that I have not forgotten that they exist).
The only item I can remember buying for myself from a museum was a gorgeous tea towel from the People's History Museum in Manchester, UK. It had a print of an old worker's rights poster, similar in style to this one from the women's suffrage movement. It's far too nice to actually use as a teatowel, but I was a student at the time, so I hung it up on my wall like a mini tapestry.