As long as the fudged details don't materially change the relationship between the anecdote and the point, I don't think it's a big deal.

Like whether it was grade 3 or 4 doesn't really matter, but if it was the teacher telling the story or a classmate recounting the tales of his summer up with his uncle in Canada and they couldn't retrieve the body because they ended up in a motorcycle fight with a band of ninjas, I'd say that's a bigger deal.

Because why would he leave out the ninja fight bit, that part is metal!