Microsoft's OneDrive spots your mates, remembers their faces, and won't forget easily
(www.theregister.com)
However, it is that seemingly arbitrary three-times-a-year limit applied to the People section that is most concerning. Why not four? Why not as many times as a user wants?
Possibly because deleting or recreating the data is resource-intensive on the servers. It might actually be a good sign that Microsoft really removes the data, not just mark it inactive, when you turn the feature off.