Maybe carpenter ants. At one point I discovered them by a pile of sawdust like that on my coffee table.
It turned out there was enough of a roof leak to cause some rot despite no visible leak symptoms, the ants were following that, and happened to discard the sawdust out a tiny hole in the ceiling above my coffee table. It sucked because that was the signal to replace the roof