I mean, the panicked animals returning to their usual haunts seems plausible. The intention of setting the place on fire is to spread chaos internally before/during a traditional assault on the walls. Generally there's an element of deception and bad faith in the stories, that the animals were requested as tribute for peace and then immediately deployed as a weapon of war.