It absolutely is not all calorie in calorie out. Or rather, how calories are stored and burned depend on your metabolic health. There's a massive amount of literature on this.
Here's a starting point: https://www.thelancet.com/journals/lanplh/article/PIIS2542-5196%2825%2900201-3/fulltext