I made a backup from my phone. When I tried logging out of Gphotos, the app did not show any photos at all, so I guess I assumed that there are currently no local photos. I was wrong. I just made a backup and found a "delete photos backup from cloud" button and it worked. Thanks.
edit: Also, my second child was born a week ago, so my brain probably/obviously suffers from too little sleep.