Nothing about copyleft causes the "owner" to not hold the copyright on a work.

Copyright gives the holder (either the author or the party to which the copyright is assigned) a few specific (but broad) exclusive rights to the work: reproduction, preparation of derivative works, distribution, public performance (which probably doesn't so much apply to software), and public display (also not applicable to software, so much). (And then there's circumvention, but that's yucky and irrelevant to this case, so we'll ignore it.)

"Exclusive" means nobody is allowed to do any of those things except the copyright holder (unless the copyright holder licenses those rights to others, but we'll get to that.)

The copyright holder can give/sell/transfer the copyright to someone else (in which case the previous holder is now excluded from doing with the work all the things in the first paragraph above because someone else now holds all those exclusive rights), but that's not what the AGPL does.

The copyright holder can also license any or all of the exclusive rights in the first paragraph to some person or party (or in the case of an "open license" like the AGPL, to everyone).

The AGPL licenses rights like distribution and preparation of derivative works to others (under certain conditions(/covenants) like "you can only distribute copies if you do so under the same license as you got it under").

So, if some hypothetical party named "Bob" started a project, they'd hold the copyright. If Bob put the AGPL on that project and also required any contributor to assign copyright on their specific contributions to Bob, Bob would hold the copyright on the entire project code, including all contributions. Someone else could take advantage of the terms of the AGPL allowing derivative works and redistribution to create their own fork (so long as they abided by the conditions(/covenants) in the AGPL), and if they did so, they could omit on their fork any copyright assignment requirement, in which case the fork could end up owned by a mishmash of different copyright holders (making it hard to impossible for the administrator of the fork to do anything tricky like changing what license future versions were under.)

However, on Bob's original (non-fork) version, if Bob, as the copyright holder, changes the license file to something proprietary, Bob has (arguably?) created a new work that is not the same work as the previous version, and Bob can license that new version under a different license. (I suppose one might be able to argue that changing just the license file isn't legally enough to make a new version, but the very next time a nontrivial change was made to the codebase, that would qualify as a new version, so it kindof doesn't matter.) Bob has already licensed previous versions of his non-fork under the AGPL, so Bob can't really rescind that license already granted on older versions. But new versions could indeed be put under a different license. (Mind you, there are licenses that have specific terms that make them rescindable on old versions. Take for instance the Open Gaming License fiasco that WotC tried to pull not terribly long ago. But I don't think the AGPL is a license that can be rescinded.)

Since Bob can't rescind the license on older versions, if Bob made a future version proprietary, the community or any particular party that wanted to could take the last AGPL version of the non-fork and make a fork from there that remained under the AGPL.

The moral of the story is: if you don't want the copylefted code project you start to be changed to a proprietary license later, don't do any copyright assignment agreement. The codebase being owned by a diverse mishmash of different copyright holders is a feature, not a bug.

And, as mentioned elsewhere in this post, Immich is owned by a lot of different copyright holders as it has no copyright assignment requirement.