It doesn't really solve my use-case, since we do a combination of direct stream when possible, as well as local sync/cache but only for transcoding purposes. As well as local users only, with the federated sharing taking care of permissions, so that we don't transfer user information between eachother.

Jellyswarrm is certainly a lot closer than other solutions people have suggested though.