533
Using rsync for backups, because it's not shiny and new (tinkerbetter.tube)
submitted 1 year ago by to c/selfhosted@lemmy.world
144 comments fedilink hide all child comments
 

You might not even like rsync. Yeah it's old. Yeah it's slow. But if you're working with Linux you're going to need to know it.

In this video I walk through my favorite everyday flags for rsync.

Support the channel:
https://patreon.com/VeronicaExplains
https://ko-fi.com/VeronicaExplains
https://thestopbits.bandcamp.com

Here's a companion blog post, where I cover a bit more detail: https://vkc.sh/everyday-rsync

Also, @BreadOnPenguins made an awesome rsync video and you should check it out: https://www.youtube.com/watch?v=eifQI5uD6VQ

Lastly, I left out all of the ssh setup stuff because I made a video about that and the blog post goes into a smidge more detail. If you want to see a video covering the basics of using SSH, I made one a few years ago and it's still pretty good: https://www.youtube.com/watch?v=3FKsdbjzBcc

Chapters:
1:18 Invoking rsync
4:05 The --delete flag for rsync
5:30 Compression flag: -z
6:02 Using tmux and rsync together
6:30 but Veronica... why not use (insert shiny object here)

you are viewing a single comment's thread
view the rest of the comments
[–] [S] 50 points 1 year ago (12 children)

Ive personally used rsync for backups for about....15 years or so? Its worked out great. An awesome video going over all the basics and what you can do with it.

  • permalink
  • fedilink
  • source
  • hideshow 12 child comments
    •
  • [–] 23 points 1 year ago (3 children)

    And I generally enjoy Veronica's presentation. Knowledgable and simple.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] [S] 19 points 1 year ago (2 children)

    Her https://tinkerbetter.tube/w/ffhBwuXDg7ZuPPFcqR93Bd made me learn a new way of looking at data. There was some tricks I havent done before. She has such good videos.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 6 points 1 year ago* (last edited 1 year ago) (6 children)

    It works fine if all you need is transfer, my issue with it it's just not efficient. If you want a "time travel" feature, your only option is to duplicate data. Differential backups, compression, and encryption for off-site ones is where other tools shine.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 4 points 1 year ago (1 child)

    I have it add a backup suffix based on the date. It moves changed and deleted files to another directory adding the date to the filename.

    It can also do hard-link copied so that you can have multiple full directory trees to avoid all that duplication.

    No file deltas or compression, but it does mean that you can access the backups directly.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 3 points 1 year ago

    Thanks! I was not aware of these options, along with what other poster mentioned about --link-dest. These do turn rsync into a backup program, which is something the root article should explain!

    (Both are limited in some aspects to other backup software, but they might still be a simpler but effective solution. And sometimes simple is best!)

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 1 year ago* (2 children)

    If you want a “time travel” feature, your only option is to duplicate data.

    Not true. Look at the --link-dest flag. Encryption, sure, rsync can’t do that, but incremental backups work fine and compression is better handled at the filesystem level anyway IMO.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 1 year ago* (last edited 1 year ago) (1 child)

    Isn't that creating hardlinks between source and dest? Hard links only work on the same drive. And I'm not sure how that gives you "time travel", as in, browsing snapshots or file states at the different times you ran rsync.

    Edit: ah the hard link is between dest and the link-dest argument, makes more sense.

    I wouldn't bundle fs and backup compression in the same bucket, because they have vastly different reqs. Backup compression doesn't need to be optimized for fast decompression.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 1 year ago

    Snapper and BTRFS. Its only adjusts changes in data, so time travel is just pointing to what blocks changed and when, and not building a duplicate of the entire file or filesystem. A snapshot is instant, and new block changes belong to the current default.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 1 year ago*

    Agree. It's neat for file transfers and simple one-shot backups, but if you're looking for a proper backup solution then other tools/services have advanced virtually every aspect of backups so much it pretty much always makes sense to use one of those instead.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 3 points 1 year ago*

    I use rsync for many of the reasons covered in the video. It's widely available and has a long history. To me that feels important because it's had time to become stable and reliable. Using Linux is a hobby for me so my needs are quite low. It's nice to have a tool that just works.

    I use it for all my backups and moving my backups to off network locations as well as file/folder transfers on my own network.

    I even made my own tool (https://codeberg.org/taters/rTransfer) to simplify all my rsync commands into readable files because rsync commands can get quite long and overwhelming. It's especially useful chaining multiple rsync commands together to run under a single command.

    I've tried other backup and syncing programs and I've had bad experiences with all of them. Other backup programs have failed to restore my system. Syncing programs constantly stop working and I got tired of always troubleshooting. Rsync when set up properly has given me a lot less headaches.

  • permalink
  • fedilink
  • source
  • parent