overview for mervincm

optimized simultaneous copy to multiple local destinations in c/datahoarder@selfhosted.forum

[–] mervincm@alien.top 1 points 11 months ago

This is an interesting option but is it suitable for a one time copy?

2

optimized simultaneous copy to multiple local destinations (alien.top)

submitted 11 months ago by mervincm@alien.top to c/datahoarder@selfhosted.forum

4 comments fedilink

I want to create a copy of my NAS hosted media folder that is about 30TB. I have a bunch of 4-8TB local (most USB3, some SATA) disks, and I would like to copy these files to the various destinations maximizing space used and time required. Since I have a 10GBE network, I can read data far faster than I can write to any of the destinations, so multiple simultaneous file copies is required at the same time to maximize this activity. doing this manually is painful. trying to select the maximum number of files that can fit (but not go over) each destination is a pain. Any thoughts on a script or an app I can use to assist here is appreciated. I want to leave the files in their native format, so I am looking for a file copy, not block-based backup etc.

Help Choosing OS in c/main@selfhosted.forum

[–] mervincm@alien.top 2 points 1 year ago

Open media vault is an excellent option as it is nothing but a aquality of line / web GUI that you install over Debian. It has an excellent docker compose extension that lets you use all those apps