I am after a batch file or power shell script to copy data from a NAS to external HD. I normally use robocopy and a single command:
robocopy Y:\ O:\Archive\ /XD "Y:\Archive\Images" /E /COPY:DAT /NP /R:5 /W:1 /LOG:c:\mYsCRIPTS\archive_latest.log
(Where Y: is the mapped drive to NAS and Y: is the external HD and this command is run on my workstation).
Which seems to take forever? Is it possible to start multiple copies of robocopy to run in parallel so things would speed up! Each instance of robocopy would only copy selected folders!
I have seen many PowerShell scripts which seem very complicated, can anyone help?
![Svenbrp](data:image/svg+xml,<svg xmlns="http://www.w3.org/2000/svg" width="60" height="60"><rect fill-opacity="0"/></svg>)
I actually just stumbled upon the answer for your question...
https://support.zadarastorage.com/hc/en-us/articles/213024806-How-to-Run-Robocopy-in-Parallel
If the objective is to backup a directory containing many sub-directories then this will work for you.
If you are only trying to backup a few directories I would recommend using one or two multi-threaded robocopy instances.
To start another instance the simplest method is to open another prompt.
If you don't want the command to start in the current prompt, check out the start command.
ex:
start robocopy <<yourSettingsHere>>
Or in the background, without opening a new prompt window: (like linux bg)
start /B robocopy <yourSettingsHere>>
There are several other options you can specify to configure how to run the command. You can figure them out by reading the help for start by using start /?
It's as easy as trying it yourself. The error is that robocopy cannot gain access to the file because it's in use by different process, after this it waits number of seconds you set up with /w switch and retries. It seems to be safe.
If you want your script to skip if previous instance didn't finish, and don't want to check for robocopy process running because of other robocopy tasks, go for semaphore files:
if exist c:\windows\temp\mirror1.txt goto:eof echo. > c:\windows\temp\mirror1.txt robocopy ...(your params here)... del /q c:\windows\temp\mirror1.txtThis way, at the beginning of your mirroring script file mirror1.txt will be created and removed when mirroring is done. If by a chance another scheduler task will try to start, the file mirror1.txt will exist, so script will end with doing nothing.
Another option is to select an option of not starting another instance of given job while the previous didn't finish. You can find it in scheduled task properties.
Last, but not least - if you need to mirror contents of one disk to another, perhaps you consider RAID1?
Would it not be:
\\network\profile\user \\archive\USER /e /move
Robocopy will create the Folder USER if it doesn't exist.
4 Replies
![This person is a Verified Professional](data:image/svg+xml,<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24"><rect fill-opacity="0"/></svg>)
You can use the /MT[:N] switch to run a multithreaded copy operation. But if your external drive is connected via a USB 2 connection you are going to be limited to about a 20 - 30MBs transfer speed no matter what you do.
From the Microsoft documentation:
"Creates multi-threaded copies with N threads. N must be an integer between 1 and 128. The default value for N is 8.
The /MT parameter cannot be used with the /IPG and /EFSRAW parameters.
Redirect output using /LOG option for better performance."
But how do I run multiple instances of robocopy not just one with the /MT switch?
EBS Computer Services is an IT service provider.
But how do I run multiple instances of robocopy not just one with the /MT switch?
Why would you do that rather than multi-threading? MT gives the same result more efficiently.
I actually just stumbled upon the answer for your question...
https://support.zadarastorage.com/hc/en-us/articles/213024806-How-to-Run-Robocopy-in-Parallel
If the objective is to backup a directory containing many sub-directories then this will work for you.
If you are only trying to backup a few directories I would recommend using one or two multi-threaded robocopy instances.
To start another instance the simplest method is to open another prompt.
If you don't want the command to start in the current prompt, check out the start command.
ex:
start robocopy <<yourSettingsHere>>
Or in the background, without opening a new prompt window: (like linux bg)
start /B robocopy <yourSettingsHere>>
There are several other options you can specify to configure how to run the command. You can figure them out by reading the help for start by using start /?
This topic has been locked by an administrator and is no longer open for commenting.
To continue this discussion, please ask a new question.
