Robocopy - All data marked as modifiedBackup with Mercurial and Robocopy?How to backup 20+TB of data?Robocopy Copy “For Loop”RoboCopy A File From The Web?Robocopy and IIS log date modified behaviorrobocopy cannot find pathrobocopy wierd behavior for /copyallHow to reduce the IO Wait Time during a rsync backup?Copy single file using robocopyInvoke Powershell Script Async? (Robocopy)
Transform the partial differential equation with new independent variables
Is this story about US tax office reasonable?
Is there an evolutionary advantage to having two heads?
Should I use n only, b only, bg, bgn, or gn?
How to properly maintain eye contact with people that have distinct facial features?
Why did this prime-sequence puzzle not work?
Leading and Suffering Numbers
The Passive Wisdom (Perception) score of my character on D&D Beyond seems too high
What are the benefits of cryosleep?
Do you play the upbeat when beginning to play a series of notes, and then after?
Were pen cap holes designed to prevent death by suffocation if swallowed?
Can the Help action be used to give advantage to a specific ally's attack (rather than just the next ally who attacks the target)?
Black-and-white film where monster/alien gets fried
Can a wire having a 610-670 THz (frequency of blue light) AC frequency supply, generate blue light?
How feasible is the Delta-Glider?
Infinitely many hats
Future enhancements for the finite element method
Different PCB color ( is it different material? )
How to capture more stars?
A Mathematical Discussion: Fill in the Blank
If a massive object like Jupiter flew past the Earth how close would it need to come to pull people off of the surface?
Question about exercise 11.5 in TeXbook
Plot exactly N bounce of a ball
Crossing US border with music files I'm legally allowed to possess
Robocopy - All data marked as modified
Backup with Mercurial and Robocopy?How to backup 20+TB of data?Robocopy Copy “For Loop”RoboCopy A File From The Web?Robocopy and IIS log date modified behaviorrobocopy cannot find pathrobocopy wierd behavior for /copyallHow to reduce the IO Wait Time during a rsync backup?Copy single file using robocopyInvoke Powershell Script Async? (Robocopy)
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
We make use of the following script to backup a folder (quite large - approx. 400GB), to a NAS with EXT4 Filesystem:
robocopy "Source_Directory" "\NASDestinationDirectory" /MIR /FFT /COPYALL /W:5 /r:10 /log:"C:RoboTestrobotest.log"
Research indicated that the /FFT switch is required to make up for time differences; however the issue still persists and all files are marked as modified.
Any help on this would be greatly appreciate.
backup powershell windows-server-2012-r2 batch-file batch
asked Jan 25 '17 at 7:39
joebegborg07joebegborg07
4191619
add a comment |
We make use of the following script to backup a folder (quite large - approx. 400GB), to a NAS with EXT4 Filesystem:
robocopy "Source_Directory" "\NASDestinationDirectory" /MIR /FFT /COPYALL /W:5 /r:10 /log:"C:RoboTestrobotest.log"
Research indicated that the /FFT switch is required to make up for time differences; however the issue still persists and all files are marked as modified.
Any help on this would be greatly appreciate.
backup powershell windows-server-2012-r2 batch-file batch
asked Jan 25 '17 at 7:39
joebegborg07joebegborg07
4191619
Fyi, for me the /FFT worked in the past. but it was from ntfs to cifs. So it might be related to the EXT4? Maybe do some tests to another fs type?
– willemdh
Jan 25 '17 at 18:16
Yup, agree. Doesn't always behave as expected on non-NTFS volumes.
– Simon Catlin
Feb 1 '17 at 22:09
1
@SimonCatlin Any suggested solution ?
– joebegborg07
Feb 2 '17 at 8:43
add a comment |
We make use of the following script to backup a folder (quite large - approx. 400GB), to a NAS with EXT4 Filesystem:
robocopy "Source_Directory" "\NASDestinationDirectory" /MIR /FFT /COPYALL /W:5 /r:10 /log:"C:RoboTestrobotest.log"
Research indicated that the /FFT switch is required to make up for time differences; however the issue still persists and all files are marked as modified.
Any help on this would be greatly appreciate.
backup powershell windows-server-2012-r2 batch-file batch
asked Jan 25 '17 at 7:39
joebegborg07joebegborg07
4191619
We make use of the following script to backup a folder (quite large - approx. 400GB), to a NAS with EXT4 Filesystem:
robocopy "Source_Directory" "\NASDestinationDirectory" /MIR /FFT /COPYALL /W:5 /r:10 /log:"C:RoboTestrobotest.log"
Research indicated that the /FFT switch is required to make up for time differences; however the issue still persists and all files are marked as modified.
Any help on this would be greatly appreciate.
backup powershell windows-server-2012-r2 batch-file batch
backup powershell windows-server-2012-r2 batch-file batch
asked Jan 25 '17 at 7:39
joebegborg07joebegborg07
4191619
asked Jan 25 '17 at 7:39
joebegborg07joebegborg07
4191619
asked Jan 25 '17 at 7:39
joebegborg07joebegborg07
4191619
asked Jan 25 '17 at 7:39
joebegborg07joebegborg07
4191619
asked Jan 25 '17 at 7:39
joebegborg07joebegborg07
4191619
4191619
Fyi, for me the /FFT worked in the past. but it was from ntfs to cifs. So it might be related to the EXT4? Maybe do some tests to another fs type?
– willemdh
Jan 25 '17 at 18:16
Yup, agree. Doesn't always behave as expected on non-NTFS volumes.
– Simon Catlin
Feb 1 '17 at 22:09
1
@SimonCatlin Any suggested solution ?
– joebegborg07
Feb 2 '17 at 8:43
add a comment |
Fyi, for me the /FFT worked in the past. but it was from ntfs to cifs. So it might be related to the EXT4? Maybe do some tests to another fs type?
– willemdh
Jan 25 '17 at 18:16
Yup, agree. Doesn't always behave as expected on non-NTFS volumes.
– Simon Catlin
Feb 1 '17 at 22:09
1
@SimonCatlin Any suggested solution ?
– joebegborg07
Feb 2 '17 at 8:43
Fyi, for me the /FFT worked in the past. but it was from ntfs to cifs. So it might be related to the EXT4? Maybe do some tests to another fs type?
– willemdh
Jan 25 '17 at 18:16
Fyi, for me the /FFT worked in the past. but it was from ntfs to cifs. So it might be related to the EXT4? Maybe do some tests to another fs type?
– willemdh
Jan 25 '17 at 18:16
Yup, agree. Doesn't always behave as expected on non-NTFS volumes.
– Simon Catlin
Feb 1 '17 at 22:09
Yup, agree. Doesn't always behave as expected on non-NTFS volumes.
– Simon Catlin
Feb 1 '17 at 22:09
1
1
@SimonCatlin Any suggested solution ?
– joebegborg07
Feb 2 '17 at 8:43
@SimonCatlin Any suggested solution ?
– joebegborg07
Feb 2 '17 at 8:43
add a comment |
1 Answer
1
active
oldest
votes
After lots of trial and error I think I found the reason: the Archive attribute.
When Robocopy comes across a file with the Archive attribute set, it will systematically display the file as "Modified" in its logs and count the filesize toward the total amount of data copied to destination (Copied column). However this is misleading, because unless the file was really modified (timestamp is more recent, for example), Robocopy doesn't really copy the file and skips it instead.
Try it with a large file. Add a big file to your source tree, one that would take some time to transfer to the destination server. Run Robocopy the first time: it should mark the file as "New File" in the logs, and take some time to complete the transfer. Next, make sure the Archive bit is set on that file using the attrib +a <filename> command. Launch Robocopy once more. You should find that it will be listed as "Modified", but notice time that this second run is much faster: Robocopy didn't really transfer the file again over the network. Then, at last, remove the Archive bit with attrib -a <filename> and run Robocopy one last time. It will skip the file as expected.
Unfortunately I could not find this behavior documented anywhere, even on Microsoft's own documentaion pages. (If someone does, please let us know!)
The solution is to either ignore files marked as "Modified" in the logs, or run attrib -a /s on your source files before running Robocopy.
Several people have reported this problem on the Microsoft forums.
answered Dec 8 '18 at 19:06
jcharaouijcharaoui
262110
add a comment |
Your Answer
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "2"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f828467%2frobocopy-all-data-marked-as-modified%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
After lots of trial and error I think I found the reason: the Archive attribute.
When Robocopy comes across a file with the Archive attribute set, it will systematically display the file as "Modified" in its logs and count the filesize toward the total amount of data copied to destination (Copied column). However this is misleading, because unless the file was really modified (timestamp is more recent, for example), Robocopy doesn't really copy the file and skips it instead.
Try it with a large file. Add a big file to your source tree, one that would take some time to transfer to the destination server. Run Robocopy the first time: it should mark the file as "New File" in the logs, and take some time to complete the transfer. Next, make sure the Archive bit is set on that file using the attrib +a <filename> command. Launch Robocopy once more. You should find that it will be listed as "Modified", but notice time that this second run is much faster: Robocopy didn't really transfer the file again over the network. Then, at last, remove the Archive bit with attrib -a <filename> and run Robocopy one last time. It will skip the file as expected.
Unfortunately I could not find this behavior documented anywhere, even on Microsoft's own documentaion pages. (If someone does, please let us know!)
The solution is to either ignore files marked as "Modified" in the logs, or run attrib -a /s on your source files before running Robocopy.
Several people have reported this problem on the Microsoft forums.
answered Dec 8 '18 at 19:06
jcharaouijcharaoui
262110
add a comment |
After lots of trial and error I think I found the reason: the Archive attribute.
When Robocopy comes across a file with the Archive attribute set, it will systematically display the file as "Modified" in its logs and count the filesize toward the total amount of data copied to destination (Copied column). However this is misleading, because unless the file was really modified (timestamp is more recent, for example), Robocopy doesn't really copy the file and skips it instead.
Try it with a large file. Add a big file to your source tree, one that would take some time to transfer to the destination server. Run Robocopy the first time: it should mark the file as "New File" in the logs, and take some time to complete the transfer. Next, make sure the Archive bit is set on that file using the attrib +a <filename> command. Launch Robocopy once more. You should find that it will be listed as "Modified", but notice time that this second run is much faster: Robocopy didn't really transfer the file again over the network. Then, at last, remove the Archive bit with attrib -a <filename> and run Robocopy one last time. It will skip the file as expected.
Unfortunately I could not find this behavior documented anywhere, even on Microsoft's own documentaion pages. (If someone does, please let us know!)
The solution is to either ignore files marked as "Modified" in the logs, or run attrib -a /s on your source files before running Robocopy.
Several people have reported this problem on the Microsoft forums.
answered Dec 8 '18 at 19:06
jcharaouijcharaoui
262110
add a comment |
After lots of trial and error I think I found the reason: the Archive attribute.
When Robocopy comes across a file with the Archive attribute set, it will systematically display the file as "Modified" in its logs and count the filesize toward the total amount of data copied to destination (Copied column). However this is misleading, because unless the file was really modified (timestamp is more recent, for example), Robocopy doesn't really copy the file and skips it instead.
Try it with a large file. Add a big file to your source tree, one that would take some time to transfer to the destination server. Run Robocopy the first time: it should mark the file as "New File" in the logs, and take some time to complete the transfer. Next, make sure the Archive bit is set on that file using the attrib +a <filename> command. Launch Robocopy once more. You should find that it will be listed as "Modified", but notice time that this second run is much faster: Robocopy didn't really transfer the file again over the network. Then, at last, remove the Archive bit with attrib -a <filename> and run Robocopy one last time. It will skip the file as expected.
Unfortunately I could not find this behavior documented anywhere, even on Microsoft's own documentaion pages. (If someone does, please let us know!)
The solution is to either ignore files marked as "Modified" in the logs, or run attrib -a /s on your source files before running Robocopy.
Several people have reported this problem on the Microsoft forums.
answered Dec 8 '18 at 19:06
jcharaouijcharaoui
262110
After lots of trial and error I think I found the reason: the Archive attribute.
When Robocopy comes across a file with the Archive attribute set, it will systematically display the file as "Modified" in its logs and count the filesize toward the total amount of data copied to destination (Copied column). However this is misleading, because unless the file was really modified (timestamp is more recent, for example), Robocopy doesn't really copy the file and skips it instead.
Try it with a large file. Add a big file to your source tree, one that would take some time to transfer to the destination server. Run Robocopy the first time: it should mark the file as "New File" in the logs, and take some time to complete the transfer. Next, make sure the Archive bit is set on that file using the attrib +a <filename> command. Launch Robocopy once more. You should find that it will be listed as "Modified", but notice time that this second run is much faster: Robocopy didn't really transfer the file again over the network. Then, at last, remove the Archive bit with attrib -a <filename> and run Robocopy one last time. It will skip the file as expected.
Unfortunately I could not find this behavior documented anywhere, even on Microsoft's own documentaion pages. (If someone does, please let us know!)
The solution is to either ignore files marked as "Modified" in the logs, or run attrib -a /s on your source files before running Robocopy.
Several people have reported this problem on the Microsoft forums.
answered Dec 8 '18 at 19:06
jcharaouijcharaoui
262110
answered Dec 8 '18 at 19:06
jcharaouijcharaoui
262110
answered Dec 8 '18 at 19:06
jcharaouijcharaoui
262110
answered Dec 8 '18 at 19:06
jcharaouijcharaoui
262110
262110
add a comment |
add a comment |
Thanks for contributing an answer to Server Fault!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fserverfault.com%2fquestions%2f828467%2frobocopy-all-data-marked-as-modified%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Fyi, for me the /FFT worked in the past. but it was from ntfs to cifs. So it might be related to the EXT4? Maybe do some tests to another fs type?
– willemdh
Jan 25 '17 at 18:16
Yup, agree. Doesn't always behave as expected on non-NTFS volumes.
– Simon Catlin
Feb 1 '17 at 22:09
1
@SimonCatlin Any suggested solution ?
– joebegborg07
Feb 2 '17 at 8:43