December 04, 2003
Windows xcopy limitation

Just uncovered an interesting limitation in Windows' xcopy command...

I was trying to copy a large number of files from one drive to another using xcopy.

The copy had run for a while when I received an "insufficient memory" error and the process stopped.

Because I had used the /m switch to turn off the archive bit after copy, I figured I could just restart the process and it would continue.

Such was not the case. As soon as I reissued the command, I received the insufficient memory error again.

After some searching and tinkering, I discovered a freeware alternative to Microsoft's xcopy - XXCOPY.

XXCOPY significantly enhances xcopy's functionality, but mirrors the command line options which is very convenient if you already have a number of batch files with long hairy option filled command lines for xcopy (as I do).

Anyway, turns out that what was choking xcopy was a file whose fully qualified file name (i.e. the full directory path starting with the drive letter plus the file name) was longer than 254 characters - Windows' limit. Fortunately, XXCOPY was nice enough to provide a clear error message identifying the situation, as well as the option to skip these files and continue copying the rest.

Highly recommended.

Digg It!  Digg It!   del.icio.us bookmark  Bookmark it!  

Posted by David at December 04, 2003 10:01 AM | Categorized under: Reference
Comments

David,

Cool! I run a bunch of xcopy's in a DOS bat file as my nightly backup. This should work better. Thanks.

Al

Posted by: Al Kirchner on December 4, 2003 01:49 PM

David: Very cool. Question is, how can you quickly locate those files that are doing the choking if you DON'T use XXCOPY?

Tim

Posted by: Tim Jeffryes on February 16, 2004 07:30 AM

Tim: Unfortunately, xcopy dies without any sort of error message. I guess you could run xcopy in verbose mode, and redirect the console output to a file, then review the file after the copy dies. I would guess that the last filename in the output file would be the culprit, although you still wouldn't know why the copy died. Being the pragmatist that I am, I'd say just download XXCOPY since it is small, free, and easy to install, copy your existing batch files, then edit the batch files to call XXCOPY rather than xcopy.

Posted by: David on February 16, 2004 12:34 PM

This was very cool and very useful indeed. Many Thanks.
Amit

Posted by: Amit Roy on May 25, 2004 02:00 AM

You are right on the money, thanks! Also, to see which file was bombing out xcopy, simply run xcopy with the /L switch. This will list all files that would have been copied, without actually copying. This way, when you receive the error, the file giving the error will probably be near the last file listed that would have been copied.

Posted by: Jeff on June 30, 2004 10:09 AM

I was having the same problem, running xcopy with the switches: /D /E /C /I /H /R /Y

I then added the /F switch so I had: /D /E /C /I /F /H /R /Y and the insufficient memory error no longer occured. To double check I removed the /F switch, tried again and the error came back!

Hmmm...

Posted by: Ryan on July 15, 2004 07:35 PM

Looks like you can just use /n to copy using short filenames. They won't be a pretty, though they will be there if need be.


Sounded like xxcopy would also not complete the copy, that is just reports the error correctly.

Get used to stuff that works (linux) and it makes M$ shortcomings obvous!

B-)

Posted by: BadMagic on July 16, 2004 05:44 PM

Actually, in most cases there is probably a much easier way to resolve the issue. The error is occurring because the destination path for XCOPY is adding extra characters to the complete file path. Simply use the SUBST command to replace the destination path with a drive letter. That will eliminate the character limit problem.

BTW, thanks for the above info. I just came across this problem today and the above ideas would also have helped.

Posted by: Jamie C on November 10, 2004 03:24 PM

XXCOPY doesn't copy file permissions...

Posted by: Daniel on November 29, 2004 01:02 PM

I got Xcopy to run by not using the /f /h /r switches. Hope this helps.

Posted by: StuManChu on January 5, 2006 04:06 PM

I was also having the "insufficient memory" problem and found the free Windows Resource Kit download contained RoboCopy.exe which was just the ticket. Try http://www.microsoft.com/downloads/details.aspx?FamilyID=9d467a69-57ff-4ae7-96ee-b18c4790cffd&displaylang=en, or Google for a download link. Only thing I didn't like as well was that I was unable to get it to accept directory paths with spaces for exclusion (but it does accept wildcards).

Posted by: Walter on January 14, 2006 07:08 PM


Hello,

Can anyone please tell me if this email box is still being monitored? I have wasted so much time in the past posting questions, comments, and suggestions to website, only to learn that most of the sites were no longer being monitored.

For this reason, I decided to create this form letter as a means of checking to verify that the site is still active, this way, no time is wasted responding to dead email boxes.

If this site were still being monitored, a simple “Yes” would be an ample reply; I will then post my comments.

Thanks,

Chat

Posted by: Chat on February 28, 2006 07:09 PM

its still being monitored

Posted by: Michael on March 14, 2006 05:18 PM

I do not use xcopy a lot, but recently had to when I wanted to copy files newer than a certain date to another folder, and could not find my usual tool beyond compare to do that. I hit upon same problem with xcopy -- 'insufficient memory'.

On trying more, I found it happened on long path/file name. Now of course, xcopy does not give any error message to that effect. It simply jumps out saying 'insufficient memory'. So how did i find what was actual error? Use 'xxcopy'. It clearly mentions the error about long pathname as it happens, but continues processing for other files.

Posted by: Vivek Deveshwar on June 14, 2006 03:14 AM

I ran into the same issue and downloaded XXCopy as well. But then I stumbled upon another sweet utility I already had ... robocopy from the Windows Resource Kit tools. This simple command will mirror one directory from another!

robocopy f:\Download\Dev \\clarksrv\d$\systembackup\Download\Dev /MIR

Posted by: Damon Clark on July 12, 2006 11:08 AM

Oops, sorry but I noticed I posted the same thing that Walter did above. DOH!

Posted by: Damon Clark on July 12, 2006 02:24 PM

Hi,

I had a similair problem with XCOPY - Insufficient Memory error, and this was caused by one file having a huge file name, in it itself in a deep folder structure, taking it past the 256 max length.

I was struggling to fix the file (it could not be opened, renamed, moved, or deleted) but fixed it by using another PC to map to the folder concerned. I was able to rename the file from the other PC.

Beautiful!!

Posted by: martin on September 6, 2006 07:29 AM

Thanks for the Robocopy reference, ran into the XCOPY - Insufficient Memory error via xcopy /s /e /v /c /d, used robocopy and all is well. After 20 years of (occasional use) on xcopy, I've switched to robocopy
-Scott

Posted by: on September 14, 2006 12:50 PM

Had same problem today, solved it with SUBST command as suggested above. Thanks!

Posted by: Wolfgang on September 17, 2006 05:54 AM
Post a comment

Ability to add comments removed due to spam.

If you wish to add a comment, send an email to comments at blogdom dot org

replacing the at above with @ and the dot above with .