A smarter way to copy large numbers of files

Aside from the bit I did in uni, I am not much of a programmer. But from what I understand of data structures and I/O, I believe that large file transfers (as in: large numbers of files and folders, as well as large file sizes) could be better implemented.

Now, I’m not specifically talking about any operating system or platform, but I will use Windows Vista and Ubuntu Linux 9.04 as examples, as they are the latest (and supposedly smartest) versions of their respective lines.

Over the weekend I backed up my Linux laptop onto my Vista desktop so I could install the latest Ubuntu (9.04). I did that and then started the process of copying all my old files back onto my laptop.

To make the whole process easier for me, I simply backed up my entire home directory. This meant that when I went to copy it into the new installation, some common files and folders would require a conflict resolution (eg /home/owen/Desktop). Of course, I hadn’t thought about this at the time and went away for a few hours, only to come back and find the transfer stalled, waiting for me to resolve a conflict.

My question is: why are file transfers not smarter than this? Surely my 95Gb of files could have been copied faster if only there was a more intelligent way. For example: why can the list of files to be copied be some kind of linked list, as opposed to a FIFO queue? In that case, if a file conflict arises, and no response is received within some sane time, the file causing the conflict is removed from its current position in the list and placed further down, so that the next file may be copied in the background while the conflict resolution for the previous file is on hold. With this action, the bulk of non-conflicting files could then be copied in the background, thus saving time later.