Workaround for Failure of Copying Many Files


 
Moving on from “Elephants Dream”

After successfully trying out both lossy and lossless encoding from the master copy of “Elephants Dreams”, I decided to carry on the experiments on the subsequent “open movies” created by Blender Foundation. One consistent nasty problem that I faced was the failure of “Copy and Paste”. I’ve used multiple storage devices/directory for data. I used to copy large files successfully before. Apparently, for unknown reasons, Windows had difficulty to copy large number of files, regardless of file size, in a comparable speed or reliability with copying a few large files. This was proven to be especially problematic when I practically worked with tens of thousands of image files. The failures eventually forced me to repeatedly restart Windows Explorer or File Explorer using task manager. I did not know what caused the failures at first. It would seem to process of copying would simply stop randomly in the process. I reckoned the process of copying might be less prone to failure if I just had to copy small number of files as many people usually did.

Contents

1. Moving on from “Elephants Dream”

2. File Manager

3. Workaround with File Archiver

4. See Also/References

File Manager

Before we discussed my workaround of the issue of “Copy and Paste”, let me talked about type of program known as file manager. A file manager provides a user interface to work with file systems. The most common operations performed on files or group of files are: create, open, edit, view, print, rename, move, copy, delete, search/find, and modify file attributes, properties, and permissions. In Windows 8, two types of applications exist. The traditional applications that exist in the desktop environments require user to understand the fundamentals of file managements in computing. The “Metro” environment, which touch-first and sandboxed, much like many other environments in the mobile operating systems, usually obscures the necessity of understanding of file managements. The integration of various online services, such as Amazon Cloud, Google, SkyDrive, within the apps, as well as the categorization of file types such as music, videos, Word documents, Excel Spreadsheets, while greatly simplify the file management process; sandboxed environment also limited the capabilities of the apps and platform; the introduction of “sharing between apps” somewhat mitigated the limitation of sandboxed environment. Ultimately, power user will have to work with file systems directly, hence favor desktop apps and desktop environment instead of the shiny new toy of “Metro”. The mainstream consumption-based environment found in mobile operating systems as well as “Metro” may lower the bar of entry of using a computing device; the fundamentals of computing and information processing cannot be changed by any user interface, eventually.

In Windows 8, the file manager is called “File Explorer”, according the shortcut name in the “%USERPROFILE%\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\System Tools” and “%USERPROFILE%\AppData\Local\Microsoft\Windows\WinX\Group2”; but its File Description, which Task Manager uses to identify running apps, is still called “Windows Explorer” as is in Windows 7. Typically, a Windows user will use the user interface provided by “File Explorer” to manage files.

Top

Workaround with File Archiver

A file archiver is a computer program that combines a number of files into one archive file, for easier transport or storage. File archiver may employ lossless data compression in their archive formats to reduce the size of the archive.

As a file archiver, File Explorer does include a rather rudimentary set of capabilities of archiving and compression. It only allows the creation of zip (one of the archive file formats) file using only the data compression algorithm of deflate. The free open-source file archiver, 7-zip, does allow the creation of archive file of different formats using different compression algorithms, including deflate, LZMA, LZMA2, PPMd, BZip2 or without compression. The lossless encoding relies on predefined algorithms to identify and eliminate redundancy. If the compression ratio is not low enough, archiving will actually result in the increase of overall file size. Compression ratio of 1 means no compression; the archive’s file size will exceed that of the original files. The speed of decoding or extraction relies on the complexity or mathematical intensity of the algorithms being used and the speed of the CPU.

For a bunch of PNG files, they have already been compressed under deflate; File Explorer cannot further compress them, thus the resulted archive would have larger file size. For such files, the support of ‘Store’ or “no compression” by 7-zip or any other third-party file archivers is useful; as smaller file size, though still larger than the original file size, can be achieved. For text files, the compression ratio different algorithms is obviously different.

How does creation of file archive help copy large number of files? As the stability of copying is apparently linked to number of files, rather than the total file size, you can just copy one file archive, rather than large number of individual files. That’s one option. Or, you can create the file archive in the same directory. Instead of going through the usual “copy and paste” process, you can extract some or all files within the file archive to the directory you designate; in other words, it directly writes in the directory you designate. Note that if you create zip file using File Explorer or other file archiver with deflate, you can actually open that zip file using File Explorer. Once that zip file is opened with File Explorer, it’s treated as if it is a folder, which it is not technically. You can then “copy” the files from the file archive; the actual “copy” of files from zip file is actually extraction or decoding.
Top

See Also/References

1. Case Study: Encoding of Elephants Dream on Jia Chie’s Space

2. File Manager on English Wikipedia

3. File Explorer on English Wikipedia

4. File System on English Wikipedia

5. NTFS on English Wikipedia

6. Zip and unzip files on Microsoft’s official web site.

7. 7-zip’s Official Web Site

8. 7-zip on English Wikipedia

Top

Advertisements
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s