Damaged Files:
When you download large files, there are multiple multi-part files (the files we dealt with before were simply a single multi-part file). To make matters worse, many times the files are damaged/incomplete (parts missing) and will not decode or are otherwise unusable. I will show you some new programs here and I will show you how to recover from many of the most common problems.

Using Newsgroups to Share a File:
OK, let's say that someone has taken a video and would like to share it with a large number of people. If the video was of high quality and high resolution and had a relatively long running time, it would not be feasible to email it to everyone. One way to distribute it is to post it on a binary newsgroup. Almost anyone with an internet connection has access to newsgroups and the video that's posted will be accessible to millions of people. Since I didn't have a video of my own to use in this tutorial, I chose to use a 30 minute clip from a movie that is part of the public domain. The name of the movie is Plan 9 From Outer Space.

Public Domain Material:
Note: Generally, when a movie or a piece of music is produced, those responsible for the piece have all legal rights to it. They determine when, where and how it's distributed. Sometimes, when a piece is produced all who had any interest in it have lost that interest either by dying or by simply allowing all legal rights to somehow expire. When this happens, the piece is considered to be in the public domain and is free to copy, reproduce and distribute. This is how all of these little streaming video sites can show all of the old movies without penalty or expense behond the required bandwidth they use. OK, enough of the public domain stuff.

Uploading a File to a Newsgroup:
Since I already covered downloading files from newsgroups, I'll begin here with the uploading process. Then we'll go through downloading again with both Outlook Express and a program named GrabIt. For this example, I'll upload to a newsgroup that was established solely for testing binary posts (alt.binaries.test). It allows you to work out any mistakes before posting to a different group. It also allows you to see how the posts will look after they've been posted.

Making the File Manageable:
The first step in posting a large video file is to break it up into smaller pieces. Some newsgroup servers have limits on the size of the files that they will/can accept. It's also a good idea to break it up in case the download is interrupted. It means that those who are downloading it won't have to start all over again. To break up the file, we need software. In this example, I chose a freeware program named HJ-split. There are other programs available but this is one I've used so I'll use it. To make this process as 'clean' as possible, I created a new folder and I placed the file I wanted to split into that folder. At this point, I have already used Nero to cut the file and generate a VCD (Video CD) image that could be burned if desired. I will probably cover Nero later (especially if I get requests for it). Below, you can see the file in an otherwise empty folder.

HJ-split is easy to find on Google. When you install most software, a new shortcut is created on the 'all programs' menu. This program is different. When you download it, you will be asked if you want to open the folder in which it was dropped. When you open that folder, you will see the hjsplit.zip folder. Right-click it and select 'extract all' (the options may be different if you've installed a third-party unzipping utility).

Click NEXT.

This allows you to choose the folder in which the files will be dropped. By default, it will create a new folder in the same folder in which the zipped folder is located. Unless you want to change the default location, click NEXT. When installing software, it's a good idea to allow it to install to the default location. If you change the default location and you go to upgrade later, you may end up with multiple copies of the same program.

If you want to view the newly unzipped (decompressed) files (you would in this case), click FINISH. If you don't want to view the files, uncheck the box and click FINISH.

This is the folder and all 'two' files. :)

To access this program more easily, we will create a desktop shortcut. Right-click on the file with the file with the .exe extension and send to the desktop. If you can't see the extension, you can change that. Go to TOOLS >> FOLDER OPTIONS >> select the VIEW tab and check the appropriate box. Remember that you have to have the 'view' set to 'details'. The view can be selecetd from either the right-click menu or the button on the toolbar of the file browser window.

Splitting the File:
Now, go to the desktop and double-click the HJ-split icon. The following window will appear. Click SPLIT.

Here, we have to select the file that we want to split. I navigated to the file, highlighted it and clicked OPEN.

Before we split the file, we have one more choice to make. We need to decide the size of the individual parts. Some people choose to make the parts 10MB. Others choose 15MB. Since 15MB is more common for the groups I visit, that's what I chose. Click START in the HJ-split window to continue.

This is what you will see while the software is splitting the file.

We're finished with this program. Click OK to return to the main HJ-split window and exit the program.

The highlighted files below are the files that were generated by the software. To reduce confusion, we will call these pieces of the original file 'chunks'.

Creating Parity/Recovery Files:
Remember that I said that some files on the newsgroups would be damaged. This can happen for a number of reasons. To make our file recoverable, we need to generate a set of recovery files. To do this, we will need another program. The name of the program that I use is Quickpar. You can find it with Google. Download and install it. Unlike HJ-split, it will create a shortcut on the START >> ALL PROGRAMS menu. It will also place a shortcut on the desktop (unless you tell it not to do so). After it's installed, run it. You will see the following window (or a similar one).

To add files, click the ADD FILES button in the window above. Then navigate to the files we previously generated and select all of the chunks (but not the original file). Now... If you're observant, you'll notice there is an option on the Quickpar window to split files. That's relatively new in QuickPar. Since I've split file with HJ-split in the past, that's the way I showed you here. If you didn't want to use HJ-split, when you have the option to add files, you could simply add only the original file and QuickPar would split it for you.

Below, there are a couple of choices that you COULD make but the defaults are fine. If you want all of the parity (recovery) files to be the same size, that's one of the choices. I chose to make them varying sizes. This makes it possible to download one of the smaller recovery files if you have only a single part of a chunk missing.

In the next image, you can see that the progress indicator is telling you that it's computing the Reed-Solomon Matrix. The Reed-Solomon algorithm is widely used for error correction. It's one of the tools used with compact discs that allows scratched discs to play. Without it, or some other error correction scheme, scratched CDs would be unusable.

Here, you would simply click EXIT to close QuickPar.

Uploading the Files:
The next step is to post the files. To do this, you can post each one manually or you can use Power-Post. You can find the program by using your best friend... Google. After you install and launch the program, you go to FILE >> ADD FILES or click the 'add files' button on the program's toolbar.

When the file window appears, navigate to the appropriate folder and select all of the split files (chunks), the recovery (par2) files and any other files you want to upload. Click OPEN to close the window.

When you close the file window, the following dialog box will pop up. There are several things you need to do. As you can see, there is a line of text that begins with (??...??). This is the subject line that will appear in the newsgroup. This line should reflect the content of the file. If you want to have the subject line to indicate the file number and the total number of files, you use the $1 and $2 variables. Wherever the $1 variable is placed, the file number will appear in the subject line. Where the $2 is placed, the total number of files will appear. The $F can not be removed from the subject line. The text box below the subject line shows you what the output will look like.

Note: In the previous paragraph, I said that the subject line should reflect the content of the file. Some people include other information such as their nickname. Whatever you do, PLEASE don't put the file count first. If you do and you do it for multiple files, you will have all of the files inter-twined and it will take lots of time to go through and pick out all of the chunks of the file. If you want all of the files you upload to be grouped together, do it in this order. Your name/nickname, a description of the file and the file count. Otherwise, it will look like the following mess:

In the following box, you select all of the newsgroups to which you want to post the files. You can add more groups by clicking the ADD GROUPS button (big surprise) at the bottom of the dialog box. Keep in mind that you should not post to more than one group at one time. Posting to multiple groups (known as cross-posting) is frowned upon by many newsgroup visitors.

Here, you can re-order the files if you wish. To move files up or down, select the file(s) and click the move up/down buttons. Generally no re-ordering is needed.

The following window allows you to generate checksum files. The files are used to determine if the downloaded files are corrupt. If you generate the error-checking files, they will be added to the list of files to be uploaded.

The following file is an SFV (Simple File Verification) file. It includes a checksum value for each of the chunks of the original file. I didn't include an SFV file in my upload. There are programs available to check files using the SFV file. Since we generated the par2 files, we don't need to use the SFV files.

The following window is the list of files that are to be uploaded. Clicking the 'post' button (the little yellow arrow).

Below, you can see the progress indicator window. It shows the upload speed, the progress of the individual files and the progress of the entire post.

This shows you that the post has completed. Notice that all of the posts have been 'accepted' by the server. It doesn't give the total upload time but it took more than 3 hours for the upload. ADSL is great for downloads is not good for large UP-loads.

If you were paying attention (and if you haven't fallen asleep), you noticed a file with an NFO extension. This is an information file. It was created in Windows Notepad. NFO files allow you to post relevant information about the uploaded files. There are other software programs that are used to create NFO files. If you upload lots of files, you may want to find one that suits your needs. In this NFO file, I only included the file information and a checksum of the original file (which could be used to verify the reconstructed file with HJ-split after the download).

Downloading the Files:
In the previous downloads we used Outlook Express. The following program is much better suited for downloading binary files. It can be used to post text messages but Outlook Express is better suited for that. In this window, we can see all of the files that were uploaded. In this program (GrabIt), you highlight the desired files and click the GRAB SELECTED button.

Here, you can see that the files have arrow icons on the left side of the subject line. Green arrows indicate that the file is currently being downloaded. Blue arrows indicate that the file in in queue to be downloaded. This image and the previous image show the list of all of the files on the newsgroup (although you can see only a few).

This next image shows only the items in the batch of files to be downloaded.

Here, you can see that there are 3 files that have been downloaded. The program is decoding the files. It only decodes one at a time.

The following image shows what the same files look like in Outlook Express. In GrabIt, the multi-part files showed up as a single file. Here, you can see all individual parts of the multi-part files.

Notes:

Recovering from Corrupt/Incomplete Files:
In some cases, you will simply have a single text file (as generated when Power-Post uploaded the files) missing or you may have an entire chunk missing. If you only have a single text file missing, the chunk will be considered incomplete. When you try to reassemble the original file (using HJ-split's JOIN function) from the chunks (which were reassembled when GrabIt was 'decoding' the files), an incomplete or missing file will not allow the reassembly. To repair the damaged file using QuickPar, you simply double-click on any of the par2 files. This will open QuickPar.

In this first image, you can see the program working. It's simply checking to see that all of the files are in-tact.

This shows you that all of the downloaded files are OK.

In this next example, I renamed one of the files. QuickPar noticed the error and is ready to rename the file. To do so, click RENAME.

Here, I removed one of the files, QuickPar found that one file was missing and that there were enough recovery blocks to perform the repair. Clicking REPAIR will start the process.

Working....

And... Viola. It's done.

In this next image, you can see all of the recovery files and a few of the chunk files. You cna see that the recovery files are varying sizes. If you are missing only a few parts of a chunk, you will need only a few recovery blocks. If you're missing a full chunk, you will need a recovery file (or multiple recovery files) that are equal in size to the missing chunk.

In the next image, you can see that I've removed one complete chunk and several of the recovery files. Now, there are not enough recovery files to recover the missing chunk. If there were some more recovery files on the server, you could go grab them. If there were no more, you could ask (nicely) the poster of the original message to either upload more recovery files or upload the missing chunk.

Above, if you look closely, you can see that there is a little yellow area at the end of the green bar. This indicates the available recovery blocks. When you have a missing chunk and you have enough recovery blocks, the yellow area would go all of the way to the right end of the bar. As of now, it only goeas a fraction of the way meaning that there are insufficient files to recover.

Well, we're almost there. Now we need to reassemble the file into the original NRG file. To do this, we will again use HJ-split. Use the desktop icon to again open the program. Here, we select JOIN.

We click INPUT FILE to open the file browser window.

GrabIt buries the download folder fairly deeply in the hierarchical structure. I clicked the arrow on the side of the address bar to show you exactly how deep it is. You can set the download folder to be in another location or you can create a shortcut. Or... You may simply want it to be difficult to find.

This shows the file that needs to be selected (the first one of the series). Click OPEN to close the file browser window.

This shows the input and output files. Click Start to begin the reassembly.

Joining in progress.

Done.

Now, there are a couple of things we can do to be sure that the downloaded file matches the uploaded file. Since we uploaded the file, we can compare the two files.

After pressing START above, you will see a 'working' progress bar. If the files are identical, you will see the following dialog box.

You will see something like this if the files are not equal.

If we download a post file that someone else posted, we will not have the luxury to compare the downloaded file to the original file. This is where the checksum values come in to play. The NFO file has the checksum value from the original. If we use the checksum feature of HJ-split to create a checksum from the downloaded file, we can see if they're the same. We again have to navigate to the downloaded file in the grabIt folder.

This is the checksum value of the downloaded file.

If you compare it to the value I generated from the original, you can see that they match. It's amazing that we can transfer millions of bytes of information thousands of miles and recover them and the files match exactly.

I know that this looks like a nightmare to try to do but if you already have the original file, it will only take a few minutes (depending on the speed of your computer) to go through all of the steps (not counting upload and download times).

If anyone actually reads this and thinks I need to break up this page into smaller pages, email me. You can also email me to let me know how many times you fell asleep while reading it. :)

You May Be Interested in My Other Sites All Sorts of Stuff: This site was started for pages/information that didn't fit well on my other sites. It includes topics from backing up computer files to small engine repair to 3D graphics software to basic information on diabetes. Basic Macro Photography: This site introduces you to macro photography. Macro photography is nothing more than the photography of small objects. It can take quite a while to understand the limitations associated with this type of photography. Without help, people will struggle to get good images. Understanding what's possible and what's not possible makes the task much easier. If you need to photograph relatively small objects (6" in height/width down to a few thousandths of an inch), this site will help. Air Rifle Links and Demos: If you're interested in air rifles, this site will introduce you to the types of rifles available and many of the things you'll need to know to shoot accurately. It also touches on field target competition. There are links to some of the better sites and forums as well as a collection of interactive demos. Basic Car Audio Electronics: This site is for those who want to install or update their car stereo. It begins with the most basic electronics theory, progresses into basic transistor theory as well as covering virtually everything associated with car audio.