deandra.homeip.net

Quick notes for the impatient:

• Product: Dell PowerVault 124T autoloader (LTO-3)
• System: Debian Testing
• Software: tar, mtx
• Tape drive: works
• Picker: works
• Barcode reader: works
• mailslot: doesn't work
• speed: approx 60-80GB/hr (on my system)
• useful settings: 256kb buffer in st, and 256kb block size in tar

The long-winded version:
I was recently tasked with researching how well a Dell PowerVault 124T would work on a Debian system. Since I couldn't find any single good reference on the web, I decided to make one of my own.
First, a little background. I was discussing TABS with my boss one day. Specifically, we were discussing its main limitation: lack of archival storage. TABS is mainly focused on backing up as much data as quickly and as often as possible over the network, and the system is centred around to huge storage volumes. The problem is, we can only go back about a week before lost files are simply gone forever. I have mitigated this somewhat by telling rsync not to delete files that have been deleted from the server, but this does nothing to safeguard against corrupted or accidentally modified files. So, we started discussing options.
More disk was far too expensive. The storage of one server alone ran about seven grand, and that doesn't cover the server to put the data onto those disks. What we really needed was a good tape solution. I had tried to use tape before, as part of a previous backup system, and it just didn't hold up in terms of reliability or performance. However, tape is perfectly well suited for long-term archival storage. Before I knew it, my boss was asking me whether a PowerVault 124T would be suitable.
For those of you reading along for the entertainment, I'll give you a bit of background on our PV124T. It is a 2U rack enclosure housing an LTO-3 drive (LTO-2-L and DLT160 versions are also available), one 8-cartridge magazine (with space for a second), and a robotic picker with a barcode reader. Think of it like a jukebox. In a rack. That uses tapes instead of records. And doesn't play any music.
Anyway, the autoloader connects to the server using a supplied Adaptec 39160 SCSI card. This was useful, since Statler (the TABS server I used for this project,) only came with the dual-channel PERC, and that was busy handling RAID for the two arrays in the system. Installation of the SCSI card was straightforward; plug it into a 64-bit PCI slot, and reboot the server. Since I use a stock Debian kernel, the card was detected on boot, and everything was hunky-dory. A note if you compile your own kernels. The picker and the drive share the same SCSI ID, so you need to have multiple LUN support enabled in your kernel if you want to see both devices.
Once the card was installed, it was time to hook up the drive. The Right Way to do this is to bring the server down, connect the autoloader into your SCSI chain, make sure your termination is correct, set the SCSI ID of the autoloader (this is done using the supplied web interface or the control panel,) power up all your SCSI devices, and then power up the server. Of course, my experience with SCSI is rather dated, and limited, the last SCSI thing I worked on being a Mac LC-somethingorother, so I got the order wrong, and couldn't find the autoloader. Removing and reinserting the driver for the SCSI card causes a rescan of the SCSI bus, and voila! There it was! (The other option is to issue the command echo "scsi add-single-device a b c d" > /proc/scsi/scsi, where a b c d is host, bus, target, lun. However, I didn't try this. YMMV.)
Hotplug detected the devices and loaded the correct modules for them. The tape drive was at /dev/st0, since it's the only tape drive in the system, and the picker appeared at /dev/sg5. The picker can be controlled using SCSI commands, via the MTX program. You just need to know which sg device to point mtx to. You can find this out by looking at /proc/scsi/sg/device_strs. This file lists all the /dev/sgx devices, in order, although the device name itself isn't listed. the PV124T was the sixth device in the list, so that is /dev/sg5.
Now, functionality. As I said, the picker understands generic SCSI-2 commands. These commands can be generated by MTX. Hence, MTX can be used to control the picker. The only thing that doesn't work is the mailslot function. (ie., you cannot use MTX to instruct the picker to take a tape in through the mailslot, or to eject a tape through same.) However, the web-interface or front panel can be used to accomplish this, and I find it is faster to simply eject the magazine, cycle it by hand, and pull the tape I want. The mailslot is only useful if you want to load or unload a tape while the drive is in operation, and this is something I haven't yet had to do. Bottom line, the mailslot functionality is easy to work around or live without, whatever's your fancy.
Once I got the drive connected and loaded with tapes, I tried a test backup. I put the command "mtx -f /dev/changer next" into a file called nexttape.sh, and instructed tar (via the -F switch) to call that script whenever the tape got full. After that, I loaded the first tape into the drive and let'er fly. It took about eighteen hours to do the first 1.3TB backup. Dell claims this drive is capable of 288GB/hr, and I'll explain the discrepancy here.
A tape drive is operating most efficiently when it is streaming. Streaming occurs when the drive buffer never runs out, and the drive is able to lay down data onto the tape continuously. When the buffer empties, the drive must stop the tape until the buffer is full again. The problem is that the tape is moving past the head at about 293 inches per second, or about 27 kilometers per hour. The tape takes a few moments (and a few inches) to go from 293ips to zero. Thus, the drive must now backspace the tape to the point where writing stopped. Only then can it start writing again. The net effect of this is that if you can't keep your tape streaming, your performance goes right in the toilet.
After some careful tweaking of buffer sizes, both in the st driver and in tar, I found that I ran up against the performance of the array; I simply cannot read data from the array at 288GB/hr. Other articles I have read on the internet use multiple SAN devices on fibre channel streaming to the same tape simultaneously to provide that kind of data rate. That being said, the writing process completes in 18 hours, which fits well within the window I have available.
In conclusion, I give the Dell PV124T a thumbs-up. It enables me to back up tons of data without having to play tape monkey. And that's exactly what I was looking for.