Memory Lane - Tape Backup For Linux (Recovery)

A few days after using flexbackup as a backup tool, then I've reached the next phase in the procedure; Data Recovery. In other words, how'd you want to extract the data from the tape. Being absolutely vain in tape backup, I quickly typed 'ls' to view the content of the tape. Well, I wish it was that easy.

But it was not that difficult either provided you understand some basic things about a tape. Note to self: a tape is nothing like a cd where you can easily mount and unmount. A tape is not a folder where it holds all the files and sub-folder. It is just a media that use sequential access in archiving data instead of random access method used in disk.

So I compare the requirement given to me with what Flexbackup can offers. Flexbackup has an extract feature(--extract) and it also can read(--flist) a file that has list of archives to extract. That will do it. But first I need to get the list. So using tar -itvf and mt command, I end up with this tape-list.sh:

#
#!/bin/bash
# This simple script is to create a list of files from the tape.
# This list will be use to extract - [eg. tape-extract ]
# by yoe Dec,2005

help_usage()
{
echo "Usage: $0 filename "
exit 0
}

if [ $# -ne 1 ]; then
#echo "Usage: $0 [filename] "
help_usage
exit
fi

if [ -f "$1" ]; then
echo file exist!
echo choose another filename
exit 1
fi

tape=/dev/nst0
currentdir=`echo $PWD`
now=`date +'%Y%m%d'`
tempfile="temp.$now"
h=0

/bin/mt -f $tape rewind
/bin/mt -f $tape eod
lastcount=`/bin/mt -f $tape status |grep -i file |awk '{print $2}' |tr -d "number=" |tr -d ","`
echo "There are $lastcount blocks on the tape .."
echo
echo "Preparing to create filelist $2"
echo
/bin/mt -f $tape rewind
echo "Start creating filelist $2"

cd $currentdir
while [ $h -le $lastcount ]
do
tar -itvf $tape | awk '$1 !~ /V/' | awk '{print $6}' | sed -e 's/\.\///g' | grep "." >> $tempfile
h=$((h+1))
done

cat $tempfile | sort | uniq > $2
echo "Done creating filelist $2"
echo

#remove temporary file
#echo "Clearing temp files"
#echo
rm -rf $currentdir/$tempfile
#echo DONE
EOF

eg. #./tape-list list1.txt

so every files on the tape will be listed into a file called list1.txt. So, in list1.txt I just leave which ever file I need to restore and delete the unwanted. Then here's another script to extract the files listed in list1.txt called tape-extract.sh:

#!/bin/bash
# This simple script is for extracting files from backup.
# It requires a list that can be easily created using tape-list script
# by yoe Dec,2005

help_usage()
{
echo "Usage: $0 filename "
exit 0
}

currentdir=`echo $PWD`
logdir="/usr/local/test/log/"
now=`date +'%Y%m%d'`
tape="/dev/nst0"
logfile="tape-extract.log.$now"
flex_config="/etc/flexbackup.nst0"
if [ $# -ne 2 ]; then
help_usage
exit
fi

if [ -f "$2" ]; then

echo "extracting files into $currentdir"
h=0
/bin/mt -f $tape rewind
/bin/mt -f $tape eod
#/bin/mt -f $tape bsf 1
lastcount=`/bin/mt -f $tape status |grep -i file |awk '{print $2}' |tr -d "number=" |tr -d ","`
echo "There are $lastcount blocks on the tape .."
echo
echo Preparing to extract

/bin/mt -f $tape rewind

echo "Start finding and extracting files"
echo

cd $currentdir
while [ $h -le $lastcount ]
do
#/bin/mt -f $tape rewind
#/bin/mt -f $tape fsf $h
/usr/bin/flexbackup -c $flex_config -extract -flist $2 1>> $logdir$logfile 2>> $logdir$logfile
#/usr/bin/flexbackup -extract -flist $1
#/bin/mt -f $tape rewind
#echo h: $h
#echo 2m: $m
h=$((h+1))
done

echo "Done extracting"
echo "Create logfile named $logfile in log directory"
else
echo "file does not exist!"
echo
fi
EOF

eg. #./tape-extract list1.txt

more...

less...

Memory Lane - Tape Backup for Linux

Remembering the first task given to me, to find a backup solution for server logs. Being given a Redhat 9 box (dont ask why), and a tape drive (stop asking!), I have to choose appropriate tools to get the job done. Yes, you can simply do backup using built-in commands such as tar, cpio, dump etc. But I've never done this before and I'm sort of short of time, so I needed a quick (some people called it dirty) way to do this. After numerous searh engine and reviews, I end up with flexbackup tool.


Why flexbackup? Firstly, it is flexible as it sounds. It's like a middle-man software, where you first decide what kind of archive you want to use (afio, dump, tar, cpio, star, pax, zip, lha, ar, shar) the backup device, logfiles etc. and it will take care the rest. In my case I use 'tar' as the archive type.

So I downloaded the flexbackup tarball and installed it on the machine. The tape drive (Dell PV100T) is connected to the server via SCSI interface. So on RH9, you might want to load certain module for the tape drive to be recognized.

[root@bekap]# insmod /lib/modules/2.4.20-8smp/kernel/drivers/scsi/aic7xxx_old.o
[root@bekap]# insmod /lib/modules/2.4.20-8smp/kernel/drivers/scsi/st.o

Use the mt command to check the status of the device. On linux with one tape drive, the drive may be recognized as /dev/st0(or nst0). As far as I remember, st0 and nst0 are reffering to the same device, with different condition. If you run a command using /dev/nst0, the tape will be rewinded first before the running the command. If /dev/st0 is used, the command will be run at the current location on the tape.

[root@bekap]# whatis mt
mt (1) - control magnetic tape drive operation

[root@bekap]# mt -f /dev/nst0 status
SCSI 2 tape drive:
File number=0, block number=0, partition=0.
Tape block size 0 bytes. Density code 0x25 (DDS-3).
Soft error count since last status=0
General status bits on (41010000):
BOT ONLINE IM_REP_EN

To automate almost everything(what a sys admin always do), I just need to write a simple and sluggish bash script that contains mt command to operate the tape drive, and flexbackup command to backup files/folders.
Let say I need to backup all files in directory called /var/log/msglog.
Here's what I did:

[root@bekap]# cat /usr/local/test/tape-backup-log
#!/bin/bash
# This simple script is to be run for incremental log backup
# by yoe Dec,2005

if [ $# -ne 1 ]; then
echo "Usage: $0 [full] [incremental]"
exit
fi

tape="/dev/nst0"
rew=`mt -f $tape rewind`
flex_config="/etc/flexbackup.nst0"
#rewind the tape

#backup /var/log/msglog
#echo "backup /var/log/msglog"

if [ "$1" = "full" ];
then
echo "backup full for msglog"
echo
$rew
flexbackup -c $flex_config -dir /var/log/msglog -level full &> /dev/null
$rew
echo "Done . Refer log directory for details."

elif [ "$1" = "incremental" ];
then
echo "backup incremental for msglog"
echo
$rew
flexbackup -c $flex_config -dir /var/log/msglog -level incremental &> /dev/null
$rew
echo "Done . Refer log directory for details."
else


If you want to minimize user intervention, than blow it to the cronjob

[root@bekap]# crontab -l
# after inserting new tape, rewind, erase and full backup every 1st of the month at 10:05 am
5 10 1 1-12 * /bin/mt -f /dev/nst0 rewind && /bin/mt -f /dev/nst0 erase && /usr/local/test/tape-backup-log full

# run incremental backup every thursday 11:30 pm.
30 23 * 1-12 4 /usr/local/test/tape-backup-log incremental

Finally make an appropriate schedule for tape replacement.

dirty enough?

more...

less...