| Author: | Francesco Poli |
|---|---|
| Contact: | invernomuto@paranoici.org |
| Version: | 0.39 |
| Copyright: | Expat license |
| Notice: | Copyright (c) 2007-2023 Francesco Poli Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
| About this document | |
|---|---|
| Web form | HyperText Markup Language |
| Source form | reStructuredText |
| Web stylesheet | Cascading StyleSheets |
| Build directives | Makefile |
Contents
In another document (HTML, reST) you saw how to install and configure some web browsers on our example Debian testing workstation/desktop box. Now you should set up a good backup mechanism.
Install the following package to allow regular users to mount USB mass storage devices (e.g.: USB disks, USB sticks, ...):
# aptitude install pmount
Now, when you insert, say, a USB flash memory stick into a USB port, you can see which device name it was assigned (by looking at recently created /dev/sd* files or by reading log messages with dmesg); then you can mount the right partition with a command like:
$ pmount -t vfat sdc1 usbstick
or even:
$ pmount sdc1 usbstick
if the filesystem can be figured out automatically. This command (issued by any regular user in the plugdev group) will create the /media/usbstick mount point and mount the /dev/sdc1 USB stick partition on it.
When you're done reading from and/or writing to your USB stick, you can unmount it:
$ pumount usbstick
This command will unmount the device and automatically discard the /media/usbstick mount point.
If the USB memory stick has no partitions and/or filesystems, you should create at least one partition in it (by using, e.g., cfdisk) and then create a filesystem in it (choose FAT32, if you want to maximize interoperability with other operating systems):
# /sbin/mkfs -t vfat /dev/sdc1
The above mentioned command belongs to the following package:
# aptitude install dosfstools
The above described procedure works well for unknown hotpluggable devices that you want to interactively mount and unmount. Things get a bit less practical, when you want to mount a hotpluggable device in an automatic way (say, in a script). This is the case for a USB backup disk: if you want to mount it in a backup script, figuring out which device name it gets assigned is not easy.
Luckily, you can recognize your backup disk by its serial number.
Firstoff, plug your USB disk into a USB port, and see which device name it was assigned (say /dev/sdc, with all its partitions, such as /dev/sdc1). Then, issue the following command:
# blkid /dev/sdc1 /dev/sdc1: UUID="4dba4f42-f510-46d4-8e19-2ef98b481311" TYPE="ext4" PARTUUID="bbe4d1ce-01"
or, alternatively:
$ ls -l /dev/disk/by-uuid/ | grep sdc1 lrwxrwxrwx 1 root root 10 Nov 9 17:10 4dba4f42-f510-46d4-8e19-2ef98b481311 -> ../../sdc1
You can then mount this partition with the following command:
$ pmount /dev/disk/by-uuid/4dba4f42-f510-46d4-8e19-2ef98b481311 backup
and then unmount it with:
$ pumount backup
Install the following package:
# aptitude install rdiff-backup
Create a file system on the backup disk (if not yet done):
# mkfs -t ext4 /dev/sdc1
Create a handy symlink for the backup disk device:
# blkid /dev/sdc1 /dev/sdc1: UUID="4dba4f42-f510-46d4-8e19-2ef98b481311" TYPE="ext4" PARTUUID="bbe4d1ce-01" # ln -s /dev/disk/by-uuid/4dba4f42-f510-46d4-8e19-2ef98b481311 \ /etc/backuprsync.device
and prepare the backup subdirectory:
# pmount /etc/backuprsync.device backup # mkdir /media/backup/`hostname` # pumount backup
Now, create the following include/exclude file:
# cat /etc/backuprsync.include - **lost+found - /proc/** - /sys/** - /dev/** - /run/** - /lib/init/rw/** - /tmp/** - /cdrom/** - /media/*/** - /media/backup - /mnt/** - /etc/gshadow - /etc/gshadow- - /etc/shadow - /etc/shadow- - /etc/ssh/ssh_host* - /etc/ppp/*secret* - /var/backups/gshadow.bak - /var/backups/shadow.bak - /var/lib/urandom/random-seed - /var/lock/** - /var/run/** - /var/tmp/** + /home/*/backup - /home/*/**
and place the backup script in the appropriate cron directory (e.g.: for a weekly backup, use /etc/cron.weekly):
# cat /etc/cron.weekly/backuprsync
#!/bin/sh
# BackupRSync - simple backup script using rdiff-backup
VERSION="2.3"
#
# Copyright (c) 2003-2023 Francesco Poli <invernomuto@paranoici.org>
#
# Permission is hereby granted, free of charge, to any person obtaining
# a copy of this software and associated documentation files (the
# "Software"), to deal in the Software without restriction, including
# without limitation the rights to use, copy, modify, merge, publish,
# distribute, sublicense, and/or sell copies of the Software, and to
# permit persons to whom the Software is furnished to do so, subject to
# the following conditions:
#
# The above copyright notice and this permission notice shall be included
# in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
# IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
# CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
# TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
# SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
# define commands
PMOUNT="/usr/bin/pmount"
SLEEP="/bin/sleep"
GREP="/bin/grep"
RDIFFBACKUP="/usr/bin/rdiff-backup"
DF="/bin/df"
PUMOUNT="/usr/bin/pumount"
ALLCOMMANDS="$PMOUNT $SLEEP $GREP $RDIFFBACKUP $DF $PUMOUNT"
# define bit bucket
DEVNULL="/dev/null"
# define device for backup disk
DEVICE="/etc/backuprsync.device"
# define mount point for backup disk
BACKUP_TO="backup"
# define subdirectory where backup will be stored
SUBDIR=`hostname`
# define include/exclude list file
INCLUDE="/etc/backuprsync.include"
# define give-up file
GIVE_UP="/etc/backuprsync.nobackup"
# define no-cleaning file
NO_CLEAN="/etc/backuprsync.noclean"
# define old snapshot keep time
CLEAN_OLDER_THAN="25D"
echo "BackupRSync version $VERSION"
echo
echo -n "Checking if needed commands are present... "
for COMMAND in $ALLCOMMANDS
do
if test ! -x $COMMAND
then
echo "no"
echo "$COMMAND is not present or has wrong permissions"
exit 13
fi
done
echo "yes"
echo
echo -n "Checking if backup is desired... "
EXITSTATUS=0
if test -e $GIVE_UP
then
echo "no"
EXITSTATUS=1
else
echo "yes"
fi
if test $EXITSTATUS = 0
then
echo -n "Checking if backup disk is plugged in... "
if test -e $DEVICE
then
echo "yes"
else
echo "no"
EXITSTATUS=2
fi
fi
if test $EXITSTATUS = 0
then
echo -n "Checking if include/exclude list file is readable... "
if test -r $INCLUDE
then
echo "yes"
else
echo "no"
EXITSTATUS=3
fi
fi
if test $EXITSTATUS = 0
then
echo -n "Checking if cleaning is desired... "
if test -e $NO_CLEAN
then
echo "no"
CLEANING="no"
else
echo "yes"
CLEANING="yes"
fi
echo -n "Mounting backup filesystem... "
OUTPUT=`$PMOUNT $DEVICE $BACKUP_TO 2>&1`
$SLEEP 4
FSPROPS=`$PMOUNT | $GREP $BACKUP_TO`
if echo $FSPROPS | $GREP rw > $DEVNULL
then
echo "done"
echo "Backup filesystem properties:"
echo "$FSPROPS"
echo "Backup filesystem status:"
$DF --si /media/$BACKUP_TO
else
echo "failed"
echo "$OUTPUT"
echo "Backup filesystem properties:"
echo "$FSPROPS"
EXITSTATUS=4
fi
fi
if test $EXITSTATUS = 0
then
echo -n "Checking if backup subdirectory is present... "
if test -d /media/$BACKUP_TO/$SUBDIR
then
echo "yes"
else
echo "no"
EXITSTATUS=5
fi
fi
if test $EXITSTATUS = 0
then
echo "Performing backup:"
echo
$RDIFFBACKUP backup \
--include-globbing-filelist $INCLUDE \
--print-statistics / /media/$BACKUP_TO/$SUBDIR
MYEXIT=$?
echo
if test $MYEXIT != 0
then
echo "/\/\/\/\/\/\/\/\/\/\/\/\/\/\/"
echo "Warning: exit status was: $MYEXIT"
echo "/\/\/\/\/\/\/\/\/\/\/\/\/\/\/"
echo
fi
echo "Backup filesystem status:"
$DF --si /media/$BACKUP_TO
if test $CLEANING = "yes"
then
echo "Cleaning out old snapshots:"
echo
$RDIFFBACKUP --force remove increments \
--older-than $CLEAN_OLDER_THAN \
/media/$BACKUP_TO/$SUBDIR
MYEXIT=$?
echo
if test $MYEXIT != 0
then
echo "/\/\/\/\/\/\/\/\/\/\/\/\/\/\/"
echo "Warning: exit status was: $MYEXIT"
echo "/\/\/\/\/\/\/\/\/\/\/\/\/\/\/"
echo
fi
echo "Backup filesystem status:"
$DF --si /media/$BACKUP_TO
fi
echo -n "Unmounting backup filesystem... "
if OUTPUT=`$PUMOUNT $BACKUP_TO 2>&1`
then
echo "done"
else
echo "failed"
echo "$OUTPUT"
fi
fi
exit $EXITSTATUS
Finally, set proper permissions for the backup script:
# chmod 755 /etc/cron.weekly/backuprsync
This script will periodically backup the system and any data found in regular users' ~/backup directories.
Plug your backup disk and start the first backup (in ten minutes):
# anacron -f cron.weekly
Remember that the backup disk must be plugged in, whenever a backup is due.
As stated above, the system backup has been configured to periodically backup system files and any data found in regular users' ~/backup directories. Now let's see what regular users may store in those directories.
Install the following package:
# aptitude install duplicity
Create the backup directory and prepare the include/exclude file for your regular user:
$ mkdir ~/backup $ cat ~/.duplicity.include - ./backup/** - ./music/CDs/** - ./OLD - ./var/** - ./.cache - ./.thumbnails - ./.Mix
Then prepare a trivial backup script:
$ mkdir -p ~/bin
$ cat ~/bin/duplicity-backup.sh
#!/bin/sh
COMMAND=""
if test "x$1" = "xfull"
then
COMMAND="full"
fi
OPTION=""
if test "x$1" = "xdifferent"
then
OPTION="--allow-source-mismatch"
fi
cd
duplicity $COMMAND $OPTION --encrypt-key 3E1C27E11F69BFFE \
--full-if-older-than 30D \
--include-filelist .duplicity.include . file://backup
duplicity remove-older-than 25D --force file://backup
(where mykeyid should be substituted with the regular user's GnuPG key identifier) and set proper permissions:
$ chmod u+x ~/bin/duplicity-backup.sh
Everytime the regular user runs this script, an encrypted backup of his/her home directory will be created in his/her ~/backup directory. Files will be excluded or included as configured in ~/.duplicity.include. Backup sets older than 25 days will be removed. A full backup will be performed the first time the script is run, or whenever the full command-line argument is passed to the script, or whenever the latest full backup is older than 30 days; otherwise an incremental backup will be performed. If the name of the host changed, the different command-line argument may be passed to the script, in order to proceed with the incremental backup anyway.
Sometimes you need to copy data between different directories (or between network-connected machines). For simple cases, cp (or scp) may suffice. However, more complicated cases may require something more sophisticated; install the following package:
# aptitude install rsync
or mark it as manually installed, if it is already present:
# aptitude unmarkauto rsync
You can prepare a trivial script to update your regular user's personal data on a remote host, keeping the "master copy" on the local box:
$ mkdir -p ~/bin
$ cat ~/bin/send-data-to.sh
#!/bin/sh
REMOTEDEST="remoteuser@remote.box.example:."
if test "x$1" != "x"
then
REMOTEDEST=$1
fi
cd
rsync -av --delete --include-from=.send-data.rsync-incl \
. $REMOTEDEST
(where remoteuser and remote.box.example should be substituted with your regular user name and host name on the remote machine you have SSH access to) and set proper permissions:
$ chmod u+x ~/bin/send-data-to.sh
The include/exclude file may be something like:
$ cat ~/.send-data.rsync-incl + .rolo/ + .rolo/contacts.vcf + .mozilla/ + .mozilla/firefox/ + .mozilla/firefox/*.default/ + .mozilla/firefox/*.default/places.sqlite* - *
Other directories or files of interest may be included. Please note that inclusion/exclusion globbing syntax for rsync is slightly different from rdiff-backup/duplicity one.
In the special case you need to copy data to a FAT filesystem (e.g.: a USB flash memory stick, or a portable audio/video player, or whatever is somehow constrained to use a FAT filesystem), you cannot use rsync with the options described above, since a FAT filesystem does not fully support permissions, timestamps, and so forth.
However, you can still take advantage of most rsync features, by using it the following way:
$ rsync -rctOLv --delete ~/origin/ /media/usbstick/destination/
Please note that FAT filesystems are case-insensitive, which implies that the existence of two file names that differ only by case is not allowed. If you need to copy data to a FAT filesystem, you are recommended to check that no file name in ~/origin/ is case-insensitively equal to another: otherwise, you'll end up with /media/usbstick/destination/ containing only one of the two (or more) case-insensitively identically named files!
The following script tries to fix the above-described situation, before the data are copied to a FAT filesystem:
$ mkdir -p ~/bin
$ cat ~/bin/case_insensitive_rename
#!/bin/sh
#
# case_insensitive_rename v0.5 - Prepare a directory tree to be copied
# to a case-insensitive filesystem
#
# usage:
# cd DIRECTORY
# case_insensitive_rename
#
#
# Copyright (c) 2020 Francesco Poli <invernomuto@paranoici.org>
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to
# deal in the Software without restriction, including without limitation the
# rights to use, copy, modify, merge, publish, distribute, sublicense, and/or
# sell copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# The software is provided "as is", without warranty of any kind, express or
# implied, including but not limited to the warranties of merchantability,
# fitness for a particular purpose and noninfringement. In no event shall the
# authors or copyright holders be liable for any claim, damages or other
# liability, whether in an action of contract, tort or otherwise, arising
# from, out of or in connection with the software or the use or other dealings
# in the software.
if test "$1" = "create_test"
then
tmpdir=$(mktemp -d CASE_INS_RENAME_TEST_XXXXXX)
if cd $tmpdir
then
mkdir 'dir' 'Dir' 'Dir/ANOTHER DIR' 'one direc'
touch 'dir/foo bar.txt' 'dir/FOO Bar.TXT' 'dir/baz.JPG'
touch 'Dir/foo bar.txt' 'Dir/baz.jpg' 'Dir/baz.JPG'
touch 'Dir/ANOTHER DIR/Hello dear.AVI'
fi
exit
fi
printf 'Current tree root:\n%s\n' "$(pwd)"
read \
-p 'Should this tree be prepared for a case-insensitive filesystem? [y/n] ' \
-r ans
if test "$ans" != 'y'
then
exit
fi
depth=1
while test "$(find -mindepth $depth -maxdepth $depth -print -quit)" != ""
do
find -mindepth $depth -maxdepth $depth -type d -print0 \
| rename -v -0 's/ /_/g; y/A-Z/a-z/'
find -mindepth $depth -maxdepth $depth -type d -name \*[A-Z]\* -print0 \
| rename -v -0 's/ /_/g; y/A-Z/a-z/; s/$/_b/'
find -mindepth $depth -maxdepth $depth -type f -print0 \
| rename -v -0 's/ /_/g; y/A-Z/a-z/'
find -mindepth $depth -maxdepth $depth -type f -name \*[A-Z]\* -print0 \
| rename -v -0 's/ /_/g; y/A-Z/a-z/; s/\.([^.]*)$/_b.$1/'
depth=$(($depth + 1))
done
Of course you should remember to set proper permissions:
$ chmod u+x ~/bin/case_insensitive_rename
Some digital cameras are accessible as USB mass storage devices, but some other cameras need support for the PTP protocol or other vendor-specific protocols. For the latter cases, the following package may be useful:
# aptitude install gphoto2
Sometimes the need to write CD-ROMs and/or DVD-ROMs may arise. You may install the following packages:
# aptitude --without-recommends install k3b dvd+rw-tools+M
and configure this CD/DVD burning application for your regular user:
$ k3b
Select Configure K3b... from the Settings menu. In the "Misc" section, select default action dialog settings "Saved Settings". In the "Notifications" section, for each state, uncheck "Play a sound". In the "Advanced" section, check "Do not eject medium after write process". Leave all the other settings to their default values. Click on the OK button.
Now that your data are a bit safer, you'll probably want to enjoy some music! More details in a separate document (HTML, reST).