Backup with restore/dump

After previous post, we have fresh install with hardened system(kernel+world) and basic ports installed.
It's time wasting to install everything again and again on new machine or jail or reinstall current machine. The solution is to backup whole hierarchy of system and restore if needed.
One of the methods is using "dd" tool. It has pros and cons for me.
Pros: You can do a mirror of disk and copy this mirror to new unformated disk.
Cons: With disk which has been in use for a while you have to zero-out the "free" space on the disk. Than you can compress dd output. Otherwise you will get the output size as the size of your disk. Etc. you want to backup disk with capacity 320GB. The output would be 320GB in size...because it's more like disk mirroring.
For me, it's faster to install some random FreeBSD version to create system hierarchy and than restore this system with dump/restore (offline).

###

As you recall we made a hardened system installation. If you want to restore this system to another machine, there will be a problem. We left in kernel config just those lines which we need to run system on current machine. After the restore on another machine, some peripherals/devices won't work. I don't count the "problem" with fstab edit which is quite normal with every "migration". So be aware of this. I will write about workaround later.

###

1.Dump
This is fstab on test system:
/dev/ada1s4a    /               ufs     rw      1       1
/dev/ada1s4f    /var            ufs     rw      2       2
/dev/ada1s4d    /tmp            ufs     rw      2       2
/dev/ada1s4e    /usr            ufs     rw      2       2
/dev/ada1s4b    none            swap    sw      0       0

Let say we would like to backup root directory.

dump -a -h 0 -0 -C 128 -L -f root.dmp /dev/ada1s4a

-a: autosize, I don't use tape device...
-h 0: I want to exclude some files/dirs from dump at level 0 (default L1).
-0: Level 0, which means full backup. Other ones are for incremental backup. I don't use "dump" for incremental backups.
-C 128: Use value from 32 to 128 etc. It's size of cache in MB. This is I think only one value which can really improve speed of process. You can use "-b" flag (that without "-C") to set block size but I've tested with many options and the best speed I reached with "-C". With this flag, the block size is 65535, I think in kB. Make your own testing. Dump etc. root with various options and check the disk usage in another terminal with command: iostat -c 20.
-L: To tell dump we are dumping live system
-f root.dmp: Write to file. You can use "-f - " to redirect output to stdout and process it via pipe etc...
/dev/ada1s4a: This is partition which is mounted to "/" dir.

I can reach with above dump cca 35-40MB/s speed on 5400rpm SATA 320GB disk.

2.Restore
This is not "dd" so you have to have system hierarchy created.
I don't recommend to restore live system, actually I've not tried this.
These are my steps:
I use PCBSD live DVD to run independent system so both, source and destination is mounted as offline. Why PCBSD? Because it's FreeBSD system so I know what to expect and how the disks will be named.
You can have dump files on any media but I recommend to use disks because USB, Cards, DVDs are bottleneck for restore.

mkdir /mnt_src

mkdir /mnt_dst

mount your source media (with dump files) to /mnt_src
mount your destination to /mnt_dst. Etc. mount -t ufs /dev/ada1s2a /mnt_dst
cd /mnt_dst !!!!!

restore -r -b 65536 -f /mnt_src/root.dmp

-r: restore...:)
-b: block size in kB. Use the value which was used for dump.
-f: specify the file from which the restore will read

If you want to restore again for what ever reason, first remove the content from previous restore. Some dot files are marked with "secure" flag so you can't delete them as normal.

chflags -R 0 /mnt_dst

rm -R /mnt_dst/*

I can reach with above restore cca 50-55MB/s speed on 5400rpm SATA 320GB disk.

3.Excluded files in dump process
Let say we don't want to dump file /root/exclude.exc.
First check the current flags: ls -lso /root/exclude.exc
4 -rw-r--r--  1 root  wheel  -  304 Mar 26 01:13 exclude.exc
"-" in the middle says that this file will be dumped (dump flag, default)

chflags nodump /root/exclude.exc

Now you can see the new flag from "ls" output.

To clear the flag, use: chflags dump /root/exclude.exc


Don't forget to use "-h 0" option with dump.
(If you don't want to dump the directory, I'm not sure if you have to use chflag with "-R" option or without...try.)

4.Compression of dump files
You can save a lot of space (under some conditions) with compressing the dump files.

gzip -c root.dmp > root.dmp.gz

Install port "pigz" which is same as gzip but it can uses benefits of muti-core CPUs!
(Be aware that gzip is not for compressing directories! In this case use "tar".)
gzip -c -d root.dmp.gz > root.dmp - use this for unzip.

You can save the time and temporary space with compressing on fly within dump process. Just redirect dump to stdout and use gzip via pipe.
Etc. my whole (fresh and ready to go) system is 3,3GB in size and compressed 810MB in size.

5.Hints
If you have multiple BSD installations on one computer, and you have problems with booting one of them after major change(resotre, new installation), always check if your system you want to boot has correct "/etc/fstab" information.
Etc. you created new system with same structure as your old, but in different order so /dev/ada1s2e on one system is etc. "var" and on another system /dev/ada1s4e is etc. "usr". So it's not just about changing number "2" to "4" and backward.

If you want to make another system to be bootable, run some which is working and run sysinstall, than option "Configure" and than "Fdisk". And with letter "S" as in the legend, you can choose which partition is bootable.
Before any change on live system via "Fdisk" change this before sysinstall:

sysctl kern.geom.debugflags=16

One of essential step after restore, before starting to boot new system, is to remove following on new system:

rm -R /var/run/*

That's because if you dump /var of your live system, content of this directory will be dumped and the content are pid files of current running processes.
If you started new system without deleting content before, for system it seems like services were not stopped and it causes problems. When the system is shutting down, content is removed by stopping services so system doesn't expect to find something in this directory after new boot process.