Tagged with backup

Resize physical disk of VMware guest on Linux

What

How to resize the disk of a VMware virtual machine that's using a physical disk as its storage backend. The disk the virtual machine is using is an LVM logical volume, which is very easy to extend.

In this article I'm extending the disk, which is always easier than reducing. If you're reducing your disk, make sure the new size is still MORE than the space used on the file system, or you will lose data. And make a backup before you touch anything.

Why

I'm running Windows 7 in a virtual machine on my Linux host. I figured 20 GB would be plenty for a guest OS I keep for my USB dongles that are not supported in Linux.

After the installation, Windows left me with only 7 GB of free space left. And to make use of Microsoft's cheap upgrade to Windows 8 program, 25 GB of disk space is a requirement. Time to grow that 20 to 30 GB!

How

Preparation

  • power down the virtual machine
  • go to the directory that holds the virtual machine's disk file with extension .vmdk and make a backup of the file

Extend the logical volume

The original size of my LV was 20 GB, or to be more precise: 2560 logical extents of 4 MB:

 $ lvdisplay /dev/vg00/vm-win7 
  --- Logical volume ---
  LV Path                /dev/vg00/vm-win7
  LV Name                vm-win7
  VG Name                vg00
  LV UUID                739fos-h2dt-4k9f-Ghai-HiKk-AAAA-9Xqtjz
  LV Write Access        read/write
  LV Creation host, time host, 2012-11-19 21:33:59 +0100
  LV Status              available
  # open                 0
  LV Size                20.00 GiB
  Current LE             2560
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:9

I increased the LV with 50 % by adding 1280 logical extents:

 $ lvextend -l +1280 -n /dev/vg00/vm-win7
  Extending logical volume vm-win7 to 30.00 GiB
  Logical volume vm-win7 successfully resized

There's a good reason for sizing the LV in logical extents (-l) rather than in the easier to read megabytes/gigabytes/... with -L, as you will see in the next step.

For more information on LV resizing, read Extending a logical volume in the LVM HOWTO.

VM configuration

The LV of the VM has been extended, but the VM is still seeing the original size. The size is defined in the VM's vmdk file:

...
# Extent description
RW 41943040 FLAT "/dev/vg00/vm-win7" 0
...

The LV /dev/vg00/vm-win7 is defined to be 41943040 sectors large, and one sector is 512 bytes giving a total of 21474836480 bytes.

Because I used logical extents and not megabytes or gigabytes to extend the LV, calculating the new number of sectors is very easy: 41943040 + 50 % = 62914560. So the line in the vmdk is changed to:

RW 62914560 FLAT "/dev/vg00/vm-win7" 0

Verify the new size in the virtual machine's settings: under Hardware, go to Hard Disk and you will see the resized capacity under Disk Information.

For more information on VMware's VMDK disk format, read the VMware Virtual Disk Format 5.0 technical note.

Resize the FS in the VM

The LV has been resized and VMware is aware of it, but the VM is still using a partition table and file system for the old size.

  • download the GParted Live CD and connect it to your VM's virtual CD drive
  • boot into GParted and resize the VM's disk to use the newly added space
  • shutdown the VM and remove the virtual CD
  • start the Windows VM as usual
  • Windows will want to run a file system check, DO NOT CANCEL THIS!
  • after the check, Windows will reboot and when it comes back the disk will show the new capacity!

Sources

Tagged , ,

Automatic synchronization to external disk upon insertion

I wanted to easily synchronize my backup directory to an external USB-disk by simply inserting the disk. Since udev is in charge of managing devices in Linux, I'll let it handle the syncing as well.

The disk I'm using is a 2.5" external USB drive, in this article it's recognized as /dev/sdc by my computer. It has one partition that spans the entire drive: /dev/sdc1. This partition will be used to synchronize files to.

System info

  • Debian Wheezy 64 bit
  • udev version 175-7

Find out disk info

Plug in the external disk you will use. Then run udevadm info --query=all --name=/dev/sdc1 to get the list of device properties. Replace /dev/sdc1 with the name of the partition that you will use.

P: /devices/pci0000:00/0000:00:1d.0/usb4/4-1/4-1.2/4-1.2:1.0/host9/target9:0:0/9:0:0:0/block/sdc/sdc1
N: sdc1
S: disk/by-id/ata-WDC_WD2500BMVS-11F9S0_WD-WXH908713364-part1
S: disk/by-id/scsi-SWD_2500BMV_ExternaWD-WXH908713364-part1
S: disk/by-id/wwn-0x50014ee202191538-part1
S: disk/by-label/backupDisk
S: disk/by-path/pci-0000:00:1d.0-usb-0:1.2:1.0-scsi-0:0:0:0-part1
S: disk/by-uuid/94b94b63-4ec6-4e0c-a6f1-00fa03f00144
E: CUSTOM_RULE=1
E: DEVLINKS=/dev/disk/by-id/ata-WDC_WD2500BMVS-11F9S0_WD-WXH908713364-part1 /dev/disk/by-id/scsi-SWD_2500BMV_ExternaWD-WXH908713364-part1 /dev/disk/by-id/wwn-0x50014ee202191538-part1 /dev/disk/by-label/backupDisk /dev/disk/by-path/pci-0000:00:1d.0-usb-0:1.2:1.0-scsi-0:0:0:0-part1 /dev/disk/by-uuid/94b94b63-4ec6-4e0c-a6f1-00fa03f00144
E: DEVNAME=/dev/sdc1
E: DEVPATH=/devices/pci0000:00/0000:00:1d.0/usb4/4-1/4-1.2/4-1.2:1.0/host9/target9:0:0/9:0:0:0/block/sdc/sdc1
E: DEVTYPE=partition
E: ID_ATA=1
E: ID_ATA_DOWNLOAD_MICROCODE=1
E: ID_ATA_FEATURE_SET_AAM=1
E: ID_ATA_FEATURE_SET_AAM_CURRENT_VALUE=254
E: ID_ATA_FEATURE_SET_AAM_ENABLED=1
E: ID_ATA_FEATURE_SET_AAM_VENDOR_RECOMMENDED_VALUE=128
E: ID_ATA_FEATURE_SET_APM=1
E: ID_ATA_FEATURE_SET_APM_CURRENT_VALUE=128
E: ID_ATA_FEATURE_SET_APM_ENABLED=1
E: ID_ATA_FEATURE_SET_HPA=1
E: ID_ATA_FEATURE_SET_HPA_ENABLED=1
E: ID_ATA_FEATURE_SET_PM=1
E: ID_ATA_FEATURE_SET_PM_ENABLED=1
E: ID_ATA_FEATURE_SET_SECURITY=1
E: ID_ATA_FEATURE_SET_SECURITY_ENABLED=0
E: ID_ATA_FEATURE_SET_SECURITY_ENHANCED_ERASE_UNIT_MIN=96
E: ID_ATA_FEATURE_SET_SECURITY_ERASE_UNIT_MIN=96
E: ID_ATA_FEATURE_SET_SMART=1
E: ID_ATA_FEATURE_SET_SMART_ENABLED=1
E: ID_ATA_ROTATION_RATE_RPM=5400
E: ID_ATA_SATA=1
E: ID_ATA_SATA_SIGNAL_RATE_GEN1=1
E: ID_ATA_WRITE_CACHE=1
E: ID_ATA_WRITE_CACHE_ENABLED=1
E: ID_BUS=ata
E: ID_FS_LABEL=backupDisk
E: ID_FS_LABEL_ENC=backupDisk
E: ID_FS_TYPE=ext3
E: ID_FS_USAGE=filesystem
E: ID_FS_UUID=94b94b63-4ec6-4e0c-a6f1-00fa03f00144
E: ID_FS_UUID_ENC=94b94b63-4ec6-4e0c-a6f1-00fa03f00144
E: ID_FS_VERSION=1.0
E: ID_MODEL=WDC_WD2500BMVS-11F9S0
E: ID_MODEL_ENC=WDC\x20WD2500BMVS-11F9S0\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20
E: ID_PART_ENTRY_DISK=8:32
E: ID_PART_ENTRY_NUMBER=1
E: ID_PART_ENTRY_OFFSET=63
E: ID_PART_ENTRY_SCHEME=dos
E: ID_PART_ENTRY_SIZE=488392002
E: ID_PART_ENTRY_TYPE=0x83
E: ID_PART_TABLE_TYPE=dos
E: ID_PATH=pci-0000:00:1d.0-usb-0:1.2:1.0-scsi-0:0:0:0
E: ID_PATH_TAG=pci-0000_00_1d_0-usb-0_1_2_1_0-scsi-0_0_0_0
E: ID_REVISION=01.01A11
E: ID_SCSI_COMPAT=SWD_2500BMV_ExternaWD-WXH908713364
E: ID_SERIAL=WDC_WD2500BMVS-11F9S0_WD-WXH908713364
E: ID_SERIAL_SHORT=WD-WXH908713364
E: ID_TYPE=disk
E: ID_WWN=0x50014ee202191538
E: ID_WWN_WITH_EXTENSION=0x50014ee202191538
E: MAJOR=8
E: MINOR=33
E: SUBSYSTEM=block
E: UDEV_LOG=3
E: UDISKS_DISABLE_POLLING=1
E: UDISKS_PARTITION=1
E: UDISKS_PARTITION_ALIGNMENT_OFFSET=0
E: UDISKS_PARTITION_NUMBER=1
E: UDISKS_PARTITION_OFFSET=32256
E: UDISKS_PARTITION_SCHEME=mbr
E: UDISKS_PARTITION_SIZE=250056705024
E: UDISKS_PARTITION_SLAVE=/sys/devices/pci0000:00/0000:00:1d.0/usb4/4-1/4-1.2/4-1.2:1.0/host9/target9:0:0/9:0:0:0/block/sdc
E: UDISKS_PARTITION_TYPE=0x83
E: UDISKS_PRESENTATION_HIDE=1
E: UDISKS_PRESENTATION_NOPOLICY=1
E: USEC_INITIALIZED=1769647466311

Create udev rules

Create a new file in /etc/udev/rules.d and make sure the extension is .rules, e.g. /etc/udev/rules.d/99-diskSync.rules. The file will contain a rule to match the correct partition on the inserted disk. This rule will be taken from the udevadm output above.

My /etc/udev/rules.d/99-diskSync.rules looks like this:

SUBSYSTEM=="block", ENV{ID_SERIAL}=="WDC_WD2500BMVS-11F9S0_WD-WXH908713364", ENV{UDISKS_DISABLE_POLLING}="1", ENV{UDISKS_PRESENTATION_HIDE}="1", ENV{CUSTOM_RULE}="1", ENV{UDISKS_PRESENTATION_NOPOLICY}="1"
ACTION=="add", SUBSYSTEM=="block", ENV{ID_SERIAL}=="WDC_WD2500BMVS-11F9S0_WD-WXH908713364", ENV{ID_FS_LABEL}=="backupDisk", ENV{UDISKS_PRESENTATION_NOPOLICY}="1", RUN="/usr/local/sbin/dirSync.sh -d /dev/%k -m /media/%E{ID_FS_LABEL} -f -M tristan -s /var/lib/backuppc/ -t /backuppc/"

Several important things to note:

  • The attributes I'm using to match my disk are very specific. If I simply used the serial number for the hard drive, the rule would match and the script would be executed. But it would be executed more than once, because the match is not specific enough: it matches /dev/sdc and /dev/sdc1 and possibly a number of other devices and the command specified by RUN= will be run for each match!
  • RUN= overrides all other RUNs from other matching rules. Before I created this rule, my external disk would get automatically mounted when I inserted it. I don't want that, I will have my script handle the (u)mounting.
  • RUN="/usr/local/sbin/dirSync.sh -d /dev/%k -m /media/%E{ID_FS_LABEL} -f -M -s /sourceDir/ -t /targetDir/ is run and is passed two udev fields, %k (e.g. "sdc1") and %E{ID_FS_LABEL} ("backupDisk")

dirSync.sh

In the udev rule above, a command is called to perform the actual synchronization of the directory to the external disk. I've written the script dirSync.sh for that. It's available from my Bitbucket account: dirSync.sh @ Bitbuck.

Download the script to /usr/local/sbin/ and make sure it's executable.

Restart udev

Reload the udev rules with the command udevadm control --reload-rules.

Test

Plug in the device and let udev do the rest. If you're using my dirSync.sh, you can follow what's happening in the appropriate /tmp/dirSync.out file.

References

Tagged , , ,

Dumping MySQL databases with BackupPC

I'm using automysqlbackup to handle my MySQL dumps. This worked well when launched in a shell, but when BackupPC ran the exact same command as user backuppc, it failed: Backup failed on example.com (DumpPreUserCmd returned error status 256)

After some digging, I found the actual error from the backup script: mkfifo: cannot create fifo `mypipe': Permission denied

or as automysqlbackup calls it: Error: Failed to create the named pipe (fifo) for reading in all databases. Exiting.

My workaround: create a simple wrapper script that calls the automysqlbackup script.

#!/bin/bash
cd ${HOME}
/usr/local/bin/automysqlbackup /etc/automysqlbackup/$(hostname -f).conf
exit $?

Granted, this doesn't solve the actual problem, but at least automysqlbackup dumps my DBs.

Tagged , ,