Tagged with linux

Raspbian root (/) on USB-stick for my Raspbian Pi 2

I recently bought a Raspberry Pi 2. At the same time I ordered a USB 3.0 flash disk, because I was afraid the SD-card would be too slow. I'm now going to put the root filesystem of Raspbian on the faster USB flash disk, while keeping the bootloader and /boot on the SD-card, because the rPi can't boot from USB.

Get the Raspbian image

My original idea was to use the Raspbian installer so I could install Raspbian like I'd install Debian. The installer on www.raspbian.org wouldn't boot on my rPi2. According to the timestamp, it was 2.5 years old. Maybe it doesn't support the rPi2 yet.

So instead, I downloaded the Raspbian image from the Raspberry Pi site, at the time the .zip file was 975 MB.

Inside the .zip file is the entire Raspberry OS, 3.1 GB in total. When you extract it, you end up with an .img file.

$  unzip 2015-02-16-raspbian-wheezy.zip 
Archive:  2015-02-16-raspbian-wheezy.zip
  inflating: 2015-02-16-raspbian-wheezy.img  

The USB-stick

Prepare the USB-stick

Insert the USB-stick in your computer and find out which device name it gets, using dmesg and lsblk.

Create a partition on your USB-stick for the root filesystem. My stick was 16 GB, so I used 15 GB for / and 1 GB for swap:

Disk /dev/sdc: 14,7 GiB, 15787360256 bytes, 30834688 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x000d032d

Device     Boot    Start      End  Sectors  Size Id Type
/dev/sdc1           2048 28733439 28731392 13,7G 83 Linux
/dev/sdc2       28735488 30832639  2097152    1G 82 Linux swap / Solaris

Copy / from .img to USB

The .img file contains two partitions, one is /boot and the second is /. The contents of the first go on the SD-card, and the other goes on the USB-stick.

The tricky part is dumping from the exact offset so you get all the files (blocks actually) you need, and nothing more. To figure out what the offset of the second partition is, use fdisk -l on the .img file:

$ fdisk -l 2015-02-16-raspbian-wheezy.img 

Disk 2015-02-16-raspbian-wheezy.img: 3,1 GiB, 3276800000 bytes, 6400000 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x0009bf4f

Device                          Boot  Start     End Sectors Size Id Type
2015-02-16-raspbian-wheezy.img1        8192  122879  114688  56M  c W95 FAT32 (LBA)
2015-02-16-raspbian-wheezy.img2      122880 6399999 6277120   3G 83 Linux

As you can see, the second partition starts at sector 122880. But dd, the tool we'll use to dump the data, uses bytes for its offset option. Luckily fdisk lists the sector size in its output:

Sector size (logical/physical): 512 bytes / 512 bytes

Simply multiply the starting sector of the second partition by 512 and you know the offset: 62914560 bytes!

To verify your calculation, use losetup to create a loopback device from the second partition inside the .img file:

 $ losetup --offset=62914560 /dev/loop0 /tmp/2015-02-16-raspbian-wheezy.img   

Now you can use dd to write the contents from the loopback device onto your USB-stick:

$ dd if=/dev/loop0 of=/dev/sdc1 bs=4M

Resize the USB disk

Mount the USB partition that holds the root filesystem for Raspbian we just dumped:

 $ mount /dev/sdc1 /mnt
 $ df -m /mnt
Filesystem     1M-blocks  Used Available Use% Mounted on
/dev/sdc1           2953  2421       364  87% /mnt

As you can see, the filesystem is only 3 GB in size, but the partition I created is 15 GB! That's because we used dd, to dump on a block level rather than a file level. To correct this, we just have to use resize2fs on the partition:

# umount /mnt
# fsck -fv /dev/sdc1 
fsck from util-linux 2.25.2
e2fsck 1.42.12 (29-Aug-2014)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
...
# resize2fs /dev/sdc1 
resize2fs 1.42.12 (29-Aug-2014)
Resizing the filesystem on /dev/sdc1 to 3591424 (4k) blocks.
The filesystem on /dev/sdc1 is now 3591424 (4k) blocks long.

# fsck -fv /dev/sdc1 
fsck from util-linux 2.25.2
e2fsck 1.42.12 (29-Aug-2014)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
...
root@thinker:~# mount /dev/sdc1 /mnt
root@thinker:~# df -m /mnt
Filesystem     1M-blocks  Used Available Use% Mounted on
/dev/sdc1          13745  2423     10605  19% /mnt

There you go: the entire 15 GB can be used by Raspbian now!

Update /etc/fstab on the USB-stick

While the USB-stick is still mounted, open /mnt/etc/fstab and update the line for the / file system from

/dev/mmcblk0p2  /               ext4    defaults,noatime  0       1

to

/dev/sda1       /               ext4    defaults,noatime  0       1

The SD-card

Prepare the SD-card

Create a single FAT32 partition on your SD-card. It doesn't have to be big, as we saw in the fdisk -l 2015-02-16-raspbian-wheezy.img output, the /boot partition is only 56 MB. Just make sure the partition is FAT32 (id C in fdisk) and that the boot flag is enabled.

Copy /boot from .img to SD-card

Mount the /boot partition from the .img file using the offset trick we saw before:

# mount -o loop,offset=$(expr 8192 \* 512) /tmp/2015-02-16-raspbian-wheezy.img /mnt/raspBoot

Mount your SD-card and copy the files required for /boot to the root of your SD-card. This time we use cp instead of dd because it's just a small amount of files:

# mount  mount /dev/mmcblk0p1 /mnt/sdCard
# cp -av /mnt/raspBoot/* /mnt/sdCard

Update boot parameters

Before unmounting the SD-card, open the file cmdline.txt and update the root= part so it looks like

dwc_otg.lpm_enable=0 console=ttyAMA0,115200 kgdboc=ttyAMA0,115200 console=tty1 root=/dev/sda1 rootfstype=ext4 rootwait text

Finishing up

Now umount both the USB-stick and the SD-card and plug them in your rPi2. Power on the rPi2 and it will boot from SD-card but the root file system will be on the USB-stick!

tristan@rpi2 ~ $ df -m
Filesystem     1M-blocks  Used Available Use% Mounted on
rootfs             13745  2424     10604  19% /
/dev/root          13745  2424     10604  19% /
devtmpfs             460     0       460   0% /dev
tmpfs                 93     1        93   1% /run
tmpfs                  5     0         5   0% /run/lock
tmpfs                391     0       391   0% /run/shm
/dev/mmcblk0p1       253    15       238   6% /boot
Tagged , ,

nginx User-Agent spoofing to circumvent LinkedIn's blacklisting

I have a few virtual hosts that redirect to my LinkedIn profile thanks to a rewrite rule in nginx:

server {
  server_name  example.com;
  listen       80;

  location / {
    rewrite    ^  http://be.linkedin.com/in/tristanterpelle/;
  }
}

This worked beautifully when I tested it, but today I noticed it was broken. LinkedIn returned a 999 error. After looking around a bit, it turns out that LinkedIn actively blocks HTTP requests from clients with certain User-Agent strings.

 $ curl -I --url https://be.linkedin.com/in/tristanterpelle 
HTTP/1.1 999 Request denied
$  curl -A "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71 Safari/537.36" -I --url https://be.linkedin.com/in/tristanterpelle
HTTP/1.1 200 OK

OK, so curl's User-Agent is blacklisted, and apparently so is nginx' (wget also fails). Luckily, nginx can spoof its User-Agent thanks to the HttpHeadersMoreModule.

server {
  server_name  example.com;
  listen       80;

  location / {
    rewrite    ^  http://be.linkedin.com/in/tristanterpelle/;
    more_set_input_headers 'User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.71 Safari/537.36';
  }
}

On Debian, you need to install the nginx-extras package, or you will get a [emerg] unknown directive "more_set_input_headers" error.

Tagged , ,

VMware Workstation 9.0.1 on Debian with kernel 3.10

New major Linux kernel version, new major breakage of VMware Workstation. Here's what and how to patch to be able to compile VMware's modules so you can run your virtual machines under Linux kernel 3.10.

vmnet

The compilation of the vmnet.ko module fails, a patch found on the VMware Community forum fixes this.

It has to be applied to the contents of /usr/lib/vmware/modules/source/vmnet.tar. You can find the instructions on how to do this in my older post.

vmblock

After the first patch, we're still not out of the woods: the vmblock.ko won't compile either. In the same thread a link is provided to a blog post with the appropriate patch. This time, the patch has to be applied to the files in /usr/lib/vmware/modules/source/vmblock.tar.

Compile

Now you can compile the modules with vmware-modconfig --console --install-all, provided you installed the appropriate kernel header files for your kernel. In my case, the packages I needed were:

ii  linux-headers-3.10-2-amd64            3.10.7-1                         amd64        Header files for Linux 3.10-2-amd64
ii  linux-headers-3.10-2-common           3.10.7-1                         amd64        Common header files for Linux 3.10-2
ii  linux-image-3.10-2-amd64              3.10.7-1                         amd64        Linux 3.10 for 64-bit PCs

Sources

Tagged , , ,

Prey anti-theft on Debian

Anti-theft solution

Prey is an open source anti-theft solution. I decided to install it on my laptop so I could track it in case I lost it or it was stolen.

Prey is scheduled to "call home" to check the status of the device it's protecting. If it finds out the device went missing, it build a report with useful information to track down the device and tries to send it to the user. It can do the check and send the report via the Prey Control Panel (simple to setup) or you can run it in standalone mode which offers more options. I used the simple setup to find out if and how it works.

Installation

Version 0.5.3 is currently in the Debian repositories, which is easy to install (apt-get install prey) but after a few hours of trying, I couldn't get it to do anything useful: Prey noticed when my laptop was missing and created and sent the reports but they never arrived.

A post on Prey's anwers list recommended to upgrade to version 0.5.9 to get things working. I decided to install the latest version, which was 0.6.0. There's .deb for Ubuntu that I tried first on Debian Jessie/Sid, but long story short: it didn't work. Instead, I downloaded the universal .zip for Linux and made a .deb from it, the quick and dirty way.

Because it was built the quick and dirty way, Prey was complaining about a missing cronjob and configuration file.

The cronjob was in /etc/cron.d/prey when Prey expected it to be in root's list of crons. I copied the lines from /etc/cron.d/prey to crontab -e -u root and then rm'ed /etc/cron.d/prey. I changed the frequency to every 5 minutes.

Prey looked for the configuration file in /usr/lib/prey/config, which I solved by symlinking /etc/prey/config there: ln -sv /etc/prey/config /usr/lib/prey/config.

Like I said: a very quick and dirty packaging.

Configuration

To use Prey, you need to register an account. Once you've done that, log in and you will find your API key in the Account page. This key goes in your configuration file on the api_key= line. When you've set that, you can run Prey in test mode where it will check its status and generate a device key. It will immediately add the device key to the config file as well. Some moments later, your device will appear in Prey's Control Panel.

In Prey's Control Panel you can set the information you want gathered in the Devices tab. I deselected all the actions to perform on the right to keep Prey inconspicuous to a possible thief.

Testing

Set your device's status to Missing and wait for the cronjob to kick in. You should be getting a report with some very useful information a few minutes later. I checked and even when I didn't have X running, I got a nice report with a webcam picture.

Don't forget to set the status back to normal when you've verified that everything works: the free Prey account only has space for 10 reports.

Sources

Tagged , , ,

VMware Workstation 9.0.1 on Debian with kernel 3.8

Missing C header files

I installed VMware Workstation 9.0 on a fresh Debian Jessie/Sid installation, but when I ran it after the installation, I was greeted with the error message:

C header files matching your running kernel were not found.  Refer to your distribution's documentation for installation instructions.

and a dialog to locate said missing headers manually.

I found the fix to this annoying issue on ArchLinux' Wiki. The directory structure of the header files has changed since Linux kernel 3.7, so the installer cannot find the file it's looking for. A simple symlink resolves this.

# ln -sv /usr/src/linux-headers-$(uname -r)/include/generated/uapi/linux/version.h /usr/src/linux-headers-$(uname -r)/include/linux
‘/usr/src/linux-headers-3.8-1-amd64/include/linux/version.h’ -> ‘/usr/src/linux-headers-3.8-1-amd64/include/generated/uapi/linux/version.h’

Now VMware Workstation would at least try to compile the kernel modules!

Error compiling kernel modules

But failed miserably in the process:

Using 2.6.x kernel build system.
make: Entering directory `/tmp/modconfig-rWaQRn/vmci-only'
/usr/bin/make -C /lib/modules/3.8-1-amd64/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. \
   MODULEBUILDDIR= modules
make[1]: Entering directory `/usr/src/linux-headers-3.8-1-amd64'
  CC [M]  /tmp/modconfig-rWaQRn/vmci-only/linux/driver.o
  CC [M]  /tmp/modconfig-rWaQRn/vmci-only/linux/vmciKernelIf.o
  CC [M]  /tmp/modconfig-rWaQRn/vmci-only/common/vmciContext.o
  CC [M]  /tmp/modconfig-rWaQRn/vmci-only/common/vmciDatagram.o
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:127:4: error: implicit declaration of function __devexit_p [-Werror=implicit-function-declaration]
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:127:4: error: initializer element is not constant
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:127:4: error: (near initialization for vmci_driver.remove)
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:1754:1: error: expected =, ,, ;, asm or __attribute__ before vmci_probe_device
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:1982:1: error: expected =, ,, ;, asm or __attribute__ before vmci_remove_device
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:119:12: warning: vmci_probe_device used but never defined [enabled by default]
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:121:13: warning: vmci_remove_device used but never defined [enabled by default]
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:2063:1: warning: vmci_interrupt defined but not used [-Wunused-function]
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:2137:1: warning: vmci_interrupt_bm defined but not used [-Wunused-function]
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:1717:1: warning: vmci_enable_msix defined but not used [-Wunused-function]
cc1: some warnings being treated as errors
make[4]: *** [/tmp/modconfig-rWaQRn/vmci-only/linux/driver.o] Error 1
make[4]: *** Waiting for unfinished jobs....
make[3]: *** [_module_/tmp/modconfig-rWaQRn/vmci-only] Error 2
make[2]: *** [sub-make] Error 2
make[1]: *** [all] Error 2
make[1]: Leaving directory `/usr/src/linux-headers-3.8-1-amd64'
make: *** [vmci.ko] Error 2
make: Leaving directory `/tmp/modconfig-rWaQRn/vmci-only'
Failed to build vmci.  Failed to execute the build command.
Using 2.6.x kernel build system.
make: Entering directory `/tmp/modconfig-rWaQRn/vmci-only'
/usr/bin/make -C /lib/modules/3.8-1-amd64/build/include/.. SUBDIRS=$PWD SRCROOT=$PWD/. \
   MODULEBUILDDIR= modules
make[1]: Entering directory `/usr/src/linux-headers-3.8-1-amd64'
  CC [M]  /tmp/modconfig-rWaQRn/vmci-only/linux/driver.o
  CC [M]  /tmp/modconfig-rWaQRn/vmci-only/common/vmciDoorbell.o
  CC [M]  /tmp/modconfig-rWaQRn/vmci-only/common/vmciDriver.o
  CC [M]  /tmp/modconfig-rWaQRn/vmci-only/common/vmciEvent.o
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:127:4: error: implicit declaration of function __devexit_p [-Werror=implicit-function-declaration]
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:127:4: error: initializer element is not constant
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:127:4: error: (near initialization for vmci_driver.remove)
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:1754:1: error: expected =, ,, ;, asm or __attribute__ before vmci_probe_device
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:1982:1: error: expected =, ,, ;, asm or __attribute__ before vmci_remove_device
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:119:12: warning: vmci_probe_device used but never defined [enabled by default]
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:121:13: warning: vmci_remove_device used but never defined [enabled by default]
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:2063:1: warning: vmci_interrupt defined but not used [-Wunused-function]
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:2137:1: warning: vmci_interrupt_bm defined but not used [-Wunused-function]
/tmp/modconfig-rWaQRn/vmci-only/linux/driver.c:1717:1: warning: vmci_enable_msix defined but not used [-Wunused-function]
cc1: some warnings being treated as errors
make[4]: *** [/tmp/modconfig-rWaQRn/vmci-only/linux/driver.o] Error 1
make[4]: *** Waiting for unfinished jobs....
make[3]: *** [_module_/tmp/modconfig-rWaQRn/vmci-only] Error 2
make[2]: *** [sub-make] Error 2
make[1]: *** [all] Error 2
make[1]: Leaving directory `/usr/src/linux-headers-3.8-1-amd64'
make: *** [vmci.ko] Error 2
make: Leaving directory `/tmp/modconfig-rWaQRn/vmci-only'
Failed to build vmci.  Failed to execute the build command.

The contents of /usr/lib/vmware/modules/source/vmci.tar needed to be patched, according to this thread on the VMware forums.

# wget -O /tmp/vmware9.k3.8rc4.patch http://communities.vmware.com/servlet/JiveServlet/download/2182440-102649/vmware9.k3.8rc4.patch
# cd /usr/lib/vmware/modules/source
# tar xf vmci.tar
# cd vmci-only
# patch -p1 < /tmp/vmware9.k3.8rc4.patch 
patching file linux/driver.c
# tar cf vmci.tar vmci-only
# rm -rf /usr/lib/vmware/modules/source/vmci-only

After that, I just had to run vmware as root and the modules were built successfully.

Sources

Tagged , , ,

Self signed root CA and certificates

My plan was to describe how to set up your own root CA and certificate. But then I found this wonderful guide, and I decided there was no way I could even do it half as good as Marcus Redivo did.

Source: http://www.eclectica.ca/howtos/ssl-cert-howto.php

Tagged ,

Resize physical disk of VMware guest on Linux

What

How to resize the disk of a VMware virtual machine that's using a physical disk as its storage backend. The disk the virtual machine is using is an LVM logical volume, which is very easy to extend.

In this article I'm extending the disk, which is always easier than reducing. If you're reducing your disk, make sure the new size is still MORE than the space used on the file system, or you will lose data. And make a backup before you touch anything.

Why

I'm running Windows 7 in a virtual machine on my Linux host. I figured 20 GB would be plenty for a guest OS I keep for my USB dongles that are not supported in Linux.

After the installation, Windows left me with only 7 GB of free space left. And to make use of Microsoft's cheap upgrade to Windows 8 program, 25 GB of disk space is a requirement. Time to grow that 20 to 30 GB!

How

Preparation

  • power down the virtual machine
  • go to the directory that holds the virtual machine's disk file with extension .vmdk and make a backup of the file

Extend the logical volume

The original size of my LV was 20 GB, or to be more precise: 2560 logical extents of 4 MB:

 $ lvdisplay /dev/vg00/vm-win7 
  --- Logical volume ---
  LV Path                /dev/vg00/vm-win7
  LV Name                vm-win7
  VG Name                vg00
  LV UUID                739fos-h2dt-4k9f-Ghai-HiKk-AAAA-9Xqtjz
  LV Write Access        read/write
  LV Creation host, time host, 2012-11-19 21:33:59 +0100
  LV Status              available
  # open                 0
  LV Size                20.00 GiB
  Current LE             2560
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           254:9

I increased the LV with 50 % by adding 1280 logical extents:

 $ lvextend -l +1280 -n /dev/vg00/vm-win7
  Extending logical volume vm-win7 to 30.00 GiB
  Logical volume vm-win7 successfully resized

There's a good reason for sizing the LV in logical extents (-l) rather than in the easier to read megabytes/gigabytes/... with -L, as you will see in the next step.

For more information on LV resizing, read Extending a logical volume in the LVM HOWTO.

VM configuration

The LV of the VM has been extended, but the VM is still seeing the original size. The size is defined in the VM's vmdk file:

...
# Extent description
RW 41943040 FLAT "/dev/vg00/vm-win7" 0
...

The LV /dev/vg00/vm-win7 is defined to be 41943040 sectors large, and one sector is 512 bytes giving a total of 21474836480 bytes.

Because I used logical extents and not megabytes or gigabytes to extend the LV, calculating the new number of sectors is very easy: 41943040 + 50 % = 62914560. So the line in the vmdk is changed to:

RW 62914560 FLAT "/dev/vg00/vm-win7" 0

Verify the new size in the virtual machine's settings: under Hardware, go to Hard Disk and you will see the resized capacity under Disk Information.

For more information on VMware's VMDK disk format, read the VMware Virtual Disk Format 5.0 technical note.

Resize the FS in the VM

The LV has been resized and VMware is aware of it, but the VM is still using a partition table and file system for the old size.

  • download the GParted Live CD and connect it to your VM's virtual CD drive
  • boot into GParted and resize the VM's disk to use the newly added space
  • shutdown the VM and remove the virtual CD
  • start the Windows VM as usual
  • Windows will want to run a file system check, DO NOT CANCEL THIS!
  • after the check, Windows will reboot and when it comes back the disk will show the new capacity!

Sources

Tagged , ,