Tuesday, April 14, 2015

Using a Chromebox as a low-cost VSTi player

The Google Chromebox devices are great little computers: they're small, quiet, inexpensive, and run Linux as the basis of Chrome OS. I wondered if it were possible to use it as a portable Virtual Instrument player.

Turns out with a little know-how and patience the device works surprisingly well. Here are the high-level steps to to make one of your very own!

  1. Purchase an Intel-based Chromebox. I'm using the popular Asus model.
  2. Install Ubuntu on Chrome OS (Chrubuntu).
  3. Boot Ubuntu by default and bypass the developer mode 'CTRL-D' screen.
  4. Configure Ubuntu: wireless networking, ssh, and the power button.
  5. Setup priorities for audio and confgure the audio interface. I used the built-in 3.5mm headphone port.
  6. Setup Wine.
  7. Setup jackd and fsthost.
  8. Setup a suitable VSTi. I used u-he's Zebra2 synthesizer.
  9. Configure your VSTi to start on boot.
Let's go through each of these steps in a bit more detail.

Selecting your Chromebox

We're going to be running Wine and Windows-based VSTi plugins. Both of these have to run on an x86 CPU so make sure you don't purchase an ARM-based device.

I decided on the Asus CHROMEBOX-M004U mostly because it is the most popular one and most of the information on the web seems to be about this device. There's also a more expensive model that uses a Core i3 processor which may be more useful if the plugin you want to run requires more CPU resources.

I'd also like to say that you don't necessarily need a Chromebox. If you already have an x86 based PC running Ubuntu then this software and configuration should work equally as well. Just skip the steps that are tied to Chrome OS, its firmware, or booting.

Install Ubuntu

I followed Roger Stringer's guide and he did a great job of getting me to an Ubuntu login prompt. I didn't upgrade my hardware though. Here are the choices I made when configuring the software:
chrome $ sudo chromeos-firmwareupdate --mode=todev
chrome $ curl -L -O http://goo.gl/s9ryd > /tmp/s9ryd
chrome $ sudo bash /tmp/s9ryd ubuntu-standard
I selected the dual-boot option and gave Ubuntu 6GB of space.

Boot Ubuntu by Default

Next I ran the command in ChromeOS to always boot Ubuntu:
chrome $ sudo cgpt add -i 6 -P 5 -S 1 /dev/sda && sudo reboot
When you reboot now you should see a warning about using the device in developmental mode and are required to hit "CTRL-D" to continue. DO NOT HIT THE SPACE BAR, IT WILL WIPE OUT UBUNTU!

This was a problem for the way I intended to use the device: without a computer keyboard or monitor. I needed a way to bypass the "CTRL-D" screen.

Once you have verified Ubuntu is working you need to boot back into Chrome OS to modify the firmware so that you don't need to hit "CTRL-D" every time you boot. At a Linux prompt enter the following:
ubuntu $ sudo cgpt add -i 6 -P 0 -S 0 /dev/sda && sudo reboot
You have to physically open the device and remove the firmware write-protect screw. Every model is different. I used Don Slesnick's guide for my Asus. You'll have to use Google to find out how to do it on your Chromebox if it's a different vendor. THIS WILL VOID YOUR WARRANTY! ONLY DO SO AT YOUR OWN RISK.

Next,  an article by John Lewis helped me get the Chrome OS commands to modify the firmware settings. John's guide is a bit dated and intended for ARM devices though so the commands I used are slightly different.
Backup the firmware:
chrome $ sudo mkdir /usr/local/fw && sudo flashrom -r /usr/local/fw/bios.bin
Change the firmware to a developer-friendly mode (fast boot with no CTRL-D delay and support booting from USB devices):
chrome $ sudo /usr/bin/set_gbb_flags.sh 0x11 && sudo reboot
The device should reboot into Chrome OS without needing "CTRL-D". All that's left now is to switch back to Ubuntu as your boot partition:
chrome $ sudo cgpt add -i 6 -P 5 -S 1 /dev/sda && sudo reboot
If all goes well the Chromebox should boot into Ubuntu in just under 7 seconds.

Configure Ubuntu

The "ubuntu-standard" setup is really pretty minimal. You'll need to install a few packages to make your life easier and to prepare for setting up Wine and fsthost. By default the only editor installed is nano. If you want something different add it at this time. I chose emacs, but it's not required for the build. The apt-file package also isn't strictly required but it made my life easier when searching for dependencies and packages that own particular files.

Don't forget the default username is "user" and the password is also "user". The account has sudo access to run all your requests.

Here are the packages I installed first:
ubuntu $ sudo apt-get update && sudo apt-get install gcc make apt-file unzip emacs23-nox openssh-server wpasupplicant wireless-tools acpid

Setup wireless networking

To keep the device mostly portable I opted for wireless networking in my home. I did not use the Ubuntu standard networking because I did not want the device to wait for an IP address on boot. I'd rather it connect after the plugin is loaded.  Because I run WPA2 at home there's a bit more work to setup a WiFi connection.

I created /etc/wpa_supplicant.conf with the following contents. You need to generate your own file with wpa_supplicant:
Change the WiFi module parameters to enhance performance over power savings. Create the file /etc/modprobe.d/ath9k.conf with the following:
options ath9k nohwcrypt=1 blink=1 btcoex_enable=1 enable_diversity=1
And now we make sure networking starts at boot by adding the following lines to /etc/rc.local:
# starts wifi in the background
/sbin/wpa_supplicant -B -i wlan0 -Dwext -c /etc/wpa_supplicant.conf
/sbin/iwconfig wlan0 power off
/sbin/dhclient -nw wlan0
Change the last line to assign a static IP address if you want to be able to reliably ssh into your device. Since you've already installed openssh-server you are all set to accept incoming connections.

Power button

I wanted the power button to immediately and safely power down the device when pressed. This minimizes the potential for the Linux filesystem to become corrupted and require manual maintenance. The acpid package is already installed and all it takes is a minor edit to the acpi daemon settings.  Edit the file /etc/acpi/events/powerbtn and make the following modification:
# action=/etc/acpi/powerbtn.sh

Now a quick tap of the power button will immediately power off your Chromebox. 

Audio Priorities

Ted Felix has a great introduction to Audio and MIDI on Linux. From this guide I decided to use Jack version 2.x. First we need to install it:
ubuntu $ sudo apt-get update && sudo apt-get install jackd2 alsa-utils alsa-base
When it installs be sure to answer <YES> to real-time audio priority. It will setup the audio group and configure process limits in /etc/security/limits.d/audio.conf.

Now we just need to add the default user 'user' to the audio group. (If you decide to use a different username for audio you'll need to adjust the command accordingly).
ubuntu $ sudo gpasswd -a user audio

Next we need to check for the audio interface. You should see the following output when running this command:
$ cat /proc/asound/cards
 0 [MID            ]: HDA-Intel - HDA Intel MID
                      HDA Intel MID at 0xe0710000 irq 62
 1 [PCH            ]: HDA-Intel - HDA Intel PCH
                      HDA Intel PCH at 0xe0714000 irq 64
The important line in my case is number 1 that contains "PCH". The number 0 interface is the HDMI digital audio path. A quick test of the audio will tell us it's working:
ubuntu $ amixer -c 1 set Master 100 unmute  # unmutes and maxes volume for card #1 above
ubuntu $ wget http://www.kozco.com/tech/piano2.wav  # fetch piano testfile
ubuntu $ aplay -D hw:1 piano2.wav  # play piano testfile on card #1 above
If you hear audio, great! If not you'll probably have to spend some time debugging why and is certainly beyond the scope of this tutorial. Google is your friend in this case.

Wine Setup

The installer for chrubuntu is a bit non-standard and doesn't do everything a normal Unbuntu installer does. You need to do a bit more to get Wine installed.
ubuntu $ sudo add-apt-repository ppa:ubuntu-wine/ppa
ubuntu $ sudo dpkg --add-architecture i386 
ubuntu $ sudo apt-get update
ubuntu $ sudo apt-get install wine1.7
I'm using the PPA version to get the latest optimizations to Wine. We have to add the i386 arch because Wine depends on several i386 libraries.

Compiling fsthost

We're almost done. Next we need to compile the latest trunk version of fsthost.  Use the "Download Snapshot" button on the top right of this page. This is necessary to prevent a segfault while executing.

You'll need to install a few more packages to meet the requirements to build:
ubuntu $ sudo apt-get install libglib2.0-dev libxml2-dev libxml-perl wine1.7 libjack-jackd2-dev libc6-dev-i386 wine1.7-dev libx11-dev dbus-x11
Edit the Makefile and disable building 32-bit, LASH, and GTK. I had trouble with all three of these options and found this to be the simplest way to a compiled fsthost64 binary.  Here's the diff generated to the Makefile:
ubuntu $ diff -u fsthost/Makefile.orig fsthost/Makefile
--- fsthost/Makefile.orig       2015-04-14 14:01:19.623081021 +0000
+++ fsthost/Makefile    2015-04-14 13:56:24.983086298 +0000
@@ -1,11 +1,11 @@
 ### Generated by Winemaker ... and improved by Xj ;-)
 SRCDIR             := .
 SUBDIRS            :=
-PLAT               := 32
-GTK                := 3
+PLAT               := 64
+GTK                := 0
 VUMETER            := 0
 LBITS              := $(shell getconf LONG_BIT)
-LASH_EXISTS        := $(shell if pkg-config --exists lash-1.0; then echo yes; else echo no; fi)
+LASH_EXISTS        := no # $(shell if pkg-config --exists lash-1.0; then echo yes; else echo no; fi)
 #LAST_EXISTS := 'no'

 # Modules
@@ -97,9 +97,9 @@

 # On 64 bit platform build also fsthost64
-ifeq ($(LBITS), 64)
-PLAT               += 64
+#ifeq ($(LBITS), 64)
+#PLAT               += 64

 EXES               := $(PLAT:%=fsthost%)
Now all you should need to do to compile is run make wherever you unziped the source:
ubuntu $ make

Setup your VSTi

I chose Zebra2 because I love it and because u-he have very unobtrusive plugins that only use a simple serial file to register and do not add keys to the Windows Registry.

All I had to do was Zip up the installed Windows VST directory and unzip it to ~/Zebra2 on my Linux machine. It's already registered to me and remembers my MIDI mappings for XY controllers and master volume. It looks like this:
ubuntu $ ls -F $HOME/Zebra2
Zebra2.data/    Zebra2(x64).dll

Running it all

It only takes a few commands to get everything working:
ubuntu $ eval $(dbus-launch --auto-syntax)
ubuntu $ amixer -c 1 set Master 100 unmute
ubuntu $ jack_control exit
ubuntu $ jackd -R -d alsa --device hw:1 --rate 48000 --period 256 --midi raw &
ubuntu $ $HOME/fsthost/fsthost64 -V "$HOME/Zebra2/Zebra2(x64).dll" &
At this point you should be able to plug in your favorite USB MIDI Keyboard and play notes!

The settings above give about a 5.5ms latency to the MIDI channel. If you want even faster response change --period to 128. Be warned though, this increases CPU usage.

Starting it at boot

I made a slightly more refined script to get the whole thing started at boot:
ubuntu $ $ cat ~/run-vst.sh

if [ -n "$1" -o -n "$(pgrep jackd)" ]; then
    kill $(pgrep fsthost64.so)
    kill $(pgrep jackd)
    [ -n "$1" ] && exit 0

export DISPLAY=:0
eval $(dbus-launch --auto-syntax)

card_num=$(grep ' - HDA Intel PCH$' /proc/asound/cards |
    awk '{print $1}')
amixer -c $card_num set Master 100 unmute

jack_control exit
jackd -R -d alsa --device hw:$card_num --rate 48000 \
    --period 256 --midi raw &>/dev/null &

while [ -z "$(pgrep jackd)" ] ; do
    sleep 0.2

$HOME/fsthost/fsthost64 -V "$HOME/Zebra2/Zebra2(x64).dll" \
    &>/dev/null &

And I launch it on boot with the following addition to /etc/rc.local:
# Start the plugin
runuser -u user /home/user/run-vst.sh
When all is said and done I can boot to Zebra in just under 10 seconds. Not bad at all. :)

I change patches with MIDI program change message and morph the patch with the pre-mapped XYs from my Windows setup.

Overall Performance

Zebra2 is a very CPU-efficient synththesizer. The default sawtooth sound plays without any issues on the Chromebox easily playing multiple notes and chords. However, more complex patches, heavy use of FX, arpeggiator, or CPU-intensive filter modes like the XMF can quickly cause the little Intel Celeron to hit max CPU.

Some patches will need tweaking to bring their CPU down. As long as presets are created with a mindset of reducing CPU usage I see this as a very functional setup.

If you are concerned about CPU there is an Intel i3 Chromebox made by Dell and the Intel NUC has a Core i5 option: something I may consider in a future build.

Items for further exploration

Ubuntu has a realtime-kernel option. I didn't use it because the Chromebox is doing tricky things to boot and is most certainly not doing the usual grub2 PC setup.

Completely replace the firmware with SeaBIOS. I chose not to do this for fear of bricking the device and I am uncertain to its level of support.

fsthost -S telnet option. The -S option allows you to telnet to a specified port and view/alter VST settings. I haven't tried this at all but it looks like an interesting way to interact with the plugin. Definitely worth examination.

Multicore usage. I haven't explored this at all and have no idea if plugins that attempt to execute in a multi-threaded manor will actually consume other cores. I might try using u-he's Diva to see what happens. I'm not sure how I will enable multicore though since I don't have a gui and I don't think that button can be MIDI learned.

Using the device with other plugins. I am considering an install for Camel Audio's Alchemy. Now that the company was acquired by Apple and Alchemy is discontinued it would provide for a stable way to use the plugin for years to come.

Monday, August 25, 2014

Using QEMU to run Ubuntu ARM 64-bit (aarch64)

There are several low-cost 32-bit ARM devices on the market today. Unfortunately the same is not true for 64-bit ARM devices. Thankfully the good people working on QEMU have recently added aarch64 support. An excellent blog post by Alex Bennée outlines the steps needed to get QEMU installed on your Linux system ready to run 64-bit ARM code. The goal of this post is to take the installation one step further and get a full-fledged Ubuntu Linux distribution running in QEMU.

The reason I chose Ubuntu is for one important reason: they publish nightly builds of their OS for cloud machine deployment. These images contain a full base OS install and only require a bit of tweaking. Once deployed in QEMU they work just like any other Ubuntu installed device.

When researching this I used a git checkout of QEMU HEAD as 35858955e6c6f9ef41c199d15457c13426ac6434. I also used Ubuntu 14.04 LTS on an x86_64 system to run QEMU and to prepare the Ubuntu guest OS image.

First up we need to fetch an appropriate image. I like using LTS as Ubuntu is a little more careful about package updates/changes to their long-term releases. Grabbing the latest nightly image means fewer package updates once we're done customizing the image. With that in mind I selected this 300MB image. It's important to note the "-disk1.img" files are qcow2 images which are QEMU's native disk format.

Once the image is downloaded we need to resize the filesystem. By default Ubuntu only leaves about 400MB of space free for use and becomes pretty cramped for any real work. We have to perform three steps to resize the image. Resize the qcow2 disk size to whatever you want. I used 8GB:
$ $HOME/git/qemu/qemu-img resize trusty-server-cloudimg-arm64-disk1.img 8G
Image resized.
$ $HOME/git/qemu/qemu-img info trusty-server-cloudimg-arm64-disk1.img | grep size:
virtual size: 8.0G (8589934592 bytes)
disk size: 301M
cluster_size: 65536
Next, resize the disk partition layout. We use qemu-nbd to allow us to mount the filesystem as a block device. Ubuntu's kernel has support for nbd compiled in and we just need to load the module.
$ sudo modprobe nbd max_part=8
$ sudo $HOME/git/qemu/qemu-nbd --connect=/dev/nbd0 trusty-server-cloudimg-arm64-disk1.img
$ sudo fdisk /dev/nbd0
Command (m for help): p
Disk /dev/nbd0: 8589 MB, 8589934592 bytes
4 heads, 32 sectors/track, 131072 cylinders, total 16777216 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x0008bae4
Device Boot Start End Blocks Id System
/dev/nbd0p1 * 2048 2889727 1443840 83 Linux
In the above output we need to save off the starting block of the partition /dev/nbd0p1. It's the number 2048 in the last line above. I've only ever seen 2048 but it can possibly change in the future so be sure to check what it is with fdisk. Now we delete (and re-create) the partition entry:
command (m for help): d
Selected partition 1
Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-16777215, default 2048): 2048
Last sector, +sectors or +size{K,M,G} (2048-16777215, default 16777215):
Using default value 16777215
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
Note the use of 2048 as the First Sector as mentioned previously. The last step is to resize the actual filesystem contained within the first partition:
$ sudo e2fsck -f /dev/nbd0p1
$ sudo resize2fs /dev/nbd0p1
While we have the filesystem available through nbd we need to obtain the Linux kernel and initrd.img files:
$ sudo mkdir -p /mnt/virt && sudo mount /dev/nbd0p1 /mnt/virt
$ cd /some/path
$ sudo cp /mnt/virt/boot/*-generic .
$ sudo chown $USER:$USER *-generic
$ ln -s *disk1.img disk1.img
$ ln -s vmlinuz-* vmlinuz
$ ln -s initrd.img-* initrd.img
As far as I can tell QEMU doesn't like having to search the virtual hard drive for the bootable images which necessitates that step. The symlinks just make invocations to QEMU a bit easier and shorter. And now we can remove the nbd mounted filesystem:
$ sudo umount /mnt/virt && rmdir /mnt/virt
$ sudo $HOME/git/qemu/qemu-nbd -d /dev/nbd0
We're now ready for our first boot of QEMU. Here's the command line I used:
$ $HOME/git/qemu/aarch64-softmmu/qemu-system-aarch64 \
    -machine type=virt -cpu cortex-a57 -smp 1 -m 2048 -nographic \
    -rtc driftfix=slew -kernel vmlinuz \
    -append "console=ttyAMA0 root=LABEL=cloudimg-rootfs rw init=/bin/sh" \
    -initrd initrd.img -device virtio-scsi-device,id=scsi \
    -device scsi-hd,drive=hd -drive if=none,id=hd,file=disk1.img
The boot -append line is important here: the LABEL= is necessary for Unbuntu to mount the root filesystem and we short-circuit the boot process into an emergency shell. If we didn't boot into a shell the image would try to come up as a cloud slave node and try to phone home. Not very useful for our needs. If all goes well you should see a prompt like the following:
Begin: Running /scripts/init-bottom ... done.
At this point there are some customization steps that need to be done before we can boot this image normally. In order to set things up we have to mount the soon-to-be root filesystem and then chroot into it:
# mount /dev/sda1 /mnt
# chroot /mnt /bin/bash
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell
Set root's password. Ubuntu images don't have one:
root@(none):/# passwd root
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
Remove the cloud packages from the Ubuntu image. This speeds up boot time and also gives you a console login prompt:
root@(none):/# dpkg -P cloud-guest-utils cloud-init pollinate landscape-client landscape-common apparmor apport apport-symptoms ufw
root@(none):/# rm -f /etc/init/pollinate.conf /etc/default/pollinate
root@(none):/# rm -rf /etc/cloud
Disable the VESA framebuffer. It doesn't work with QEMU and just produces boot-up errors.
root@(none):/# perl -p -i -e 's/modprobe -q -b vesafb/true/g;' /etc/init/udev-fallback-graphics.conf
Allow all users to SSH into the OS with passwords and keys:
root@(none):/# perl -p -i -e 's/^PermitRootLogin.+$/PermitRootLogin yes/g;' /etc/ssh/sshd_config
root@(none):/# perl -p -i -e 's/^PasswordAuthentication.+$/PasswordAuthentication yes/g;' /etc/ssh/sshd_config
Generate new, unique SSH server keys for each guest OS:
root@(none):/# rm -f /etc/ssh/ssh_host_*_key*
root@(none):/# dpkg-reconfigure openssh-server
Change your timezone with the ncurses menu:
root@(none):/# dpkg-reconfigure tzdata
I also recommend setting up NTP on the Guest OS, especially if you're sharing files between the guest and host OS. The change below will tell Ubuntu to synchronize time with ntp.ubuntu.com, the default setting in the "NTPSERVERS=" variable in the same file:
root@(none):/# perl -p -i -e 's/^NTPDATE_USE_NTP_CONF=.+$/NTPDATE_USE_NTP_CONF=no/g;' /etc/default/ntpdate
(optional) I like the next two commands to make root logins much more responsive. Ubuntu will try to gather OS information on every login and display it to you if you don't disable it. These commands remove that feature. Don't run these if you like seeing Ubuntu's login information:
root@(none):/# >/root/.hushlogin
root@(none):/# rm -f /etc/update-motd.d/*
(optional) Setup a shared storage location to copy files to/from the Guest and Host OS. QEMU has a nice feature called virtio to accomplish this:
root@(none):/# mkdir /shared
root@(none):/# vi /etc/rc.local
/bin/mount -t 9p -o trans=virtio lvdisk0 /shared # add this somewhere before the 'exit 0' line
Don't add this entry to /etc/fstab. There seem to be module loading conflicts during boot time which prevent it from succeeding.

(optional) Create a non-root admin user:
root@(none):/# adduser ubuntu
Is the information correct? [Y/n] y
root@(none):/# usermod -aG adm ubuntu
root@(none):/# usermod -aG sudo ubuntu
(optional) Statically set the hostname. You also can choose to pass in a hostname at boot (see below):
root@(none):/# echo "newhost" >/etc/hostname
When you're done you need to exit the chroot and halt the system to perform the final normal boot:
root@(none):/# exit
# sync; sync; sync; halt
I've had the halt command fail on me which is why I have the old-fashioned three syncs before executing it.

You can exit the QEMU console with "ctrl-a c" to get the QEMU console, and then typing "quit". Finally you can try to boot the image normally. Here's an example command line I use:
$ BASE=$PWD; HOST=ubuntu; mac=52:54:00:00:00:00; sshport=22000
$ $HOME/git/qemu/aarch64-softmmu/qemu-system-aarch64 \
    -machine type=virt -cpu cortex-a57 -smp 1 -m 2048 -nographic \
    -rtc driftfix=slew -kernel "$BASE"/vmlinuz \
    -append "console=ttyAMA0 root=LABEL=cloudimg-rootfs rw" \
    -initrd "$BASE"/initrd.img \
    -device virtio-scsi-device,id=scsi -device scsi-hd,drive=hd \
    -drive if=none,id=hd,file="$BASE"/disk1.img \
    -fsdev local,id=vdisk0,path="$BASE"/shared,security_model=none \
    -device virtio-9p-device,fsdev=vdisk0,mount_tag=lvdisk0 \
    -netdev user,hostfwd=tcp::${sshport}-:22,hostname=$HOST,id=net0 \
    -device virtio-net-device,mac=$mac,netdev=net0 \
    -monitor telnet:,server,nowait \
    -serial telnet:,server,nowait
There are a few differences from the previous QEMU invocation worthy of note. I have setup a virtio shared directory at $BASE/shared using the line starting with "-fsdev". The netdev line sets a MAC address for the device and also port-forwards traffic on the host OS port $sshport to the guest OS port 22. This port-forwarding allows both remote and local SSH connections to the running guest OS over SSH. The HOST= parameter sets the guest OS hostname.

The "-monitor" and "-serial" lines give you telnet-like interfaces to the QEMU console and Linux console respectively. It's a great way to expose those interfaces as long as your network is secure, like a lab environment. I wouldn't expose those interfaces if you don't trust others on your network as those ports will act like telnet sessions without any authentication.

At this point you should be able to SSH into $sshport on the QEMU host and login as either root or your admin user. You can then apt-get and natively compile to your heart's content.

If you want to run multiple instances of QEMU on the same host just change $sshport and $BASE to unique values and copy the kernel, initrd.img, and disk1.img files separately for each running instance. I've run 5 instances on the same host OS with no problems using this technique and I'm sure I could have run more as needed.

Feedback and comments are appreciated. I know it's a bit of a brain dump but I figured it'd be useful for others to have this information. Due to the scarce available of aarch64 hardware this is a great stop-gap until those cheap SoCs start hitting the market.

Have fun!