Archive for the ‘Linux’ Category

Grep/Regexp

remove all comments and empty lines from a php.ini

cat php.ini | grep -v "^;" | grep -v "^$"

Multiple external interfaces in Xen routed setup

For one of my virtual servers running under Xen in a routed setup, I was required to have more than one external interface, meaning the VM should be available on the net on multiple IP addresses.

First, I tried to assign multiple interfaces to the VM, but that didn’t really work as the kernel was confused on which route it was supposed to send out data. Turns out it’s even easier. Let’s say you have the two IPs 1.2.3.4 and 5.6.7.8 which you want to assign to your domU. First, you need to put a line like this into your domU config:

vif=["ip=1.2.3.4/32 5.6.7.8/32,mac=00:1D:92:AA:BB:CC"]

Within the domU, you would then create an interface (eth0) with a static configuration for the IP 1.2.3.4. Afterwards, you would simply create an alias with the second ip:

ifconfig eth0:1 5.6.7.8 up

Now you should be able to reach your domU on both interfaces.

Chatroulette: Man-in-the-middle

I’ve always wondered whether it’s possible to put a man-in-the-middle scheme on Chatroulette, connecting two other people and watching them chat. Well, the answer is, of course, YES, but it took me a while to find the right software for the job. Now I finally did and its name is WebcamStudio.

WebcamStudio is a software for modifying existing webcam streams or creating fake webcams, it comes with a kernel module to create virtual webcams. There are several input sources available, the most important one being the one called “Desktop”.

I will not give you detailed instructions of how to do it, as it probably violates the Chatroulette terms of use, but with WebcamStudio it is possible to connect two different chat partners, while sitting in the middle and watching them talk. The only obstacles left is sound (probably not an issue if you’ve got a sound card with a mixer supported by Alsa) and chat content. For the latter one, you’ll probably need to write a Greasemonkey script and a little server which exchanges messages between browser instances.

Fixing a wrong Keymap in FreeNX-Sessions

Using a FreeNX server, I encountered a problem with a wrong keymap being used within the FreeNX session, which results in certain characters not working. Especially annoying: The AltGr key, on my system, was interpreted as “Enter”.

In order to fix this problem, one needs to export the xmodmap from the client system (running the NX client) to the server and, afterwards, setting this xmodmap from within the session. Of course, this can also be automated.

First of all, you need to export the xmodmap of the client you’re connecting from:

xmodmap -pke > nxclient.xmodmap

Now you need to transfer it to the system you’re using as the server. Afterwards, whenever you start a NX session, you need to call the following command from within the session (assuming you put the file obtained above in your home directory):

xmodmap ~/nxclient.xmodmap

Of course, you can automate this by calling the command from within the .profile file. The following code only makes sense though, when you connect from clients which all use the same keymap:

if [ -n "$NXSESSIONID" ]; then
 /usr/bin/xmodmap ~/nxclient.xmodmap
fi

This snippet works by checking for the environment variable $NXSESSIONID, which indicates that the session is running from within NX.

Eclipse Bug: RenderBadPicture Bug

Today, I encountered an annoying bug in Eclipse. All of the sudden (without any updates being carried out that I remember), Eclipse would simply die and print out only the parameters the JVM was initialized with.
Only when starting Eclipse in a shell will you see an error message which looks somewhat like this:

The program 'Eclipse' received an X Window System error.
This probably reflects a bug in the program.
The error was 'RenderBadPicture (invalid Picture parameter)'.
(Details: serial 43551 error_code 158 request_code 148 minor_code 7)
...

After a short websearch, I found this Eclipse bugreport, which also contains a solution to the problem. It appears that Eclipse tries to load a wrong/nonexistent version of Mozilla’s XULRunner, which is responsible for rendering HTML parts within the IDE. Obviously, I encountered the problem as the autocompletion tried to display HTML code within a popup.

The solution is simple, you need to point Eclipse to the correct version of XULRunner. On a current openSUSE 11.3 installation, I used the direct path to XULrunner 1.9.2, the default is to rely on the update-alternatives system – and I’m not sure whether this might have caused the problem.
So if you encounter the error, you simply need to add the following line to your eclipse.ini file:

-Dorg.eclipse.swt.browser.XULRunnerPath=/usr/lib/xulrunner-1.9.2/xulrunner

On a 64 bit installtion, it needs to look like this, though:

-Dorg.eclipse.swt.browser.XULRunnerPath=/usr/lib64/xulrunner-1.9.2/xulrunner

Mysqldump and Error “Unknown command ‘\0′”

When you’re using mysqldump to put a complete database into a textfile and try to import that file into another MySQL server, you might come across the error message “Unknown command ‘\0‘” on importing the database.

The most probable cause to this is a database which contains (large) blobs, as they somehow might not get encoded to the right format.

Easy solution to this problem: Use the –hex-blob option:

mysqldump -uusername -p --hex-blob db_name > db_name.sql

This will force mysqldump to encode all blobs in binary format, which won’t get messed up in encoding problems.

Installing openSUSE (11.3) as Xen guest (domU) under CentOS 5.5 using HVM

Today, I had the mindnumbing task of installing openSUSE (current version 11.3) as a virtual Xen guest (domU) within a CentOS 5 dom0. There are, of course, multiple solutions to this problem and the most convenient is probably the one found here.

However, my setup is a little bit different as – thanks to my hoster’s secure routing – all my domUs need to route their traffic through the main IP of the dom0, thus preventing me from setting the right network information (netmask 255.255.255.255 and a point-to-point route to the default gw) in YaST installation.

As a solution to the problem, I first installed openSUSE as a HVM domain from a downloaded DVD-ISO. Afterwards, I made some small changes and ran the system in paravirtualized mode. In order to do so, you need twice as much space as the installation takes, since you need to copy the data from the HVM disk to the paravirtualized disk.

So for this example, let’s say you have a volume group named opensuse with two logical volumes disk1 and disk2, where you start the HVM on disk2 and disk1 will be your root device for the paravirtualized machine. You’ll also need to download the openSUSE DVD image in ISO format.

Create a HVM domain

First of all, we need a HVM enabled config. Be sure to enable PAE, otherwise you’ll get an error when booting from the DVD. Here’s the config I used:

name="opensuse1"
memory=768
vcpus=1
on_poweroff="destroy"
on_reboot="restart"
on_crash="restart"
disk=["phy:/dev/opensuse/disk2,xvda,w", "file:/path/to/openSUSE-11.3-DVD-x86_64.iso,xvdb:cdrom,r"]
kernel="/usr/lib/xen/boot/hvmloader"
builder="hvm"
device_model ="/usr/lib64/xen/bin/qemu-dm"
boot="d"
sdl=0
vnc=1
vncconsole=1
usbdevice="tablet"
shadowmemory=8
pae=1
acpi=0

Of course, you need to adapt the path to your local setting. If you install the domU on a remote host, make sure you’ve got X11 forwarding enabled, otherwise the VNC window will not pop up. You’ll be greeted with the standard openSUSE boot promt, so you can just carry out the installation as you normally. Just be sure to include the package kernel-xen in the installation. And if you’re smart, you’ll only create a single partition (no swap etc) on which the system gets installed, saving you the hassle of copying data from several partitions.

Copying data from HVM disk to paravirtualized disk

Once the domU has been set up, shut it down. Now, you need to copy all files from the HVM disk (/dev/opensuse/disk2) to the disk designated for the paravirtualized domain (/dev/opensuse/disk1). If you haven’t done this yet, put a file system of your choice on that disk, e.g.

mkfs.ext3 /dev/opensuse/disk1

Since the HVM domain uses a complete disk structure (including MBR etc) instead of a single partition, you need to mount only a single partition out of the disk structure. You can either specify an offset in the mount command or, if you’re a little lazy, use kpartx:

kpartx -av /dev/opensuse/disk2

This will create mappings for all partitions found in /dev/opensuse/disk2. If you only created one partition as I suggested, you’ll find the mapping at /dev/mapper/opensuse-disk21.

Now, mount the two disks to seperate mount points and begin copying:

mkdir /mnt/disk1
mkdir /mnt/disk2
mount /dev/opensuse/disk1 /mnt/disk1
mount /dev/mapper/opensuse-disk21 /mnt/disk2
cp -ax /mnt/disk2/* /mnt/disk1

After the process has completed, you need to make some small changes.

Change the file /mnt/disk1/boot/grub/menu.lst to boot the Xen enabled kernel. It only needs one entry like this:

default 0
timeout 0

title Xen
  root (hd0,0)
  kernel /boot/vmlinuz-xen root=/dev/hda1 splash=none showopts vga=0x314 console=xvc0
  initrd /boot/vmlinuz-xen

Make sure the kernel and the initrd exist. The console parameter is also important, otherwise you won’t see anything on the Xen console. We also need to make sure to enable this console in two more files.

Edit /mnt/disk1/etc/inittab and change the following line

1:2345:respawn:/sbin/mingetty --noclear tty1

to look like this:

1:2345:respawn:/sbin/mingetty --noclear xvc0

Now edit /mnt/disk1/etc/securetty and append xvc0 to the list. The last change isin /mnt/disk1/etc/fstab: Make sure you set the root device to the one from the domU config (e.g. /dev/hda1), as YaST will have put something like /dev/disk-by-id… in there.

Afterwards, you can umount the two disks:

umount /mnt/disk1
umount /mnt/disk2

Starting in paravirtualized mode

You still need to adapt the domU’s config to start in paravirtualized mode. Here’s what my config reads like:

name="opensuse1"
memory=768
vcpus=1
on_poweroff="destroy"
on_reboot="restart"
on_crash="restart"
disk=["phy:/dev/opensuse/disk1,hda1,w"]
bootloader="/usr/bin/pygrub"
vif=["ip=192.168.1.1"]

The network setting looks slightly different, but you need to change this file anyway ;)

Now, you can simply start the domU and it will get created as a paravirtualized domain.

Cleanup & Conclusion

If you don’t plan on restarting the dom0 in the next days, you should clear the mapping kpartx created by invoking:

kpartx -dv /dev/opensuse/disk2

Also, you are now free to either remove disk2 or add it to the paravirtualized machine for additional space.

I know the solution described in this post is not the most elegant one, but it worked for me right now. Hope it helps someone else, too…

Backticks in MySQL

I’ve always wondered, why software such as phpMyAdmin uses a pair of backticks around every table- or column-name, while you read a lot of SQL-statements that don’t.

The answer is quite simple and you’re likely to run into this issue sometime, if you’re creating a lot of MySQL-queries. Take the following (fictional) table name, for example:

Table-1

This is a valid table name in MySQL, so now let’s take a look what’s inside. You’re probably using a statement like:

SELECT * FROM Table-1;

And bang, you’ll get something like: “You have an error in your SQL syntax“. Why is that? If you take a closer look, you’ll see that the syntax is ambiguous. The parser does not understand if you want to reference the table named Table-1 or if you want to subtract 1 from the column named Table.

So to clarify things for the database engine, you must use a pair of backticks, which will escape all special characters contained within them:

SELECT * FROM `Table-1`;

By the way, the same rules apply if you have any other databases, tables or columns named with reserved keywords. A more classic example is to have a column named max or min (both MySQL-functions).

If you’re lazy, you can only add the backticks at the necessary places, for a more consistent look, you might as well add them everywhere. I’m not sure if this is part of any SQL-Standard or if it’s just MySQL specialty.

web2thumb: Create thumbnail of webpage without XServer

A while ago, I needed a tool that could create a screenshot/thumbnail of a webpage. There are actually quite a few available, some are webservices, others rely on the Internet Explorer, but the vast majority makes use of the WebKit rendering engine, which is conveniently integrated in Qt. Pretty much all the tools I came across have one minor flaw though: They require an XServer, at least a virtual one like Xvfb.

I initially used the webkit2png Python script created by Roland Tapken, which works just fine by using the Qt-Python bindings. But since I always wanted to do a little work in Qt, I started porting this handy little script to C++. In the meantime, I came across wkhtmltopdf, a really great tool for HTML to PDF conversion. They also use WebKit for rendering the content and initially had the same problem with the required XServer as well. But they added a little patch to Qt, allowing them to work without X, which is really great if you run a tool like this on a server (less memory/overhead required).

So I used their patch, added a little more dirty hacks to Qt for allowing JavaScript to work as well and the result is a little programm called web2thumb. If it is compiled against the patched Qt version, you can create thumbnails of websites without running X as long as you don’t enable the use of plugins like Flash or Java.

I am very aware of the fact, that there are other tools which do exactly the same as this tool (or even better), furthermore due to my limited C++ skills, the code quality is probably not that good.  I’ll put it out here anyway (licensed under the GPL v3), maybe someone finds it useful as well.
For your convenience, I’ll also include a version which is statically linked against the patch Qt library and boost_program_options, so you don’t have to patch & compile Qt yourself. It is packed with UPX, I’m currently limited to providing an i386 binary though.


Files

web2thumb, v. 0.1

QuickCommands

This is a list of useful command/parameter combinations I always keep forgetting.

Spellcheck a LaTeX document using aspell

aspell --mode=tex --lang=en_US -c fileToCheck.tex

Using find with commands

find . -iname “pattern” -exec rm -rf ‘{}’ \;

Add a new virtual interface

ifconfig eth0 add 1.1.1.1

Return top