mycroes

There's always time to play

Tuesday, January 13, 2009

Kernel Mode-Setting on Intel hardware

Today I was going to try kernel modesetting, or kms in short, on my machines with intel graphic chipsets. First I was going to try it on was my 'old' laptop with an intel 915 chipset. I configured a 2.6.29-rc1 kernel and tried it, but as soon as it wanted to change mode my screen would turn black and only show something as soon as X was started again. One time it did seem to have working consoles again after X was started, but that didn't seem reproducible.

I decided to move to the next victim, my MSI Wind with intel 945 chipset. I was hoping a newer chipset would work better, so I took the same route again and rebooted, and it worked! The only thing that didn't work was X, but I knew I had to get something near version 2.6 of the intel Xorg driver to have that working with kms again.

So while I installed new drivers I went looking for the differences between the kernel configurations of both machines. I started at the framebuffer section, because I knew I had the uvesafb driver compiled in on my 'old' laptop and I had intelfb compiled in on both laptops, so my first guess was to disable the uvesafb driver and try what happens. Immediate success, I had native 1680x1050 on the console on my 'old' laptop, and X would even start. I didn't have recent intel Xorg drivers however, so switching between X and consoles was still as slows as it had always been. But having the consoles at native resolution is already so much of an improvement, that you shouldn't really care.

Meanwhile, my MSI Wind had finished installing newer intel Xorg drivers and I was ready to start X, success again. Switching between X and consoles is simply amazingly fast. I have however noticed a small issue here, somehow gdm thinks the vertical resolution is about twice what it really is, so I only see the top half of the screen and thus no login box. Once I'm logged in the issue seems to persist and my cursor is invisible, but my terminal keyboard shortcut works and the terminal is completely visible, so I can live with that for now.

Next machine is my Asus EEEpc 701/4G surf, which features an intel 915 chipset again. I compiled a 2.6.29-rc1 kernel again, using the config I made last week when I created my bootable usb stick. Copied the kernel to my usb stick, booted and... failure. Same kind of problem as on my 'old' laptop, screen turns black. Because I use the gentoo minimal image I never get to a X server either, so it stays black. Back to the kernel config, what went wrong? Somehow console framebuffer support was turned off, and I also had some other options turned off like low level display driver support. I don't think the latter really matters, but the first was an issue. After that it booted fine, except for the fact that I now used the in-kernel squashfs support which was at version 4.0 already, while the patch I previously used to patch my kernel sources was at version 3.4. Seems like the disk-format changed in between, so I can't mount the image, but I can get the fancy native resolution console.

Everything seems to work now, except for one thing, my 'old' laptop refuses to start X after updating to one of the 2.6 beta intel graphic drivers. Seems to get a hard lock, so I'll have to dig into that. I might have some newer packages installed on my msi wind, so I'll start by comparing versions and then I'll see. So far I'm happy.

Other issues I noticed are that on all the machines I seem to get some output from the drm driver that something is going wrong, including occasional call traces where something apparently went terribly wrong, but almost everything works. I also noticed that after a while the screen blanks, even though I think it shouldn't. Even worse, pressing a key won't turn it back on again, but switching to console, pressing a key does. After that I can switch back to X again and it works.

So now I only have one thing left to do, and that's check if it also works on the MacBook me and my girlfriend use together, but I guess I'll have to do some more work on my bootable usb stick before that's going to work.

Wednesday, January 7, 2009

Creating a multiboot USB-stick using syslinux

Since I have an eeepc-701, I need a way to boot it from external media so I can mess with it. I have a Corsair Flash Survivor (16GB), so I'm going to use that for rescue-goodness. I'll give step by step instructions on how to create a similar boot disk. In these instructions I'll assume that /dev/sdc is the usb drive, /dev/sdc1 is the partition we're using for the filesystem. Further mine will be mounted at /media/Corsair and I'll also assume you can find the appropiate packages if you haven't gotten them yet (dosfstools and syslinux in gentoo). Last but not least most of these commands will need root permissions, so prepend them with sudo if that's your thing, or become root and do it all as root.

Format the usb drive as fat 32
Of course there are other options than fat 32 to create a bootable usb drive, but syslinux needs a FAT filesystem to work.
# mkdosfs -F 32 -n Corsair /dev/sdc1

Mount the usb drive
Unplugging and replugging will work for me as I use Gnome with automount support, should probably work for most people using a larger desktop environment. If not:
# mkdir -p /media/Corsair
# mount /dev/sdc1 /media/Corsair

Create a syslinux directory
Syslinux needs somewhere to store its files, I prefer doing that in a directory so I don't get a cluttered view every time I look at the files on my usb drive.
$ mkdir /media/Corsair/syslinux

Unmount and install syslinux
Unmount in whatever way suits you, just don't unplug yet. The -d option tells syslinux to use our directory 'syslinux' to store its files, just make sure the directory is there before running syslinux.
# umount /media/Corsair
# syslinux -d syslinux /dev/sdc1

Mount again
Remount the device to put more on it than just syslinux.
# mount /dev/sdc1 /media/Corsair

There's only two things left to do now:
1. Move images to the usb drive
2. Create the syslinux.cfg file

Move images to the usb drive
I create two directories to start with: gentoo and gparted. Inside the gentoo directory I put the kernel, initrd and squashfs from the minimal install cd and renamed them to include the release number in the files. At first I named my kernel just 'kernel-2008.0', but that seems to confuse syslinux so I renamed it to kernel-2008.0.img in the end. Better extensions suggestions for the kernel are welcome. I also copied all the .msg files from the gentoo cd to the syslinux directory I created on the root of the filesystem, so I can press F1-F6 and see irrelevant gentoo messages, however, changing these is easy and will be done when I have more time. It also seems gentoo needs a file called 'livecd' on the root, else it won't find the media as the fs media. It's cluttering the drive, but removing the file will just cause gentoo not to find a valid root fs. Last but not least I compiled a kernel that has support for most of the hardware in my eeepc 701/4G surf, and named that 'kernel-eee' straight on the root of the USB drive. I needed at least 2.6.28 so I could have support for the onboard network card, whatever way I'm installing it I'm gonna want network support. The kernel is monolithic (no modules and no module loading support) and works fine with the gentoo 2008.0 squashfs.

Inside the gparted directory I again put the kernel, initrd and squashfs files. I just took them from the livecd, left names as they were.

Create the syslinux.cfg file
I took most of this file straight out of the isolinux.cfg that came on the gentoo cd and customized where necessary.
default gentoo
timeout 150
prompt 1
display boot.msg
F1 kernels.msg
F2 F2.msg
F3 F3.msg
F4 F4.msg
F5 F5.msg
F6 F6.msg
F7 F7.msg

label eee
kernel /kernel-eee
append root=/dev/ram0 init=/linuxrc dokeymap looptype=squashfs loop=/gentoo/minimal-2008.0.squashfs cdroot initrd=/gentoo/initrd-2008.0.igz

label gentoo
kernel /gentoo/kernel-2008.0.img
append root=/dev/ram0 init=/linuxrc dokeymap looptype=squashfs loop=/gentoo/minimal-2008.0.squashfs cdroot initrd=/gentoo/initrd-2008.0.igz vga=791

label gentoo-nofb
kernel /gentoo/kernel-2008.0.img
append root=/dev/ram0 init=/linuxrc dokeymap looptype=squashfs loop=/gentoo/minimal-2008.0.squashfs cdroot initrd=/gentoo/initrd-2008.0.igz

label gparted
kernel /gparted/vmlinuz
append initrd=/gparted/initrd1.img boot=live union=aufs noswap vga=791 ip=frommedia nolocales live-media-path=gparted

label memtest86
kernel memtest86

Syslinux uses relative paths by default, so if you use no '/' syslinux will look for the files inside the dir syslinux is. The solution is simple, use absolute paths for the kernel and initrd and it's all OK. Most important change made to this file is the append lines. For gentoo, the loop parameter needs to point to the squashs image, so I changed that to /gentoo/minimal-2008.0.squashfs (name of the image in my case). I added a line for my eeepc kernel, which just boots into a gentoo minimal cd.

Next was gparted. Gparted supports a 'live-media-path=' parameter, this needs to be used when the gparted squashfs image is not directly in the root of the filesystem on the usb drive. Other than that gparted just looks for any file with name *.{squashfs,ext2,ext3,jffs2} so the name doesn't really matter as long as the extension hasn't changed.

I also left the memtest line in place, added the memtest image to the root of the drive and my rescue-drive is mostly done.

It should also be possible to do the same thing using grub, except for the F1-F6 messages and the default message (or can grub do that too?). I'm definately gonna add grub to it, probably replacing syslinux with grub. Using grub might also allow for use of other filesystems on the drive, which might end up being a big plus too. Come back again to read about that, once I've tried it myself.

Sunday, January 4, 2009

Configuring a linux kernel for another architecture than host

On my 64 bit gentoo installation I wanted to compile a kernel for 32 bit machines (specifically my eee pc 701), but somehow menuconfig was as smart to notice I was running a 64 bit machine and so showed me stuff to build a 64 bit kernel. After a quick search I found the solution:
make ARCH=i386 (oldconfig|menuconfig|)

Sunday, December 21, 2008

GDM in Xnest

I wanted to change my GDM theme and watch it without logging out, so the logical thing for me was to run it inside an Xnest. I had no clue how to do that though, but after some searching I found the following:
$ gdmflexiserver --xnest

Voila, you never need to know more...

Thursday, September 4, 2008

Using ext4 on Gentoo (and probably other distro's)

Keeping it short:
when creating ext4 filesystems with e2fsprogs' (1.41.1 works great for me) mkfs.ext4 I couldn't mount them. The error I found in dmesg was
[ 5994.790520] EXT4-fs: loop0: not marked OK to use with test code.
After a bit of googling I found that to use ext4 filesystems with the current in-kernel ext4dev driver you need to tell the filesystem it can be used by an experimental driver. The option for mkfs.ext4 to do this is -E test_fs. After this you can mount your ext4 filesystems without any issues (not without specifying ext4dev as filesystem with -t though).

Friday, August 8, 2008

Pairing the Itunes Remote app with your own server

I got a bit further today, but first I'll start with information you would need to try this yourself.

You need a working installation of a zeroconf implementation, the implementation of choice is avahi in my case. You need to set up two services for your zeroconf implementation, I'm using avahi so I made two files in /etc/avahi/services:

touch-remote.service:
<?xml version="1.0" standalone='no'?><!--*-nxml-*-->
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
<name>Long magic iPod identifier</name>
<service>
<type>_touch-remote._tcp</type>
<port>50508</port>
<txt-record>DvNm=Test remote</txt-record>
<txt-record>RemV=10000</txt-record>
<txt-record>DvTy=iPod</txt-record>
<txt-record>RemN=Remote</txt-record>
<txt-record>txtvers=1</txt-record>
<txt-record>Pair=pairing code</txt-record>
</service>
</service-group>

The first bold string is to be replaced with your iPod's id exported in _touch-remote._tcp on your ipod (it actually is the service name, as the xml shows). The next bold string is to be replaced with the paircode. Because ultimately my goal is to have the remote app talk to my own application, I will need to copy this from the ipod as well. I always first click 'Add library' (I assume that's what it is, I've got mine set to Dutch...), then copy the id, then save the file (make sure to keep the pincode screen open, new pincode means new pairing code). Now on my laptop with iTunes (in Windows) I immediately see 'Test remote' appear at the remotes list, isn't that great?

touch-able.service:
<?xml version="1.0" standalone='no'?><!--*-nxml-*-->
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
<name>47975973649ECA79</name>
<service>
<type>_touch-able._tcp</type>
<port>3689</port>
<txt-record>Ver=131072</txt-record>
<txt-record>DvSv=1905</txt-record>
<txt-record>DbId=47975973649ECA7B</txt-record>
<txt-record>DvTy=iTunes</txt-record>
<txt-record>OSsi=0xD97F</txt-record>
<txt-record>txtvers=1</txt-record>
<txt-record>CtlN=Test library</txt-record>
</service>
</service-group>

This file has some magic strings in it too, but these are less interesting. Name is obviously an identifier, this is how the remote will find the server after the server application has requested pairing. DbId is almost the same as name, in my case it happened to be 3 more than DbId. I changed both numbers (but kept the difference the same) because I would otherwise immediately get an error that the remote was not paired, seems it was connecting to iTunes (duplicate IDs). CtlN is the name the remote app will list as the library name, it actually works...

Now on to the next part, the connection.
When you click on a remote in iTunes it asks you to enter a pincode. As soon as you enter the code iTunes will make a http request to the port the _touch-remote._tcp service is running on (changes everytime on ipod touch, always seems to be port 505xx). Actually it probably should be seen as a DAAP request, but since DAAP is just special http it won't really matter. The request will feature a pairing-code and a database id, both as GET vars. The pairing code seems to be a hash of the pairing code the _touch-remote._tcp service lists combined with the pin code. I have not even tried to figure out how it's calculated, except for the fact that it's just as long as a md5 sum.

In the case with our fake remote, we can run netcat on port 50508 (port I chose/used), to do this, run 'nc -l -p 50508' in a terminal. Now if I click on 'Test remote' and enter my pin, iTunes sends a nice pairing-code. Because we used the service pairing code from the real remote for our fake remote, and typed the correct pincode, the remote will be happy if we forward the request. You can do this by just copying/pasting the request iTunes made into telnet or netcat connected to the real remote app. When forwarding the request, be sure to replace the database id iTunes passes along for the id you entered as name in touch-able.service. Now you should have paired your iTunes remote with your computer.

This pairing process is currently quite complicated, but I don't think I'm able to solve the hashing riddle without reverse engineering iTunes or the remote application, but I'm not sure I would actually manage to do that and also I'm not sure if it would be legal to do so. Not that I really care that much about it being legal (just applies to this case), if someone else would solve the issue I'd be happy to use it.

So what's next? Well now that it's paired the remote will connect to the port listed for the touch-able service and request server info ('GET /server-info HTTP/1.1'). It's easy to get this from iTunes (just forward the request), the real issue is that almost everything that iTunes and the Remote pass to eachother is gzip or zlib compressed. Wireshark seems to be able to decompress most of it, but the response the remote sends to complete pairing seems to be compressed too and I haven't succeeded in decompressing it yet. It should be valid http traffic, actually it also starts with '200 ok', so I know I succeeded in pairing.

The next step is to send some server info (and see what the remote likes and doesn't like) and then figure out the login process. So far I know the remote sends a login request with a paring-guid or something, at least it's a magic number that should identify the remote to iTunes so iTunes can check if it's really paired. Nice for security, but for now I'll guess that part will be happy with another '200 ok'. The remote does however continue after this by passing a session id along when asking for information, so it could be I have to return a session id on login, after which the session id can be used to identify connected remotes.

When there's a real session I can start digging into the harder stuff (at least I'll guess it'll be hard). Some things will probably be quite easy (fetching album-art, fetching volume settings, changing them), getting a playlist will probably be harder...

What also comes to mind right now is that you can use the remote to set speaker volumes for the so-called airtunes speakers. Perhaps it would be nice to abuse this, I can definately understand anyone that wants to create a fake speaker to control whatever slider on his system.

That's it for now, if someone can give me tips on the compression (maybe I should look at wget, curl or whatever code?) I would be very happy, other than that I'm hoping to postpone programming as long as possible so I have more time to think about design.

Edit: Short-term conclusions:
* The DbId/name for the server isn't used in the pairing hash
* The remote port number isn't used in the pairing hash (makes life a bit easier)
* Having the remote paired will make it request /server-info, on teh port specified for the touch-able service, and it stays paired for now... (I guess login can kill pairing)

Thursday, August 7, 2008

iPod Touch iTunes Remote without iTunes?

With the new iPod firmware Apple also released a remote for iTunes as iPhone/iPod Touch application. The application is quite nice, but I won't go into much detail on usability etc right now (if I ever will at all). The thing I wanted to discuss is using the remote application without iTunes. I would love to use the remote for my normal machine or my home theatre pc. The application has support for 'Movies' and 'Television Shows' too, so it would be usable for mythtv. Actually it could be possible to abuse playlists or whatever (ratings for priorities?) to use the application to schedule recordings.

So this is all nice and stuff, but where is the documentation? Well I guess there isn't any. It seems however that Apple is using Daap to control iTunes. A daap server is just a special HTTP server, so the protocol is easy. Also the services (remote and touch-able) are exported using bonjour/zeroconf, so I can fake these services using avahi. So far I managed to create the services so iTunes will think my computer is a iTunes remote. It sends the pincode it uses for connection, but that's how far I've gotten up to now. The pincode is encrypted or hashed, but I guess the hash contains more than just the pincode. I used telnet to send a pincode to the remote application (getting the pincode by having iTunes sending it to netcat), but it just failed with a 404. Perhaps I'm doing something wrong.

I'm going to continue work on this, I already thought of some other stuff to try, so if you're interested leave a comment so I know people want this. Also if you know more than me already, feel free to leave it in a comment too...