Firebug 1.7 moving headers to raw POST source

Edit : The issue discussed here has been fixed in Firebug 1.9a2

It took me a whole day to track this bug down. At first I thought something was wrong in my request, I tried to replay it using Firebug and I kept hitting a "malformed request" error.

Debugging it in reverse order, using Wireshark and other tools I finally came to the conclusion that my request was indeed valid.

So what ?

The guilty part was indeed Firebug. When I ran a simple POST request from Flash, and inspect it from Firebug, I could spot that no Content-Length nor Content-Type headers were displayed in the "Headers" part. I then saw that the "source" part of the POSTtab contains my missing headers. What was that ?

HTTP POST Request are supposed to be split into two parts. First part are the headers, each separated by a \r\n. Then, after and empty line was the POSTraw content. Seeing that my headers gets mixed up with the POSTdata I thought that the request was malformed and a \r\n was added too early.

I checked with Wireshark and with PHP on the back end, and no, the request was indeed absolutly valid. Wireshark did not show any \r\n that shouldn't be there and PHP correctly parsed my request. The only issue was the details displayed by Firebug.

So it's a display bug ? I can live with it.

Well, actually, it's a little more than just a display bug. It kind of blocks my usual debugging workflow.

To debug an XHR request, I don't usually restart the whole page request just to debug one of the inner request.

The Firebug ability to "open request in new tab" is a time saver for me. I can play and replay the same request over and over until it's debuggued.

But with this bug, I can't use this feature. Opening the request in new tab will not send the slipped headers, and will instead send them in the POSTsource, resulting in a corrupted posted data.

I've posted an issue on the Firebug bug tracker. I hope it will get resolved soon. Until then, I'm using Live HTTP Headers to capture and replay the requests I need to debug.

Update

Seems like the bug come from Firefox internals and not Firebug. More details on this post.

Edit : Seems to be resolved in Firefox, will be available in next release (8.0 I guess).

Installing Ubuntu in VirtualBox to work as a development server

I was tired of trying to install Lighttpd on Windows (hint : don't try, it's a mess. I found no way to get vhosts and php as cgi at the same time). So I finally decided that Linux was the way to go.

I have always developped using Windows, and even if I'm slowly migrating to Ubuntu at home, I'm still using Windows 7 at work. The solution that best fits me was still using my Windows environment for coding, but using Ubuntu inside a VM to act as a server.

Let's see how I did that :

Required components

First, let's clarify what I'll have to install for this project : Lighttpd as the webserver, mysql and memcached for storing data and php5 to run cakePHP.

Step One : Running Ubuntu inside a VM

Start by downloading the Ubuntu iso (i's 687Mo, you'd better start this download first). I personnaly used the 10.04 Desktop version (I tried the 10.10 Server version first but I feel more confident with a UI).

Then, download the latest VirtualBox version(4.0.4 at the time of this writing).

And mount the iso in VirtualBox and start the install. This is pretty easy and you should be done in 10mn. Once the install is finished, you'll be prompted to reboot. Do not reboot just now. Instead, shutdown your guest machine (you can type sudo halt if the GUI does not work).

Once back in Windows, go to your VM config and unmount the install iso, otherwise the install procedure will be started again everytime you load your VM.

Once it's done, you can reboot the gues and once loaded, select the in VirtualBox Menu => Devices => Install guest additions. The next popup can take some long minutes to show, so be patient. This will install the guest additions that will allow you to easily communicate between the guest and host machines.

Once installed, reboot once more, and you should have the guest addition image sitting on your desktop.

Start a terminal and install the guest additions by doing :

cd /media/{name of your VBOX image}
./autorun.sh

This will finish the guest addition installation. Reboot your guest machine one last time and you'll be ready for step 2.

Step 2 : Installing the server stuff

We will now install the various components we'll need. You could probably install them all with one command but I'll split them in different line so you'll correctly see what's installed.

First, lighttpd :

sudo apt-get install lighttpd

Now, memcached

sudo apt-get install memcached

And mysql server. This one will prompt you to enter the root password you'll want.

sudo apt-get install mysql-server

Now we will install php5 as CGI (Lighty will run php as cgi), as well as the needed dependency so php can connect to both mysql and memcache.

sudo apt-get install php5-cgi php5-memcache php5-mysql

Ok, you should now have everything correctly installed. We will configure all that stuff later. You should now completely close your Virtual Machine and get back to Windows. It's time to configure your host and guest so they can correctly communicate

Step 3 : Enabling communication between host and guest

What we will do in this section is configure your network so the guest machine is considered part of your network and so you can connect to id using SSH and classic http.

First, go back to your VM config and change the Network mode to "Bridged". This will allow both machine to see each other in your network easily.

Then, define the Shared folder you want shared from your host in your guest. I shared my websites directories so Lighty could access them from the guest.

Now, start your VM, and type sudo ifconfig in a terminal. You should see your guest machine ip. Mine was 192.168.1.16.

Knowing that IP, you can define hosts in your Windows C:\Windows\System32\drivers\etc\hosts

You should also install openssh server so that you'll be able to connect to your guest machine using ssh :

sudo apt-get install openssh-server

We will now take care of the shared folder defined earlier. Selecting them in VirtualBox only made them available for mounting in the guest, but you'll now have to type some more commands to effectively see them. I needed to have access to a project I coded in my host (let's name it myproject).

Here's how I made it available to Lighty :

sudo mkdir /var/www/myproject
sudo mount -t vboxsf -o rw,uid=$(id -u),gid=$(id -g www-data) myproject /var/www/myproject

This will create a directory in /var/www to hold the project, and will then mount the shared folder to that directory. The uid/gid stuff indicate that you (the current user) is the owner and www-data is the group. Doing so will fix access rights issues you may have with Lighty.

You should have to re-run the last command on every login, so I strongly suggest you to put it in /etc/rc.local so it gets executed automatically.

Step 4 : Configuring all that stuff

I won't go into much details on how to configure lighty or php because it is not the scope of this post. I might post on that subject later, though. But here is one important step you shouldn't forget : uncomment the cgi.fix_pathinfo=1 line in your php.ini.

Don't forget to reload Lighty after changes to its config files by running:

sudo /etc/init.d/lightppd restart

I also had one additional side-effect on my install : installing php5 also installed Apache, causing Lighty to fail on startup because Apache was already using the port 80. I fixed it by removing every reference to Apache by running :

sudo update-rc.d -f apache2 remove

Final note :

If you followed this instruction, you should have a server running inside a VM that runs your website stored in your host. Isn't that pretty ?

Making Flash and Javascript speaks to each other

I needed a Flash app and my Javascript code to be able to communicate. The Flash needed to call some methods in the Javascript, and in turn the Javascript needed to inject back some data into the Flash.

I let my Flash coder colleague took care of the Flash side, he needed to implement ExternalInterface, register some callbacks methods that I could call and more importantly change some security settings to make it work.

On my side, I just needed to get the objectelement holding the Flash (thanks to swfobject.getObjectById it was pretty easy) and call the callback method registered in Flash on it.

Piece of cake.

Have a look at this blog post from CircleCube, it does a pretty neat job at explaining all of that.

Update :

I finally had some issues with this implementation. It was working perfectly on some servers but not on others. We finally found that calling Security.allowDomain('*.mainhost.com') wasn't working.

You have to explicitly allow each domain/subdomain.

As our code would be deployed on various duplicate domains, we had to manually pass as a flash var the domain to allow.

Migrating from Windows to Ubuntu : KeePass

I'm slowly trying to move my development machine from a desktop Windows Xp to a laptop running Ubuntu. This is quite a long task because there are so much softwares I depend on on my day to day work. Most of them are crossplatform but I often need o find a Ubuntu equivalent, and sometimes it does not exists and I have to run the Windows app using an emulator.

This is exactly the case with KeePass. KeePass 2.0 is an improved version of KeePass 1.0 (obviously), that allows one to save all password in one place. The 2.0 version also permit to save more than single password, virtually anything, and adding little icons to each entry (when you start having 200+ entries, icons are a must have).

Enter Ubuntu world

The official 2.0 version is not supported under linux. There is a linux port, named KeePassX, but it does not support the .kdbx file format used by the official 2.0 version.

So I tried to run the Windows .exe file with Wine at first (I already had Wine installed to run another app), but it wasn't working with KeePass. I tried instead an other emulator, Mono, and running mono KeePass.exe totally worked.

Little tweaks ahead

First of all, I had to install two additionnal packages to make it open without errors : System.Windows.Forms and System.Runtime. So far, so good.

But, I quickly discovered that one of the most usefull features of KeePass, auto-type (Ctrl+V automatically fills a login/password field) wasn't working correctly. There was one more missing dependency : xdotool.

If you are running Ubuntu like me, do not install it from the official packages, this version is completly outdated and won't work. Instead, you have to manually install the latest version. If you are lucky and running a 64bits system, you can download directly the .deb file from here.

If you are running under a 32bits system like me, you'll have to manually install it. First grab the files from this link. Then, install the libxtst-dev package (it is needed for the install to complete), export the content of the .tar.gz file into a temporary directory and execute sudo make install.

Finally !

Now, you got KeePass 2.0 working under Ubuntu, and installed all dependencies needed to run the auto-type feature.

In a Windows environment, one can change KeePass preferences so that closing it only send it to the tray bar.This option does not seem to be working under Linux, KeePass still shows in the task bar even when minimized. I spent some time trying to fix that too, tetsing AllTray and other tray softs for linux but didn't managed to get what I wanted.

Differences between Mercurial and Subversion

People at my current job are using Subversion, and our project will be tracked using it. I never used Subversion before, the only versionning system I ever used was Mercurial. And moving from one to the other meant changing a lot of reflexs I had.

Here is a little list of changes, mostly as a reminder for myself.

Directories

Mercurial uses a single .hg directory at the root of your project to store all your project history while Subversion adds multiple .svn directories in each directory, to track history changes to that directory only.

I prefer the Mercurial approach, it keeps all changes centralized in one place. You can simply remove the .hg directory to transform your versionned version into a stand-alone one.

While the Subversion approach litters your app with countless hidden directories, making copy and pasting a real pain.

Centralised vs Distributed

Subversion is centralised while Mercurial is distributed.

As far as I understand the difference, it means that Subversion uses one main directory to store the versionned version and can deploy (export) a specific revision at anytime.

That revision do not hold any history information, it is only a copy of your project at a given time.

On the other hand, each Mercurial repository holds both the current public version and all the history. You do not have to deploy anything anywhere, just update your current repo with data from one of the revision.

Tortoise

I am using both TortoiseSVN and TortoiseHg. When you commit with TortoiseSVN it displays the list of files that where updated since the last commit. If you added new files, they won't show unless the "Show unversionned files" is checked.

In TortoiseHg, all new files are automatically seen in the commit window, as well as a diff. It allows me the easily see what changes where made, and help me write my commit message.

I really like the TortoiseHg vision better.

Tracking directories

Subversion can track empty directory, just by adding them. Mercurial can't. You have to add an empty file in each to allow tracking.

Also, when doing a commit in a specific directory in Mercurial, it will commit the whole repo, while with Subversion it will only commit the current repo. I can see the benefits of both and I'm not quite sure which is better.

Subversion will allow me to do a commit of one special feature by committing only one directory, but the TortoiseHg integration help me doing commits more easily no matter where I'm browsing.

Ignoring files

I have some files in my project I don't want to track (like auto-generated cache files). In Mercurial, all I need is editing the .hgignore file with regexps. The syntax can be a little strange sometimes, it took me a while to correcly understand it, but it definitely works.

On Subversion, I can add files to the ignore list so they don't show as "unversionned", but I can also add a svn:ignoreproperty to a specific directory to set regexp to files that I don't want to track. The end result is the same, but the way Mercurial handle it with one file is more appealing to me.

Conclusion

Having all history in one dir and all ignore rules in one files seems a better approach to me. I guess on some aspects Mercurial is still more centralised than Subversion.