As I mentioned yesterday my WordPress blog was hacked. The security hole has been picked up by hacker news and from there, The Register, ZDNet, PCWorld, and Geek.com among others. The publicity will hopefully get Theme developers to update timthumb.php or switch to a different thumbnail generator.
I’ve been contacted with requests for detailed info, so I’m going to post the technical details of how my site was hacked along with the scripts that the hacker used to get in. This is targeted at a technical audience.
The server that served you this web page is the one that was hacked. It runs Ubuntu 10.10 with all security updates installed. It is a virtual server hosted by Linode.
I also run the latest version of WordPress.org.
My WordPress root directory was writable, but making it read only would not have prevented the hack.
Timthumb.php in it’s default configuration allows site visitors to load images from a predefined set of remote websites for resizing and serving. Timthumb offers a caching mechanism so that it doesn’t have to continually re-process images. The cache directory lives under the wordpress root and is accessible by visitors to the website.
The ability for a site visitor to load content from a remote website and to make the web server write that remote content to a web accessible directory is the cause of the vulnerability in timthumb.php.
To be clear, timthumb.php does not actually execute any remote malicious code that causes this vulnerability. This was a point of confusion among some commenters in my blog post yesterday. It simply gets a remote file and places it in a web accessible directory.
Timthumb only allows remote content from a small range of websites to be loaded remotely. In it’s default configuration these included Blogger, WordPress.com and other sites that are writeable by the general public.
Timthumb’s verification that remote content was only being loaded by these domains was also broken. You could for example load content from hackersiteblogspot.com or from blogspot.com.hackersite.com.
I’ve submitted a patch that fixes the pattern matching and removed all default public hosting sites from the allowed sites list. The developer has opted to keep a small list in which I’m not in favor of.
In my case the hacker uploaded a script to my cache directory which timthumb.php stores as “external_<md5 hash>.php”. He/she then accessed this script directly in my timthumb cache directory as something like http://markmaunder.com/wp-content/themes/Memoir/scripts/cache/external_md5hash.php
The script uploaded was Alucar shell which is base64 encoded and decodes when it executes. That makes it a little harder to find it using grep or similar tool. You can see the encoded version of Alucar here and the decoded version of Alucar here (without the username and password preamble at the top).
Here’s a screenshot of the UI:
This script which gives a web based shell access was then used to inject base64 code to one of my core wordpress files wp-blog-header.php which lives in the wordpress root directory. The file with injected code looked like this.
Again, this file is periodically updated with new PHP code, so the attacker could have his way with my server until I found out about it. The code could be altered to instead become a spam system and work it’s way through a long list of spam emails.
The way I tracked this to conclusion was:
- Heard audio on my blog telling me I’d won something.
- Checked Chrome network tools and saw ad content loading and I don’t serve ads.
- Grepped wordpress source and themes for hostname I saw in ad. Nothing.
- Dumped mysql databases on server (all of them) and grepped for hostname. Nothing.
- Confusion reigns.
- Started working my way through nginx (which is my front end proxy to apache) and apache access and error logs.
- Spotted lines in apache error log like this: “[Mon Aug 01 11:09:12 2011] [error] [client 127.0.0.1] PHP Warning: file_get_contents(http://blogger.com.zoha.vn/db/load.php): failed to open stream: HTTP request failed! in /usr/local/markmaunder/wp-content/themes/Memoir/timthumb.php on line 675”
- Checked timthumb’s cache directory and found Alucar.
- Realized base64 encoding is why I didn’t find anything with grep.
- Regrepped wordpress source and database and found injection in wp-blog-header.php
- Decoded base64 stuff and played with Alucar
- Found tmp file in /tmp
- Cleaned everything and fixed permissions. Ran chkrootkit and other utils on machine to see if anything else was compromised. Changed passwords, etc.