How to Reduce the size of a Kindle Book

Leave a reply

Recently I published an image heavy Kindle title and found that it had quite a large file size. The culprit of course was the number of images. Kindle books only support either the .JPG or .GIF image format, so using highly optimized .PNGs was out of the question.

As I was using Jutoh to write my book, I didn’t have access to the image files used by Jutoh once they had been imported. It would have taken days to reinsert every image file after optimizing them. The quickest way to optimize all images that I found was to compile the book to a .mobi file, use KindleUnpack to unpack the .mobi file, open the zip file containing the source files, optimize all the .gif images using Trout Image Optimizer and then finally open the content.opf file in KindlePreviewer which automatically compiles a new .mobi file with the optimized images.

Mod_Evasive Blocks Googlebot

Leave a reply

I discovered an issue with mod_evasive and Googlebot recently. I noticed that after installing mod_evasive the number of impressions in Google Webmaster Tools had been reduced. Additionally, some of my page results in Google where displaying “A description for this result is not available because of this site’s robots.txt – learn more.”

In /var/log/mod_evasive are the logs of all the IPs that have been banned at some point. I found that some of these IPs were from the Googlebot, and mod_evasive blocking Googlebot was likely the cause of the reduced impressions. I’ve now increased the default setting of mod_evasive which are

DOSHashTableSize 3097
DOSPageCount 2
DOSSiteCount 50
DOSPageInterval 1
DOSSiteInterval 1
DOSBlockingPeriod 10

and have increased them to the following in order to relax the DOS trigger

DOSHashTableSize 3097
DOSPageCount 10
DOSSiteCount 100
DOSPageInterval 1
DOSSiteInterval 1
DOSBlockingPeriod 10

So far Googlebot has not been blocked again.

In mod_evasive it is possible to whitelist IPs and whitelisting Google IPs might be an option. The problem is that it seems that Google IPs are constantly changing. The IP ranges that some other blog posts had recommended whitelisting were not even close to the Google IPs I was getting. So i’ve opted for more relaxed DOS constraints.

How to Migrate/Setup a WordPress site on Linode

Leave a reply

Recently I decided to move away from my Hostgator shared hosting and set up camp on a new Linode VPS. I was able to do this despite having no server administration experience and just basic to moderate Linux terminal skills. This post is about how I did it.

This guide assumes you are running Windows on your local machine. So you will first need to install PuttY and WinSCP. Putty is a SSH Terminal and WinSCP will allow you to transfer files graphically.

Install and Secure

#1. Deploy a Ubuntu 14.04 Linux Distribution. To do this click on Deploy a Linux Distribution in your Linode configuration page. Set the distributions to Ubuntu 14.04 LTS, deployment disk size to the maximum size indicated below the entry box and leave the swap disk size at the default setting of 256 MB. Enter the password you want to use for your root Linux user.

#2. In your Linode dashboard watch the host job queue until your Linode finishes being created.

#3. Click the Boot button to boot up your new Ubuntu 14.04 Linode.

#4. Once booted go into the Remote Access tab and write down your Linode’s IP address.

#5. Open Putty. Choose SSH and type in the IP address on your Linode in the Host Name box. Set the Port to 22 if it is not already. Then press Open.

#6. You will now see a PuTTY Security Alert box. This is just alerting you to the fact that your computer has not previously connected to the Linode server. Click Yes.

#7. Now your terminal window will pop up and you will be able to log in. Log in as user: root and your password is the one you set when deploying your Ubuntu 14.04 Linode.

#8. Once logged into the system we should first update Linux. Type in

apt-get update
apt-get upgrade --show-upgraded

#9. Next we need to set the servers hostname. This is just a random name that identifies the server. In the Linode tutorial they use the name “plato”. Type in the following.

echo "plato" > /etc/hostname
hostname -F /etc/hostname

#10. Now we need to write to our hosts file to set our hostnames IP address. NOT DONE.

#11. Set your desired timezone by typing in

dpkg-reconfigure tzdata

#12. Next we want to add a new user as we do not want to log into our Linode as root. The root user has the power to do anything on the system and so it is easy to make a mistake like deleting important files. Create a new user with

adduser example_user

#13. Now we need to give the new user admin rights and add them to the www-data group which apache and PHP run as. Use the following.

usermod -a -G sudo example_user
usermod -a -G www-data example_user

#14. Now close and reopen your PuTTY session and log in as your new user.

#15. Next we want to secure our server by disabling the root login (most hack bots will try to log in using the root user). We also want to change the SSH port from the default of 22 to something else as most hack bots will only try to break in through port 22. Type in

sudo nano /etc/ssh/sshd_config

You will be asked to type in your user password now as you are not logged in as the root user.

Once in the file change the following as such. (To easily search for text in the nano text editor press CTRL+W and then type in what you want to search for, then press enter.)

PermitRootLogin no
Port 3333

#16. Next we will want to set up a firewall. A firewall will block access to all ports apart from the ones that we allow. Open the firewall rules by typing in

sudo nano /etc/iptables.firewall.rules

and then copy as paste in the following default firewall. (If you chose a different SSH port be sure to change it in the firewall line below)

*filter

# Allow all loopback (lo0) traffic and drop all traffic to 127/8 that doesn't use lo0
-A INPUT -i lo -j ACCEPT
-A INPUT -d 127.0.0.0/8 -j REJECT

# Accept all established inbound connections
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

# Allow all outbound traffic - you can modify this to only allow certain traffic
-A OUTPUT -j ACCEPT

# Allow HTTP and HTTPS connections from anywhere (the normal ports for websites and SSL).
-A INPUT -p tcp --dport 80 -j ACCEPT
-A INPUT -p tcp --dport 443 -j ACCEPT

# Allow SSH connections
#
# The -dport number should be the same port number you set in sshd_config
#
-A INPUT -p tcp -m state --state NEW --dport 3333 -j ACCEPT

# Allow ping
-A INPUT -p icmp -j ACCEPT

# Log iptables denied calls
-A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables denied: " --log-level 7

# Drop all other inbound - default deny unless explicitly allowed policy
-A INPUT -j DROP
-A FORWARD -j DROP

COMMIT

#17. Now we need to activate the firewall with the following

sudo iptables-restore < /etc/iptables.firewall.rules

#18. The firewall is now set up, but if you restart the server the rules will be lost. To ensure the rules are loaded on boot open the following file

sudo nano /etc/network/if-pre-up.d/firewall

and then copy and paste in this script

#!/bin/sh
/sbin/iptables-restore < /etc/iptables.firewall.rules

#19. Next close and save the file with CTRL+X and give the script permission to execute with the following.

sudo chmod +x /etc/network/if-pre-up.d/firewall

#20. Now we will want to set up fail2ban. This is a program which monitors logfiles and will stop any attempts by a hacker to brute force their way into your server by guessing login details very fast. Essentially, if an attacker fails to login X times in X seconds, fail2ban will ban the attackers IP for a set amount of time. To install fail2ban type in the following.

sudo apt-get install fail2ban

#21. Next to setup fail2ban by first copying the configuration file to a local file which will be used. This is so that your configurations survive updates. Then open the fail2ban configuration.

sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
sudo nano /etc/fail2ban/jail.local

#22. Under [ssh] and [ssh-ddoss] jails, set the ports to be whatever port you have chosen to use for SSH as configured in the previous steps.

#23. Close the file with CTRL+X and then restart fail2ban with the following.

sudo service fail2ban restart

I don’t recommend using fail2ban to block anything other than SSH attacks. This is because many hacker bots will used spoof IPs. In my case the bots where using Google IPs causing SEO rankings to drop.

Install Apache

#1. Next install the Apache webserver with the following.

sudo apt-get install apache2

#2. Backup the default Apache configuration just in case you mess something up.

sudo cp /etc/apache2/apache2.conf /etc/apache2/apache2.backup.conf

#3. Now edit the configuration file with the following.

sudo nano /etc/apache2/apache2.conf

#4. Add in this code at the bottom in order to optimize it for low memory usage. If you have a Linode with more than 1GB of RAM, you may wish to adjust these values for better performance. Here we assume that each Apache process uses about 30 MB of RAM and that we want to give at most 750 MB to Apache. So we set MaxClients to 25 x 30 = 750 MB.

<IfModule mpm_prefork_module>
StartServers 2
MinSpareServers 6
MaxSpareServers 12
MaxClients 25
MaxRequestsPerChild 3000
</IfModule>

#5. Under  <Directory /var/www/>, change AllowOverride None to AllowOverride All as shown below. This will enable .htaccess files to be used which WordPress requires. This must be done otherwise you will get errors like ‘You are not authorized to view this page.’

<Directory /var/www/>
Options Indexes FollowSymLinks
AllowOverride All
Require all granted
</Directory>

Install MySQL

#1. Now let’s install MySQL to manage our databases. You will be asked to setup a root password. Go ahead and do this, choosing a strong password.

sudo apt-get install mysql-server

#2. Secure the MySQL setup by using the line below. Be sure to answer as N, Y, Y, Y, Y.

sudo mysql_secure_installation

#3. Now we will optimize the MySQL database for low RAM. If you have a Linode with more than 1 GB of RAM you might want to increase some of these values or use MySQL Tuner to determine optimal values. Open the file in the following location.

sudo nano /etc/mysql/my.cnf

#4. Change the following key as so.

max_connections = 75
key_buffer = 32M
max_allowed_packet = 1M
thread_stack = 128K
table_cache = 32

#5. Restart MySQL.

sudo service mysql restart

#6. Set MySQL to optimize the databases daily or weekly using a cron job. First open up the cronjob editor with the following. (Choose nano if it asks)

sudo crontab -e

#7. Now copy and paste in the following line making sure to change PASS to your own MySQL root user password.

@daily mysqlcheck -o --user=root --password='PASS' -A

Install PHP

#1. Install PHP with the following.

sudo apt-get install python-software-properties
sudo apt-get install php5
sudo apt-get install php5-mysql

#2. Optimize PHP for low memory performance by opening the following file.

sudo nano /etc/php5/apache2/php.ini

#3. Now make changes as follows. (Most of these will already be set, but you may need to add in some missing lines)

max_execution_time = 30
memory_limit = 128M
error_reporting = E_COMPILE_ERROR|E_RECOVERABLE_ERROR|E_ERROR|E_CORE_ERROR
display_errors = Off
log_errors = On
error_log = /var/log/php/error.log
register_globals = Off

#4. Create the PHP log directory and give it the correct owner ship of www-data so that PHP can write to the folder.

sudo mkdir -p /var/log/php
sudo chown www-data /var/log/php

#5. Restart Apache.

sudo service apache2 restart

Create a Virtual Host

A virtual host is what will serve your website. You can run several virtual hosts on one server, meaning that you can host multiple websites on the same server.

#1. Go to to the following directory.

cd /var/www

#2. Create the following directories making sure to change example.com to your own domain.

sudo mkdir -p example.com/{public,log,backup}

#2. Create a file as below, making sure to replace example.com with your own domain name.

sudo nano /etc/apache2/sites-available/example.com.conf

#3. Copy and paste this into the file, changing example.com to your own domain and your email address too.

# domain: example.com
# public: /var/www/example.com/

<VirtualHost *:80>
# Admin email, Server Name (domain name), and any aliases
ServerAdmin example@example.com
ServerName www.example.com
ServerAlias example.com

# Index file and Document Root (where the public files are located)
DirectoryIndex index.html index.php
DocumentRoot /var/www/example.com/public

# Log file locations
LogLevel warn
ErrorLog /var/www/example.com/log/error.log
CustomLog /var/www/example.com/log/access.log combined
</VirtualHost>

#4. Now enable your site.

sudo a2ensite example.com.conf

#5. Restart Apache

sudo service apache2 restart

Test Your Site

Here we will create a simple test site to check that everything is running properly.

#1. In /var/www/example.com/public create a document index.php

sudo nano /var/www/example.com/public/index.php

#2. Copy and paste the following into it.

<?php
echo 'Hello, PHP!';
?>

#3. In your local machine hosts file (Windows: %systemroot%\system32\drivers\etc\) add a line as follows where XXX.XXX.XXX.XXX is the server IP and example.com is your domain. This will allow you to visit your website even though you have not yet set up the DNS servers.

XXX.XXX.XXX.XXX www.example.com

#3. Save your hosts file and then load your domain in a webbrowser. You should see the text “Hello, PHP!”

Migrate Your WordPress site to Linode

Now that our webserver is set up, we can begin migrating our WordPress site.

#1. Log on to your old webserver either through terminal or cPanel and make a backup of your WordPress database. In cPanel this can be done under the Backups icon and in terminal this can be done with the following command from the home directory.

mysqldump -u user -p  db_name > backup.sql

#2. Open WinSCP and log on to your old server. Find your WordPress installation folder (probably under public or public_html somewhere). Select all the files and choose custom commands Tar/GZip to compress the entire folder contents into a single example.tar.gz file.

#3. Using the WinSCP download button, download both the backup.sql database and your compressed WordPress installation folder to your local disk.

#4. Open a second WinSCP instance and log in to your new Linode server (remember to use the SSH port you specified in the security set up section).

#5. Copy the database and compressed WordPress install folder to the Linode server using WinSCP. Place the .tar.gz file into the /var/www/example.com/public folder and the .sql file anywhere convenient such as the home folder.

#6. Extract the contents of the WordPress install with tar -zxvf example.tar.gz.

#7. Set the permissions of your /var/www folder to be owned by you the user and to have group www-data.

sudo chown -R "$USER":www-data /var/www

#8. Set all files to have permissions of 0644.

sudo find /var/www/example.com -type f -exec chmod 0644 {} \;

#9. Set all folder to have permissions of 0755.

sudo find /var/www/example.com -type d -exec chmod 0755 {} \;

#10. If you were using WP-Supercache change the WP-Content folder to have permissions 0775.

sudo find /var/www/example.com/public/wp-content/ -type d -exec chmod 0775 {} \;

#10. If you were using WP-Supercache set the permissions of /var/www/example.com/wp-content/wp-cache-config.php to 0666.

sudo chmd 0666 /var/www/example.com/wp-content/wp-cache-config.php

#11. If you were using WP-Supercache set the .htaccess permissions to 0664.

sudo chmod 0664 .htaccess

#12. Enable Apache rewrite, expires and headers functionality.

sudo a2enmod rewrite
sudo a2enmod expires
sudo a2enmod headers

#13. Restart Apache.

sudo service apache2 restart

#14. Create a new database in MySQL as follows. I recommend you create a separate user for each database you create. (Don’t forget the semi-colons)

mysql -u root -p
create user 'NEW_DBUSER_USERNAME'@'localhost' identified by 'NEW_DBUSER_PASSWORD';
create database new_database;
grant all privileges on new_database.* to ' NEW_DBUSER_USERNAME'@'localhost' identified by 'NEW_DBUSER_PASSWORD';
FLUSH PRIVILEGES;

#15. Go to the directory in which you stored the backup.sql file and then type the following to copy the old database  to your new one.

sudo mysql -u NEW_DBUSER_USERNAME -NEW_DBUSER_PASSWORD' new_database < backup.sql

#16. Now if you named your new WordPress database something different to what it was previously or changed the password open up your wp-config.php file.

sudo nano /var/www/example.com/public/wp-config.php

#17. Change DB_NAME, DB_USER and DB_PASSWORD to your new values.

define('DB_NAME', 'new_database');
define('DB_USER', 'NEW_DBUSER_USERNAME ');
define('DB_PASSWORD', 'NEW_DBUSER_PASSWORD');

#18. If you were previously using WP-Supercache also change the WPCACHEHOME defined variable to the new location of the WP-Supercache plugin folder.

define( 'WPCACHEHOME', '/var/www/example.com/public/wp-content/plugins/wp-super-cache/' );

#19. Now if you visit your website it should all be up and running!!!

Setting up a Mail Server

Although the site is now up and running there is still some work to do. In order to allow WordPress to send out email when a comment is received or for any other reason we must first set up a mail server. The Linode tutorial on mail servers is pretty much all your need for this.

Setting up FTP

When you try to add new or delete plugins on your newly migrated or installed WordPress site you might get errors asking your for your FTP credentials. There are ways around this like granting WordPress the ability to directly write to the file system or by changing permissions to allow WordPress to write to directories but this may not be security safe. Instead it is best to install a FTP server and allow WordPress to work as it was designed.

#1. Install VSFTPD with the following.

sudo apt-get install vsftpd

#2. Edit the config file /etc/vsftpd.conf and set write_enable=YES.

#3. Edit your wp-config.php file and add in the following lines. Where LINUX_USER is the name of the user you are currently logged in as and LINUX_USER_PW is your Linux password.

define( 'FTP_USER', 'LINUX_USER' );
define( 'FTP_PASS', 'LINUX_USER_PW' );
define( 'FTP_HOST', '127.0.0.1' );

What’s Left to do?

At this point everything should be set up and running fine. But now you may wish to do the following.

#1. Install mod_evasive to help block DOS attacks.

#2. Secure your wp-login page.

#3. Optimize your WordPress install.

#4. Run MySQL tuner to tune your database.

#5. Migrate or create new sites following the same process as above from “Create Virtual Host” onwards.

Stopping WP-LOGIN Attacks, Hacks and Floods

Leave a reply

One of my blogs gets attacked regularly by hackers trying to break into the WordPress admin area by spamming various passwords on the wp-login page (the page you use to login to your WordPress admin area). This increases CPU usage on my server from an normal average of 5% to about 40%, effectively slowing the site down. There are various ways to stop these flood attacks and in this post I show some methods that I used. But first, although this post is about stopping server floods for performance reasons I should mention that if you are using the ‘admin’ user name you should stop right now and change that username to something else. Almost all the bots operating this kind of attack try to brute force their way in using the ‘admin’ username.

HTTP Auth Login

The best method I have found to stock attacks is a to add a second login screen to your WordPress with an HTTP Auth login. An HTTP Auth login will ask you to input a username and password before you can even access the wp-login page. Most wp-login attack bots won’t be programmed to try and crack HTTP Auth logins and most spammers will simply toss your site away instead of trying to waste time brute forcing two login screens. Additionally, if a spammer were to try and brute force the HTTP Auth login anyway, the server would not see a large increase in CPU usage. This is because HTTP Auth is extremely efficient and uses very little resources since it does not need to access the WordPress database. As long as you use a unique username and strong password brute force methods on HTTP Auth should be secure. And if they do get in they still have to get past the standard WP-Login screen. If you’re extra concerned about security you could set up another two factor authentication of the wp-login screen using Google Authenticator or other methods.

To create an HTTP auth login you will need to have access to your .htaccess file and the home folder on your server. First create a .htpasswd file by going to http://www.htaccesstools.com/htpasswd-generator/ and entering in your desired username and password. Upload the generated file to your home directory or wherever you want to put it. Make sure it is not accessible from the web. Or, instead you could install apache2-utils (sudo apt-get install apache2-utils) and then use htpasswd -c /home/user/.htpasswd HTTP_AUTH_USERNAME where you should replace HTTP_AUTH_USERNAME with your own choice of login username. You’ll then be asked to enter your pick of password twice, then the .htpasswd file will be generated.

Next in your .htaccess file copy and paste the following, being sure to change AuthUserFile to the actual location of your .htpasswd file.

# Protect wp-login
<Files wp-login.php>
AuthName "Authorized Only"
AuthType Basic
AuthUserFile /home/USER/.htpasswd
require valid-user
</Files>

Note: You must use the FULL path to the .htpasswd file. I found that even using ~/.htpasswd would not work even though they both pointed to the same location.

Also note that if you have multiple WordPress users or if you allow registrations on your site then this method may not be for you if you don’t want to annoy your users with two login screens. A simple fix for this might be to change the AuthName string to some user instructions like “ANTI-BOT MEASURES: Use the username ‘access’ and password ‘accesspwd’ to access the login screen”, making sure to set the .htpasswd username and password accordingly. It is unlikely that a human spammer would even bother to look at your site and discover the HTTP Auth login credentials.

Look for No-Referrer Requests

When a spam bot tried to access your wp-login page it will usually try and access the file directly, without first going to your site. By using some simple .htaccess code we can detect if the bot was reffered by your URL or not. And if not, block access to the bot. Simply add this code to your .htaccess file where you should replace MY_DOMAIN.com with your domain. This code will also protect the comments submissions form from some spammers.

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{REQUEST_METHOD} POST
RewriteCond %{REQUEST_URI} .(wp-comments-post|wp-login)\.php*
RewriteCond %{HTTP_REFERER} !.*MY_DOMAIN.com.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^$
RewriteRule (.*) http://%{REMOTE_ADDR}/$ [R=301,L]
</IfModule>

Unfortunately, modern advanced bots will get around this easily, but there are still some using older methods looking for unsecured sites.

Block Logins from all IP’s other than your own

If you have a static IP address and only access your WordPress site from fixed locations the best method at stopping login attacks is to block access to the wp-login page from ALL IP’s other than your own. This can be easily implemented by adding the following code to your .htaccess file.

<FilesMatch wp-login.php>
Order deny, allow
allow from XXX.XXX.XXX.XXX (replace XXX with YOUR OWN IP address)
allow from XXX.XXX.XXX.XXX (can use multiple IP addresses)
deny from all
</FilesMatch>

I didn’t end up using this method as I don’t have a static IP and I access like to access my blog while travelling.

WP-Security Plugin

Another method i’ve tried adding to my blog is the WP Security-Protection plugin. This plugin works by detecting the presence of Javascript. Most bots will not have Javascript enabled and will be detected as a bot. Then the plugin sends a cookie that tricks the bot into believing that it has gained entry. In most cases this will stop the bot from continuing. However, I decided not to use this plugin as with HTTP Auth the bot should never even get to the login screen.

IP Blocking with Fail2Ban

This method uses Fail2Ban (an IP blocking server tool) to check access logs for wp-login.php floods. The best tutorial I found on this method was here. However, I ended up not using this method because many of the spammers IPs where spoofed and were pointing to Google servers. Not wanting to block Google IPs and probably get a SEO problem I ditched this method in favor of the HTTP Auth and no-referrer request methods. Another danger is that many of these bots will be coming from hacked PCs around the world. Blocking one of these IPs could be potentially blocking a customer or even an entire company as some companies will use one external IP address for all their computers.

The ONLY Way to Stop WordPress Comment Spam

Leave a reply

Like most WordPress sites my popular ones get hit by 100’s of spam messages a day. I’ve only found one method that works well against spam and that is to use a custom question field. A custom question field asks the poster to answer a simple question correctly when submitting the post. This works because the website owner chooses the question, so a bot will have no idea what to do with it. The only way the bot can get around the question is for a human to come in and program the bot with the answer. Most spammers will not bother to do this.

I recommend choosing a question that is related to the content of your site to make it easy for your readers. For example in this blog I might choose a question like “What is the aquatic animal in the name of this blog?”. Make sure to set multiple answers to the question like “fish”, “Fish” and “FISH”. Also don’t choose a question like “What is 7 + 3?” as some bots might be able to figure this out. I use the plugin WP No-Bot Question to implement this service.

Previously I had tried other antispam services and i’ll list them below and say why they didn’t truly work.

#1. Askismet

Askismet is a service that uses a computer to read the contents of the comment and from that it will attempt to detect if the comment is a spam message. Askismet works fairly well, BUT the major problem I had with Askismet was that it was constantly detecting legitimate comments as spam. This was because some of my commenting users did not have perfect English or grammar – characteristics of spam messages. It got so bad that it was flagging about 75% of legitimate messages as spam. And with such a huge spam folder I had no chance of finding these real comments.

#2. Image Captchas.

Image captchas work by asking the user to type in some text or numbers displayed in an image. The problem with this is that most spam bots now have the capability to actually read these images and translate them into text. It is possible to make the captchas harder to read, but then they become harder to read for humans as well. Captchas have been broken for a few years now.

#3. Cookies/Javascript Detectors.

These plugins detect spam through the presence of a specially placed cookie and/or check if Javascript is enabled. As most bots don’t have cookies or Javascript enabled they will be caught. The problem is that spam bot programmers are catching on to this and are enabling their bots with cookies and Javascript. I must say though that these plugins still do reduce the amount of spam you receive and in my case reduced spam by about 50%. You might be best suited to running one of these types of plugins in conjuction with a custom question field.

#4. IP Blocking Plugins.

These plugins work by detecting spam in some way through the above methods (or through a global database of known spam IPs) and then blocking the IP address of the spammer. A blocked IP means that the spammer will be unable to access your site at all. The problem with this is that most bots these days spoof their IP address (use fake IPs). In my case the bots were using the IPs of Google services causing my site to be blocked from Google. Not good for SEO at all. Even if they do not spoof their IP, spammers tend to have many IPs at their disposal. Another danger is that sometimes one IP address could belong to many PC’s. For example a university or company might have one IP with many PCs behind it. Blocking an IP could unintentionally block a large amount of legimate users.

So in the end the only method that really worked was the custom question solution. I must note though that custom questions will only work against bots. If you have a human spammer targeting your site it will not work.

How to Seriously Speed Up and Optimize WordPress

Leave a reply

A few months ago one of my WordPress sites began to get popular with almost 6,000 unique visitors a day. As the site grew, I also noticed that it was getting very slow to load. This post is about how I managed to speed up and optimize my WordPress site to make it blazing fast.

#1. Move Away from Shared Hosting

If you are paying under $10 for hosting a month you are likely on a shared web hosting server. On a shared server there are multiple users running websites alongside yours. If another website served from the same server as yours is popular it will use up a lot of CPU time, memory and bandwidth starving your site of needed resources. Good hosting companies will run their shared server efficiently, making sure that there are not too many sites per server and that the loads are balanced, but there is still no guarantee of minimum performance.

Once your site begins to get popular I recommend moving to a Virtual Private Server (VPS). This is kind of similar to shared hosting in that there are multiple users running sites on the same server. However, the major difference is that each user gets their own dedicated slice of the server resources to do with what they please. This gives a guarantee of minimum performance.

I had previously hosted my sites on Hostgator shared hosting. Once my site got popular I moved my site to a Linode VPS server. This one change made a huge difference in performance. My average site load times we from around 5 seconds down to only 2 seconds. It’s important to note that Linode is an ‘unmanaged’ host. This means that you will need web hosting and Linux experience to set up your sites. Although Linode provides many tutorials, if you are a novice I strongly recommend using managed hosting. One managed host that has the best reviews out there is KnownHost and I recommend considering them for your VPS services.

If you are an absolute complete novice to web hosting and WordPress then I highly recommend you check out WP Engine for your hosting. They are a managed host that specialize in WordPress sites. If you switch to WP Engine for your hosting you can probably ignore most of the tips shown in this post as their staff and infrastructure will take care of all the optimizations for you!

#2. Use WP Supercache

This is an obvious one. If you don’t already have some sort of caching method plugin like WP Supercache installed you should install one now. WordPress sites are dynamically generated. That means that a page’s code is generated from scratch each time it is loaded. This is slow as the server CPU must first spend time generating the page code. A caching plugin saves the generated page code and serves the pre-generated page directly.

Installing WP Supercache is easy, just find it in the WordPress Add New plugins search. Then once downloaded and activated enable caching under the easy tab in the WP Supercache settings page. Now your pages are cached! But to improve caching speed I recommend going into the advanced tab and changing the caching method to mod_rewrite then scrolling down an clicking on the Update Status button. After this you’ll need to scroll down again in the Advanced tab page and click the Update Mod_Rewrite Rules button.

I also recommend using the “Compress pages so they’re served more quickly to visitors” option and also the “Don’t cache pages for known users”. Not caching for known users will ensure that any changes you make to the site that may be cached will show up straight away for you.

#3. Use a CDN

A Content Delivery Network (CDN) will dramatically improve your page load speeds. A CDN company works by having multiple server locations around the world. Instead of a user requesting a page from your server across the world, a CDN will step in and cache elements of the page at a server that is closer to the user. This can help distribute and reduce the load on your server whilst at the same time making your site load faster all around the world.

I suggest that people use the well trusted MaxCDN service. The lowest tier of MaxCDN services costs $9 and includes 100GB of data transfer which should be plenty for any moderately successful site. With a CDN like MaxCDN you will be able to see vastly improved page load times.

There is also Cloudflare which is another CDN provider. As well as paid services Cloudflare provides a free option with reduced performance. You will need to do careful speed testing with the free option however as some users have reported that the free Cloudflare can actually slow down some websites.

#4. Optimize MySQL Databases

Most webservers won’t be set up to automatically optimize your WordPress databases. Optimizing a WordPress database is kind of like dragmenting your PC hard drive. After it is optimized accessing the database will be much much faster. A plugin called WP Optimize can be installed that will automatically optimize your WordPress database. WP Optimize will also remove old post revisions and drafts that eat up database space and increase access times. Please ensure that you first back up your databases before optimizing as in some very rare instances optimization could break a database. I recommend using the Updrafts Plus backups plugin for this purpose.

If you don’t want to use a plugin and you have PHPMyAdmin installed on your server (usually available in most managed hosts and in cPanel through Databases >> phpMyAdmin) you can simply use the dropdown Optimize menu to optimize your databases.

My WordPress database had not been optimized for a year and after optimization its file size halved and there was a noticeable performance boost.

#5. Ensure your site is Compressed

Just like zip files, websites can be compressed before they are sent. This can in many cases significantly reduce the amount of data that needs to be transferred over the internet therefore speeding up page load times.

To check if your site is compressed you can use the online tool at checkgzipcompression.com. If it isn’t already you can enable gzip compression through cPanel if you have it. In cPanel go to Software/Services >> Optimize Website and make sure that the “Compress all content” radio button is selected. The WP Super Cache plugin also has a setting which can be used to serve compressed files.

#6. Use EWWW Image Optimizer

If you have an image heavy site it will pay to use an image optimizer like EWWW. An image optimizer will compress your images optimally to save space and thus improve page load speeds. I recommend enabling Lossy PNG optimization as this will thoroughly optimize PNG images usually without any noticeable loss by removing unused colors. If you run your site on a private server and have a decent CPU setup I recommend also enabling pngout and setting it’s optimization level to extreme.

To optimize any images uploaded before you install EWWW go to Media >> Bulk Optimize and use the options there to optimize old files. If you are on a shared server be careful that you do not set the optimization settings to the maximums as it could use up a lot of shared CPU if you were to try and bulk optimize all your images. On shared hosting using too much CPU might get your account temporarily restricted. For shared hosting, the default settings work well.

#7. Remove or Reduce Share and Follow Buttons

Having too many share and follow buttons on your blog can have a very noticeable affect on page load times. This is because each share button loads code from external servers. Some ways to get around slow share buttons are to load them asynchronously using the Asynchronous Share Buttons plugin. Normally share buttons will load sequentially, that is one after another. With asynchronous buttons they can all load simultaneously.

For follow buttons in the side bar I recommend simply uploading your own images and directly linking your social media pages to the images. Here is some example HTML code. I simply uploaded the sharing icons to my blog and then copy and pasted the link to them in the src=”” section of the code. Place this code into a sidebar text widget and remember to change the images to your own hosted images and the parts of the code that say YOUR_PAGE to your own page.

<h3 class=title>Follow Us</h3><div><a target=_blank rel=nofollow title="Follow us on Facebook" href="http://www.facebook.com/YOUR_PAGE" style="font-size: 0px; width:48px;height:48px;margin:0;margin-bottom:5px;margin-right:5px;"><img alt=facebook title="Follow us on Facebook" width=48 height=48 style="display: inline; width:48px;height:48px; margin: 0; padding: 0; border: none; box-shadow: none;" src="http://www.rtl-sdr.com/wp-content/uploads/2014/07/facebook.png"/></a><a target=_blank rel=nofollow title="Follow us on Twitter" href="http://twitter.com/YOUR_PAGE" style="font-size: 0px; width:48px;height:48px;margin:0;margin-bottom:5px;margin-right:5px;"><img alt=twitter title="Follow us on Twitter" width=48 height=48 style="display: inline; width:48px;height:48px; margin: 0; padding: 0; border: none; box-shadow: none;" src="http://www.rtl-sdr.com/wp-content/uploads/2014/07/twitter.png"/></a><a target=_blank rel=nofollow title="Subscribe to our RSS Feed" href="http://www.projectilefish.com/feed/" style="font-size: 0px; width:48px;height:48px;margin:0;margin-bottom:5px;"><img alt=rss title="Subscribe to our RSS Feed" width=48 height=48 style="display: inline; width:48px;height:48px; margin: 0; padding: 0; border: none; box-shadow: none;" src="http://www.rtl-sdr.com/wp-content/uploads/2014/07/rss.png"/></a>

This code will result in the following

Follow Us

facebooktwitterrss

#8. Use Image Thumbnails

When you insert an image into WordPress it is inserted at the full image size. I recommend reducing the image size down to a smaller size for the post. The user can then simply click on the image to view the full sized version. To set thumbnail sizes that should be created go to Settings >> Media in the WordPress Admin area. Choose the sizes that your small, medium and large thumbnails should be. Then to insert a thumbnail simply change the size option to one of the thumbnails when adding or editing the image.

#9. Use a Minifier

A minifier WordPress plugin can be used to help optimize CSS and Javascript code being used on your site. A minifier works by rewriting the CSS or Javascript code down so that its text size is much smaller. This reduces the amount of data that needs to be sent over the internet. There are various minifiers available as plugins and I recommend trying out Autoptimize.

After using a minification plugin you must check that your site functions properly. In some cases minification can break some sites. But don’t worry as changes can be undone simply by removing or turning off the options in the minification plugin.

#10. Add Far Future Expiration for Web Browser Caches

Far Future Expiration sets how long a users web browser will locally cache content. When a user who has locally cached content reloads your page, that content will be served from the users local machine. Generally, we want things like images, icons and fonts to be cached for a long time, like maybe a month or more. Add this to your .htaccess file to enable far future expiration.

# BEGIN Far Future Expiration
<IfModule mod_expires.c>
ExpiresActive on
<FilesMatch "\.(gif|jpeg|jpg|png|ico|woff)$">
ExpiresDefault "access plus 720 hours"
</FilesMatch>
# CSS
ExpiresByType text/css "access 24 hours"
# Javascript
ExpiresByType application/javascript "access plus 24 hours"
</IfModule>
# END Far Future Expiration

#11. Delete Unused Plugins

Every plugin you add to WordPress increases it’s load time. Some more than others. Make sure you are only using plugins that you absolutely need and uninstall any that you are not using.

#12. Use P3 Plugin Profiler

P3 Plugin Profiler is a WordPress plugin that will analyze each plugin you have installed and determine how much CPU time they take up per page load. By using this profiler you can identify any plugins that are causing noticeable performance issues.

#13. Use YouTube Lyte

If you place YouTube videos on your blog you should use the YouTube Lyte plugin. This plugin ‘lazy loads’ the slightly bloated YouTube media player. Lazy loading means that the YouTube media player will only load when the user clicks on the video to watch it. Before that point a simpler lighter display only version of the YouTube media player is shown.

This plugin is especially useful and can make a huge difference if you have multiple YouTube videos per page.

#14. Delete Comment Spam Folder

Another major issue was that with the new found popularity spam bots had begun hitting the site pretty hard. After a month of neglect, my comment spam folder had over 6,000 spam comments marked by Askismet waiting to be cleared. A large amount of comments in the spam folder will increase the comment database size making it slow to access. To fix this simply delete all spam comments from your spam folder.

#15. Set Askismet to Delete Spam Straight Away

If you are using Askismet then to prevent the spam folder from growing too large again I set the Askismet strictness setting to “Silently discard the worst and most pervasive spam so I never see it.” Askismet is a free service that will detect and filter spam comments into your comments spam folder. Though for me at least I find that Askismet causes a lot of false positives. Instead I use a combination of Cookies for Comments (below) and WP-No-Bot-Question to stop spam.

#16. Use the Cookies for Comments Plugin

Although Askismet catches most spam comments it still lets some through every now and then. The plugin “Cookies for Comments” is a highly effective method at stopping comment spam. Essentially it works using two methods. One is that it checks for the presence of cookie that it sets. Spam comment bots often do not store cookies. Secondly, it treats any comment that was posted within X seconds of a page load as spam as it is unlikely that a human would post a comment so fast.

Be sure to add the recommended code to your .htaccess file as this will stop spam bots from even entering your site and eating up precious resources.

An alternative to Cookies for Comments that might work even better at stopping spam is Spam Destroyer. As well as the cookie technique spam destroyer requires the presence of Javascript, which many bots do not have enabled. Although, with Spam Destroyer you cannot block bots through your .htaccess file.

#17. Block XMLRPC Requests

After the site got popular when looking at my server logs one thing that I noticed was an inordinate amount of XMLRPC requests coming through. These requests were eating up significant CPU time and causing large slowdowns. This was probably caused by some sort of malicious bot trying to spam pingbacks or take down the site.

XMLRPC is used by the trackback and pingback functions of WordPress. Some plugins like Jetpack also require it. As I didn’t really need these functions I decided to simply block all XMLRPC requests. This can be done by editing your sites .htaccess file and adding the following code to the top of it.

ErrorDocument 503 "System Undergoing Maintenance"
RewriteEngine On
RewriteCond %{THE_REQUEST} "/xmlrpc.php" [NC]
RewriteRule .* - [R=503,L]

After blocking these spammy requests the server CPU usage went down and the site started loading faster.

#18. Stop WP-Login Attacks

WP-Login attacks are run by bots who attempt to bruteforce their way into your Wordpress account through the WP-Login admin page by guessing passwords many times a second. These attacks can hammer a site causing excessive CPU usage. I recommend using the Security-protection plugin to stop these attacks. WP-Security is my pick as it tricks the bot into believing that it has gained entry thus stopping the attack early and saving you from excessive server load. As a security tip I also recommend not using the ‘admin’ username for your account as the bots often only check this username.

If you have admin access to your server I also recommend setting up a fail2ban jail to stop these attacks.

#19. Use Fail2Ban to stop Repeat Spammers

By using fail2ban together with a WordPress spam logging plugin we can block repeat spammers at the firewall level. This will stop repeat spammers hammering your site and eating up precious resources. This is really only needed if your website it being targeted by aggressive spam bots which try and post a new spam message every few minutes. I used this tutorial to implement fail2ban spam protection.

#20. Install Mod_Evasive

Mod_evasive is a tool that will help your server to evade denial of service (DOS) attacks. If one of these attacks occurs on your server you hosted sites will take a massive performance hit. Mod_evasive blocks access to any IP that is DOSing your site.

#21. Serve Images Directly

I found a modification to the WordPress .htaccess file which will allow images to be served directly without invoking WordPress at all. Credit goes to this post by Rod Holmes. In your .htaccess file find the following

RewriteEngine On
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]

and replace it with the following

RewriteEngine on
#
# Unless you have set a different RewriteBase preceding this point, you may delete or comment-out the following RewriteBase directive:
#RewriteBase /
#
# if request is for image, css, or js file
RewriteCond %{REQUEST_FILENAME} \.(png|gif|jpe?g|css|js|ico)$ [NC,OR]
# or if URL resolves to existing file
RewriteCond %{REQUEST_FILENAME} -f [OR]
# or if URL resolves to existing directory
RewriteCond %{REQUEST_FILENAME} -d
# then skip the rewrite to WP
RewriteRule ^ - [S=1]
# else rewrite the request to WP
RewriteRule . /index.php [L]

#22. Upgrade PHP

Many servers are still running PHP 5.3 which has been declared end of life. Significant speed gains can be found by using the latest PHP 5.5 or 5.4. PHP 5.5 now has the Zend opcache built in which is a method that is used to significantly speed up PHP requests through code caching.

On some hosts with cPanel, upgrading PHP is a simple matter of going to Advanced >> PHP Configuration and changing the default PHP to 5.5. Note that after upgrading you will need to check your websites functionality as upgrades can sometimes cause problems.

#23. Use the latest WordPress

WordPress is constantly under development and new versions are released every few months. Usually new versions bring improvements such as speed boosts.

#24. Control Post Revisions

Every time you make a change to a post or page WordPress stores a revision copy just in case you need to go back to an old revision. By default WordPress stores ALL revisions you make. So if you edit your posts a lot there could be dozens of revisions taking up database space. I recommend you install the Revision Control plugin which will control the number of revisions kept. Set the number of revisions to how many you think you’ll need. In my case I chose to only keep 3 revisions at a time.

#25. Stop No-Referrer Comment Spam and Login Attacks

We can block spam bots and hackers brute forcing their way into the wp-login page with simple .htaccess file code. When a bot tried to access a page, it will usually try to access it directly without first visiting your site. If this happens we can detect it and block access to the bot with the .htaccess code below. Unfortunately, this isn’t effective as it used to be as modern advanced bots will get around this, but it will still stop many bots using simpler methods.

# Stop spam attack logins and comments
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{REQUEST_METHOD} POST
RewriteCond %{REQUEST_URI} .(wp-comments-post|wp-login)\.php*
RewriteCond %{HTTP_REFERER} !.*YOUR_DOMAIN.com.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^$
RewriteRule (.*) http://%{REMOTE_ADDR}/$ [R=301,L]
</IfModule>