Note: I have updated the article with changes made by WPEngine regarding overage charges until 2nd March 2076.

When I shifted to WpEngine hosting few months back,  I shared the complete guide for moving and also added why I love their hosting. If you have missed all these details, here are few things which you should know:

Before moving to WPEngine, I was hosted on a private VPS, and one of the freelancer networking guys was managing my server. The performance was good, but there were few issues related to downtime and site slowness. Moreover, one biggest problem with the freelancer or less known hosting company is the accountability. So I decided to spend $250/month on WPEngine hosting and moved ShoutMeLoud to their server. Without any second thought, WPEngine delivers what they promised, and so far I’m in love with the quality of hosting and website performance. Though $250/month is indeed a high-price, and any knowledgable guy would recommend going for a VPS or dedicated server. In my case, I decided to stick to WPEngine, as I don’t want to see even a second downtime for my main blog, and when quality is everything, the price is the least of concerns.

So What’s the problem with WPEngine?

Two of the biggest complaint you will hear about WPEngine is Pricing and customer support. In past two months, They have significantly improved their customer support by adding live-chat, but the pricing issue remains the same. After using WPEngine for more than two months, I encountered one of the biggest pricing issues with WPEngine. This post is about the problem and the possible solution to the problem.

First of all, WPEngine pricing is based on the visits and the way they count the visits is different from what you see on your Google Analytics or WordPress.com stats program. According to ShoutMeLoud Google analytic stats, I get close to 325,000 visits a month, and I opted for WPEngine Business plan which allows up to 400,000 visits a month and is priced at $250.

Last time when I logged into my WPEngine dashboard, I was surprised to see an extra $188 bill for overage visits, and for this month, I was about to cross the threshold of 400,000 visits. I was quite surprised as, my WordPress.com stats and Google analytics were showing lower than 400,000 visits. Before I raise my concern to their team, I again had a look at their help page related to visits, and I realized I missed understanding few important points. Here is what their visits help page has to say:

1. When a human being first arrives on the site and loads the page, staying there for 31 seconds, that’s a visit.
2. If that same human then clicks a link and sees another page, that does not count as a new visit; that’s part of the same visit.
3. If that same human doesn’t have cookies or javascript enabled, still all that should count as one visit.
4. If that same human loads the site with different browsers, that’s still not a new visit; that’s part of the same visit.
5. If that same human bookmarks the site, then 11 days later comes back to the site, which is a new visit.
6. When a robot loads your site, our servers render it just like they would for a human visitor. However, this visit may not represent a value to you, which is why we will not count bot traffic toward your total visits.
7. If a robot scans 20,000 pages over the course of a month, that’s not just one visit.  It shouldn’t be 20,000 visits, but neither should it be 1. Something in the range of 100-1,000 visits is acceptable.
8. There are additional cases too where the “right thing to do” is less clear. For example, take the case of a “quick bounce.” Suppose a human clicks a link to the site, then before the site has a chance to load the human clicks “back.” Does that count as a visit? Our servers still had to render and attempt to return the page, so in that sense “yes.”  But a human didn’t see the site and Google Analytics isn’t going to see that hit, so in that sense “no.” Because we need the notion of a “visit” to correspond to “the amount of computing resources required to serve traffic,” we round off in favor of saying “yes.”

Two important points that I missed reading on the same page was:

• We reset our notion of “unique IP address” every day.
• Robots have the same few IP addresses, so they will be consolidated within one day, but will count again the next day.

Though everything is well documented on WPEngine help page, but I missed reading the point about resetting the unique I.P. Address every day, and even bot visits are counted as new visits after every 24 hours. ShoutMeLoud is one active blog, and not only Google bots, but there are virtually many Good + bad bots which crawl ShoutMeLoud daily, and I realized that added a lot to my number of visits according to WPEngine.

First of all, this notion of counting visits from WPEngine is not only flawed, but it adds a big hole in the pocket of any existing WPEngine users.  As I was considering to move away from WPEngine hosting after knowing how they are counting visits, and certainly this is not a customer-friendly model of pricing, I wanted to see if I could do something to make my WPEngine stats closer to my Google analytic stats. I asked them few questions, which will help any existing WPEngine user or anyone who is planning to purchase hosting from WPEngine:

Q) CDN: If I serve my images using CDN powered by WPEngine will that be counted as visits?

Ans.) Any request that hits CDN instead of the server does not count as a visit. So CDN is a helpful tool as well. But if these hotlinks pre-exist the enabling of CDN, it’s a tougher issue to address. Again, because CloudFlare is at the DNS level, it’s a simple, comprehensive solution.

Q) I see that WPEngine counts bad bots as visits too, do you have a solution for blocking these bad bots/spam/scrapers bots?

Ans.) CloudFlare allows you settings to fight abusive bots. But your bot traffic looks healthy and legit; although you did have over 5k visits from the bot for the popular Chinese search engine “Baidu.”

Q) Why does Google Analytics show a much lower visitor number?

Ans.) The number of visitors in Google Analytics is always going to be considerably lower. If you type the question into a Google search, you will see many people out there with various hosts are experiencing the same thing. The reason being is GA tries very hard to provide information only on human visitors that matter for Ad Traffic and will not include any visitors that have JavaScript disabled on their browsers.

A unique IP is counted as unique in an entire month while our system looks at unique IPs on a per day basis. They also will not include Google bots or various other bots in their visitor’s count. GA is used for marketing/advertising purposes, and that is why it works so hard only to count what they believe to be human visitors. Our server stats monitoring tracks all unique visitors because, from a server impact standpoint, a bot or a human is still pulling data down from the server and using CPU resources in the process.

Q) Is there a problem or something wrong with the GA implementation?

Ans.) Absolutely not. They have a specific purpose, to try and show you how many human visitors have come to your site. Our own stats has a specific purpose to show from a server impact perspective how much data is being transferred over our servers

Q) Also wondering if you could look into my above question related to WordPress.com stats program? Is it also flawed? or they work different than yours?

Ans.)  I wouldn’t say those stats are flawed. It’s just a tool that’s designed more for advertising purposes, for example, to measure “reach”. A good way to get stats that accurately reflect the traffic that actually hits the server is to see the stats (like “Daily Usage CSV”) available in your User Portal.

• Update 15th October 2015: WPEngine announced that they have removed bot traffic from the overage calculation. I hope one day they would eliminate counting bot visits as usual traffic.
• Update May 2016: WPEngine do NOT count “image visits” towards traffic charges.

How to lower down WPEngine Bills?

First of all, despite liking everything about WPEngine, I hate the way they are over-charging their customers. I always considered WPEngine to be a customer-centric company, and they have implemented many great features like Website security, free DNS, Website backup, Staging area, and many more to give significant features to their customers. But, sadly they have not implemented any measures to block the bad bots from accessing your website. None the less, they are charging their customers for such visits, which doesn’t even matter.

WPEngine team, if you are listening to this, I would love to see you implementing such features in your future iteration.

Anyways, to sum everything in simple words:

• WPEngine charges for bad bots visits.
• They reset their visit counters every 24 hours for human and for bots.
• They will charge you for hot linked images or images visits from Google image search.
• They don’t have any server level implementation for blocking bad bots visits

Now, after going through all this; I had a choice to move away from WPEngine, or keep paying the price. But, there is no fun leaving something which is good, and without making an extra effort. So first, I tried finding few .htaccess code to block bad bots visits. Alas! I couldn’t find any updated list of bad bots, so .htaccess was not a viable option. For now, I have implemented Cloudflare CDN, and will see how WPEngine stats shows the difference. If it all works out, I will be happy to stick with them, else there is no thought on saying Goodbye to WPEngine.  So, if you are an existing WPEngine customer, I highly recommend you to implement cloud flare CDN which is free (Also suggested by WPEngine staff), and it will help you to lower down your hosting bill.

• Also read: How to prevent Image hot linking from specific domains using .htaccess
• Use WP-Picshield plugin to prevent Image hot linking

Is WPEngine a good hosting company or bad?

There is a very thin line between deciding anything as bad or good. My part of the job is to share my experience, and regarding hosting quality WPEngine is par excellence, but in terms of pricing,it’s not affordable for everyone.  Their price is justified when you are running a business based WordPress site.

In my opinion, any busy site will usually face the issue of high-bill on WPEngine hosting, as busy websites usually get more bot visits, more hot linked images due to copy-paste posts. Undoubtedly, it might not be a very cost-effective solution for a busy website, whereas a mid-size WordPress website will not face such issue. It’s important for any existing WPEngine users to know how they are charging, and count visits. Moreover, you should implement measures like Cloudflare CDN or any custom solution to block bots visits to keep your limited hosting bill in check.

WPEngine alternative: Kinsta managed WordPress hosting & this is where I’m currently hosting ShoutMeLoud & ShoutMeHindi.

If you are an existing WPEngine customer, I would love to hear your review and opinion regarding their pricing plans and if you have faced any issue with their over-charging, I would love to hear your story.