Mitigating DDoS Attacks nginx + ngx_mruby + http-dos-detector

https://github.com/matsumoto-r/http-dos-detector

Detect Huge Number of HTTP Requests on Apache and nginx using mruby code.
http-dos-detector use same Ruby code between Apache(mod_mruby) and nginx(ngx_mruby).

It seems, programmable DDoS firewall by mruby on nginx.
This solution provides regulating the incoming HTTP/S traffic and controlling the traffic as it is proxied to backend servers.
Let's try.

Environment:  Amazon Linux AMI 2015.09.1 (HVM), SSD Volume Type - ami-383c1956
c4.xlarge

Update Ruby
(Just update Ruby for Amazon Linux AMI. Actually, no meaning for ngx_mruby. ngx_mruby uses mruby.)

# yum remove ruby*
# yum install ruby22 ruby22-devel rubygem22 rubygem22-rake aws-amitools-ec2
# ruby -v
ruby 2.2.3p173 (2015-08-18 revision 51636) [x86_64-linux-gnu]

Install ngx_mruby with nginx
Why install nginx by yum version? because yum helps some setup operation easily. for example, mkdir, set logrotate, set start script and more.
Use /opt for FHS (http://www.pathname.com/fhs/pub/fhs-2.3.html#OPTADDONAPPLICATIONSOFTWAREPACKAGES).

# yum install git gcc bison openssl-devel pcre-devel nginx
# cd /opt
# git clone git://github.com/matsumoto-r/ngx_mruby.git && cd ngx_mruby
# NGINX_CONFIG_OPT_ENV='--prefix=/opt/nginx' sh build.sh
# make install
# cp /opt/nginx/sbin/nginx /usr/sbin/nginx
# nginx -V
nginx version: nginx/1.9.6
built by gcc 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC)
configure arguments: --add-module=/opt/ngx_mruby --add-module=/opt/ngx_mruby/dependence/ngx_devel_kit --prefix=/opt/nginx

Running "Hello mruby"

# emacs /etc/nginx/nginx.conf
    server {

        location /hello {
            mruby_content_handler_code '
                Server = nginx
                Server.echo "Hello mruby"
            ';
        }

# service nginx start
# curl 127.0.0.1/hello
Hello mruby

Benchmark nginx and ngx_mruby "Hello mruby"
nginx returns static file.

# echo "Hello nginx" > /usr/share/nginx/html/index.html
# ab -c 100 -n 1000000 127.0.0.1/
Requests per second:    38185.31 [#/sec] (mean)
Time per request:       2.619 [ms] (mean)

nginx + ngx_mruby returns "Hello mruby"

# ab -c 100 -n 1000000 127.0.0.1/hello
Requests per second:    36466.16 [#/sec] (mean)
Time per request:       2.742 [ms] (mean)

This is 95.04% power than "nginx static file".

Setup http-dos-detector
Git clone source code.

# cd /opt
# git clone git://github.com/matsumoto-r/http-dos-detector.git
# cp -r /opt/http-dos-detector/dos_detector /etc/nginx/conf.d

Customize dos_detector.rb file. This example is IP address base blocking.

# emacs /etc/nginx/conf.d/dos_detector/dos_detector.rb
Server = get_server_class
r = Server::Request.new
c = Server::Connection.new
cache = Userdata.new.shared_cache
global_mutex = Userdata.new.shared_mutex
host = r.hostname

ip_address = c.remote_ip
if r.headers_in["X-Real-IP"]
  ip_address = r.headers_in["X-Real-IP"]
elsif r.headers_in["X-Forwarded-For"]
  ip_address = r.headers_in["X-Forwarded-For"].split(",").first
end

config = {
  # dos counter by key
  :counter_key => ip_address,
  :magic_str => "....",
  # judging dos access when the number of counter is between :behind_counter and 0
  :behind_counter => -20000,

  # set behind counter when the number of counter is over :threshold_counter
  # in :threshold_time sec
  :threshold_counter => 10000,
  :threshold_time => 5,

  # expire dos counter and initialize counter even
  # if the number of counter is between :behind_counter and 0
  :expire_time => 10,
}

unless r.sub_request?
  # process-shared lock
  timeout = global_mutex.try_lock_loop(50000) do
    dos = DosDetector.new r, cache, config
    data = dos.analyze
    Server.errlogger Server::LOG_NOTICE, "[INFO] dos_detetor: detect dos: #{data}"
    begin
      if dos.detect?
        Server.errlogger Server::LOG_NOTICE, "dos_detetor: detect dos: #{data}"
        Server.return Server::HTTP_SERVICE_UNAVAILABLE
      end
    rescue => e
      raise "DosDetector failed: #{e}"
    ensure
      global_mutex.unlock
    end
  end
  if timeout
    Server.errlogger Server::LOG_NOTICE, "dos_detetor: get timeout mutex lock, #{data}"
  end
end

Modify nginx.conf. add error.log to notice and mruby_init, mruby_init_worker and mruby_access_handler.

# emacs /etc/nginx/nginx.conf

error_log  /var/log/nginx/error.log  notice;

http {

    mruby_init /etc/nginx/conf.d/dos_detector/dos_detector_init.rb cache;
    mruby_init_worker /etc/nginx/conf.d/dos_detector/dos_detector_worker_init.rb cache;

    server {
        location /dos_detector {
            mruby_access_handler /etc/nginx/conf.d/dos_detector/dos_detector.rb cache;
        }
    }
}

# service nginx restart

Benchmark http-dos-detector
nginx + ngx_mruby + dos_detector returns "Hello dos_detector"

# mkdir /usr/share/nginx/html/dos_detector/
# echo "Hello dos_detector" > /usr/share/nginx/html/dos_detector/index.html

Run Apache Bench

# ab -c 100 -n 1000000 127.0.0.1/dos_detector/

Complete requests:      1000000
Non-2xx responses:      800000

Requests per second:    9280.54 [#/sec] (mean)
Time per request:       10.775 [ms] (mean)

"nginx static file" can run much faster this. But, If backend application provides small power than this response (you have no C10k problem), nginx + ngx_mruby + http-dos-detector will be good solution for Mitigating DDoS Attacks.

See also.
ngx_mruby
https://github.com/matsumoto-r/ngx_mruby
mruby-logo provided by h2so5

Tweet