How to install BFD (Brute Force Detection) Published: Jan 21, 2004
  • Rating

    5/5

If you are reaching the limits of your server running Apache serving a lot of dynamic content, you can either spend thousands on new equipment or reduce bloat to increase your server capacity by anywhere from 2 to 10 times.

Optimize High-Traffic Servers
Focus: Linux, Apache 1.3+, [PHP], [MySQL]
Notes: Use at your own risk. If this has any errors, please let me know and I will correct them.

Summary
If you are reaching the limits of your server running Apache serving a lot of dynamic content, you can either spend thousands on new equipment or reduce bloat to increase your server capacity by anywhere from 2 to 10 times. This article concentrates on important and poorly-documented ways of increasing capacity without additional hardware.

Problems
There are a few common things that can cause server load problems, and a thousand uncommon. Let's focus on the common:

Drive Swapping - too many processes (or runaway processes) using too much RAM
CPU - poorly optimized DB queries, poorly optimized code, runaway processes
Network - hardware limits, moron attacks

Solutions: The Obvious

Briefly, and for completeness, here are the most obvious solutions:

Use "TOP" and "PS axu" to check for processes that are using too much CPU or RAM.
Use "netstat -anp | sort -u" to check for network problems.

Solutions: Apache's RAM Usage
First and most obvious, Apache processes use a ton a RAM. This minor issue becomes a major issue when you realize that after each process has done its job, the bloated process sits and spoon-feed data to the client, instead of moving on to bigger and better things. This is further compounded by a bit of essential info that should really be more common knowledge:

If you serve 100% static files with Apache, each httpd process will use around 2-3 megs of RAM.
If you serve 99% static files & 1% dynamic files with Apache, each httpd process will use from 3-20 megs of RAM (depending on your MOST complex dynamic page).

This occurs because a process grows to accommodate whatever it is serving, and NEVER decreases again unless that process happens to die. Quickly, unless you have very few dynamic pages and major traffic fluctuation, most of your httpd processes will take up an amount of RAM equal to the largest dynamic script on your system. A smart web server would deal with this automatically. As it is, you have a few options to manually improve RAM usage.

Reduce wasted processes by tweaking KeepAlive
This is a tradeoff. KeepAliveTimeout is the amount of time a process sits around doing nothing but taking up space. Those seconds add up in a HUGE way. But using KeepAlive can increase speed for both you and the client - disable KeepAlive and the serving of static files like images can be a lot slower. I think it's best to have KeepAlive on, and KeepAliveTimeout very low (like 1-2 seconds).

Limit total processes with MaxClients
If you use Apache to serve dynamic content, your simultaneous connections are severely limited. Exceed a certain number, and your system begins cannibalistic swapping, getting slower and slower until it dies. IMHO, a web server should automatically take steps to prevent this, but instead they seem to assume you have unlimited resources. Use trial & error to figure out how many Apache processes your server can handle, and set this value in MaxClients. Note: the Apache docs on this are misleading - if this limit is reached, clients are not "locked out", they are simply queued, and their access slows. Based on the value of MaxClients, you can estimate the values you need for StartServers, MinSpareServers, & MaxSpareServers.

Force processes to reset with MaxRequestsPerChild
Forcing your processes to die after a while makes them start over with low RAM usage, and this can reduce total memory usage in many situations. The less dynamic content you have, the more useful this will be. This is a game of catch-up, with your dynamic files constantly increasing total RAM usage, and restarting processes constantly reducing it. Experiment with MaxRequestsPerChild - even values as low as 20 may work well. But don't set it too low, because creating new processes does have overhead. You can figure out the best settings under load by examining "ps axu --sort:rss". A word of warning, using this is a bit like using heroin. The results can be impressive, but are NOT consistent - if the only way you can keep your server running is by tweaking this, you will eventually run into trouble. That being said, by tweaking MaxRequestsPerChild you may be able to increase MaxClients as much as 50%.

[pagebreak title='Tweaking Apache Continued']

Apache Further Tweaking
For mixed purpose sites (say image galleries, download sites, etc.), you can often improve performance by running two different apache daemons on the same server. For example, we recently compiled apache to just serve up images (gifs,jpegs,png etc). This way for a site that has thousands of stock photos. We put both the main apache and the image apache on the same server and noticed a drop in load and ram usage. Consider a page had about 20-50 image calls -- the were all off-loaded to the stripped down apache, which could run 3x more servers with the same ram usage than the regular apache on the server.

Finally, think outside the box: replace or supplement Apache

Use a 2nd server
You can use a tiny, lightning fast server to handle static documents & images, and pass any more complicated requests on to Apache on the same machine. This way Apache won't tie up its multi-megabyte processes serving simple streams of bytes. You can have Apache only get used, for example, when a php script needs to be executed. Good options for this are:

TUX / "Red Hat Content Accelerator" - http://www.redhat.com/docs/manuals/tux/
kHTTPd - http://www.fenrus.demon.nl/
thttpd - http://www.acme.com/software/thttpd/

Try lingerd
Lingerd takes over the job of feeding bytes to the client after Apache has fetched the document, but requires kernel modification. Sounds pretty good, haven't tried it. lingerd - http://www.iagora.com/about/software/lingerd/

Use a proxy cache
A proxy cache can keep a duplicate copy of everything it gets from Apache, and serve the copy instead of bothering Apache with it. This has the benefit of also being able to cache dynamically generated pages, but it does add a bit of bloat.

[pagebreak title='Replace Apache Completely']

Replace Apache completely
If you don't need all the features of Apache, simply replace it with something more scalable. Currently, the best options appear to be servers that use a non-blocking I/O technology and connect to all clients with the same process. That's right - only ONE process. The best include:

thttpd - http://www.acme.com/software/thttpd/
Caudium - http://caudium.net/index.html
Roxen - http://www.roxen.com/products/webserver/
Zeus ($$) - http://www.zeus.co.uk

Solutions: PHP's CPU & RAM Usage
Compiling PHP scripts is usually more expensive than running them. So why not use a simple tool that keeps them precompiled? I highly recommend Turck MMCache. Alternatives include PHP Accelerator, APC, & Zend Accelerator. You will see a speed increase of 2x-10x, simple as that. I have no stats on the RAM improvement at this time.

Solutions: Optimize Database Queries

This is covered in detail everywhere, so just keep in mind a few important notes: One bad query statement running often can bring your site to its knees. Two or three bad query statements don't perform much different than one. In other words, if you optimize one query you may not see any server-wide speed improvement. If you find & optimize ALL your bad queries you may suddenly see a 5x server speed improvement. The log-slow-queries feature of MySQL can be very helpful.

How to log slow queries:

# vi /etc/rc.d/init.d/mysqld

Find this line:
SAFE_MYSQLD_OPTIONS="--defaults-file=/etc/my.cnf"

change it to:
SAFE_MYSQLD_OPTIONS="--defaults-file=/etc/my.cnf --log-slow-queries=/var/log/slow-queries.log"

As you can see, we added the option of logging all slow queries to /var/log/slow-queries.log
Close and save mysqld. Shift + Z + Z

touch /var/log/slow-queries.log
chmod 644 /var/log/slow-queries.log

restart mysql
service myslqd restart
mysqld will log all slow queries to this file.

References
These sites contain additional, more well known methods for optimization.

Tuning Apache and PHP for Speed on Unix - http://php.weblogs.com/tuning_apache_unix
Getting maximum performance from MySQL - http://www.f3n.de/doku/mysql/manual_10.html
System Tuning Info for Linux Servers - http://people.redhat.com/alikins/system_tuning.html
mod_perl Performance Tuning (applies outside perl) - http://perl.apache.org/docs/1.0/guide/performance.html

Once again, if this has any errors or important omissions, please let me know and I will correct them.
If you experience a capacity increase on your server after trying the optimizations, let me know!

Article written by spagmoid additions by: albo,huck on the ev1 forums.

  • Rating

    5/5

Related Articles

Comments (36)

  • Gravatar - Olly
    Olly 15:26, January 21, 2004
    What if your IP isnt static, (if you are talking about the ip which you use to connect, not your box ip) how would you get around this problem?
  • Gravatar - Mike
    Mike 02:14, January 23, 2004
    "What if your IP isnt static, (if you are talking about the ip which you use to connect, not your box ip) how would you get around this problem? "

    I was wondering the same thing, what if your IP isn't static.
  • Gravatar - matlex
    matlex 04:49, January 27, 2004
    I think its simple then.

    Add the whole IP range of your ISP.

    for example:
    If you ISP IP range
    12.234.345.0-12.234.345.255

    Add just this:
    12.234.345
  • Gravatar - bman
    bman 19:37, February 14, 2004
    any idears how to fix this ?
    tail -f /var/log/bfd_log
    Feb 14 20:35:00 xxxxxx BFD(8570): could not locate $FWFILE, aborting.
  • Gravatar - Alex
    Alex 15:03, March 16, 2004
    could not locate $FWFILE is because you didnt installed APF before
  • Gravatar - Ryan
    Ryan 23:48, July 20, 2004
    Q: what if 1, you dont have static ips, and 2, your isp has more then 2 b class ip blocks? as i dont want to add them all and open it upto all 3 1/4 million users they host.
  • Gravatar - pierre
    pierre 19:17, August 19, 2004
    how to install BFD can you write the command pls .<br />
    thank you
  • Gravatar - Nonel S. Pagdato
    Nonel S. Pagdato 09:12, October 4, 2004
    Hi, is there any BFD application for any flavor of microsoft server, hope to hear from you soon, thanks..
  • Gravatar - netcopan
    netcopan 16:58, December 19, 2004
    should we add it into cron? After I restart server it will still be there??
  • Gravatar - Steel Rat
    Steel Rat 22:05, January 12, 2005
    Also a line on how to have BFD startup automatically (unless it's tied to APF in that regard...
  • Gravatar - Alan
    Alan 22:39, January 21, 2005
    >Q: what if 1, you dont have static ips<br />
    <br />
    don't get your password wrong <br />
    <br />
    :)
  • Gravatar - Matrafox
    Matrafox 09:18, March 23, 2005
    >Q: what if 1, you dont have static ips<br />
    <br />
    Then if u type rong password and u are banned, u just have to reboot your machine to take another Ip, and just try again :)
  • Gravatar - xaver
    xaver 02:38, April 2, 2005
    "There is an included 'install.sh' script that installs all files to<br />
    '/usr/local/bfd/' and places a 8-minute cronjob in '/etc/cron.d/bfd'. The setup<br />
    is really as simple as that."
  • Gravatar - Chris
    Chris 00:03, July 22, 2005
    Thanks works perfect.
  • Gravatar - robbert
    robbert 16:00, August 8, 2005
    many thnx! works perfect!!!!
  • Gravatar - Mike
    Mike 00:37, September 6, 2005
    It would help ppl if you actually gave some good examples of what configurations are preferable on a cPanel or other server, all you are doing here is stating the contents on the RFX site or in a README.
  • Gravatar - yenren
    yenren 13:05, September 11, 2005
    do we have to restart bdf manually everytime we restarted server?
  • Gravatar - grabt
    grabt 10:58, October 22, 2005
    How to stop BFD
  • Gravatar - sam
    sam 11:44, January 18, 2006
    Its realy good!
  • Gravatar - Alexender
    Alexender 08:21, March 5, 2006
    grabt:<br />
    <br />
    to stop BFD , type<br />
    <br />
    /usr/local/sbin/bfd -s <br />
    <br />
    <br />
  • Gravatar - Luis
    Luis 02:49, March 15, 2006
    I actully managed to lock myself out of my server the other day after a few typing errors when trying to login... just went and turned my linksys router off, waited a few minutes and then reconnected... I had another IP and was able to enter again.
  • Gravatar - James Tervit
    James Tervit 19:07, April 17, 2006
    Installed BFD along with a few minor changes to my ports and it stopped failed authentications dead in its tracks..... very cool
  • Gravatar - Ramkriz
    Ramkriz 20:21, July 31, 2006
    BFD is not working with the default rules. Please give me some rules for BFD.<br />
    <br />
    Thanks.
  • Gravatar - Alan
    Alan 10:55, August 17, 2006
    does this bruteforce auto start with system reboot?
  • Gravatar - Jose
    Jose 06:52, September 14, 2006
    Q: Can i type in ignore hosts my dyndns host?<br />
    <br />
    Ex. abcdefg.dyndns.org<br />
    <br />
    Thanks for you teach
  • Gravatar - hello123
    hello123 05:51, October 25, 2006
    how to check whether BFD is started
  • Gravatar - Turkulerdiyari
    Turkulerdiyari 22:08, November 17, 2006
    I get this error :<br />
    could not locate $TLOGP, aborting.<br />
    <br />
    How can i fix this problem.
  • Gravatar - sss
    sss 11:51, January 16, 2007
    Really gud one
  • Gravatar - ePak
    ePak 22:09, February 1, 2007
    Q: Do I have to restart bfd everytime the server reboots?<br />
    <br />
    A: No. BFD is not a daemon. There is a cron that runs every 10 minutes that sweets the logs for "attacks." So basically, every 10 minutes, it runs on its own. Location of cron: /etc/cron.d/bfd<br />
    <br />
    <br />
    Q: How do I disable BFD?<br />
    <br />
    A: You can remove the cron from /etc/cron.d/bfd or you can use the uninstall script that came in the BFD tar.<br />
    <br />
    I like the script, but my question is, how many lines does it take to cause an IP block?
  • Gravatar - Radu
    Radu 15:55, February 23, 2007
    Has anybody try to install on suse ? Or
  • Gravatar - alex
    alex 22:39, March 18, 2007
    My bfd can't trace proftpd attack, why?<br />
    Can anyone help me?<br />
    Mar 18 22:23:47 ServerFTP proftpd[8035]: ServerFTP (82.191.154.xxx[82.191.154.xxx]) - USER anonymous: no such user found from 82.191.154.xxx [82.191.154.xxx] to 80.21.159.xxx:21
  • Gravatar - Riaan
    Riaan 21:52, September 24, 2007
    Hi there<br />
    <br />
    How can I change the rules so that bfd will block an IP after 2 attempt and what is the default attempts?
  • Gravatar - Tim
    Tim 14:10, October 13, 2007
    RE: Alexander<br />
    <br />
    /usr/local/sbin/bfd -s<br />
    <br />
    That does not STOP bfd, it runs it with standard output. BFD doesn't run as a service, but as cron job. So you'd have to remove the cron job as staed by ePak
  • Gravatar - Sabarish......
    Sabarish...... 21:44, January 20, 2008
    Hi Riaan,<br />
    <br />
    BFD is configured to block after 10 attempts. See the TRIG value inside the files on /usr/local/bfd/rules<br />
    <br />
    Let us know through mail, if u need any further assistance.
  • Gravatar - JT
    JT 18:29, September 26, 2008
    After restarting BFD, I got the following:<br />
    <br />
    /usr/local/sbin/bfd -s<br />
    BFD version 0.9 <bfd@r-fx.org><br />
    Copyright (C) 1999-2004, R-fx Networks <proj@r-fx.org><br />
    Copyright (C) 2004, Ryan MacDonald <ryan@r-fx.org><br />
    This program may be freely redistributed under the terms of the GNU GPL<br />
    <br />
    /usr/local/bfd/tlog: line 71: [: : integer expression expected<br />
    /usr/local/bfd/tlog: line 74: [: : integer expression expected<br />
    <br />
    What is the tlog bit mean? Please help...
  • Gravatar - dt
    dt 06:51, August 6, 2009
    I installed afp. ran it. see it in the log. then install bfd. ran it numerous times. nothing happens other than printing the copyright message, no matter what params I enter. i try failed logins from my other servers over and over and nothing happens. huge messages logfiles, doesn't seem to even try to scan trhem. bfd log file is completely empty. what is wrong? tried this on 2 servers, both didnt work

Add Your Thoughts

WebHostGear.com is a hosting directory, not a web host.

Copyright © 1998-2017 WebHostGear.com