Categories
Documentation Mail

Calculate statistics from your Qmail logfiles using Awstats

Awstats is a commonly used program to calculate web statistics from your webserver logfiles. It can detect useragents, referers, unique visitors… But one of its another feature is to build usage reports from your mail server’s logfiles, as seen on this demo.

Here is how to quickly configure Awstats to take profit of your Qmail log files…

 

mailboxes

 

Categories
Documentation PHP

How to package PHP extensions by yourself

I often receive emails telling me that Dotdeb is a great tool, but that some useful packages are missing, such as some PECL extensions. I wish I could maintain many and many packages, but I don’t think it’s a good idea for the Dotdeb’s overall quality and for my free time 😉 Sorry for that.

Then, this article will show you how to build packages from your favorite PECL extensions in a strict Debian way, using the dh-make-php package.

Categories
Documentation Mail

How to bind your Qmail server to a specific IP address

Just a little tip :

By default, Qmail listens to all the available IP address on the machine (0.0.0.0). It is possible to change this behaviour to bind Qmail to a specific IP.

We suppose that you are using Qmail from Dotdeb and launching it using the provided init script, /etc/init.d/qmail. Just edit it and change these lines :

          sh -c "start-stop-daemon --start --quiet --user qmaild \
          --pidfile /var/run/tcpserver_smtpd.pid --make-pidfile \
          --exec /usr/bin/tcpserver -- -H -P -R -l 0 \
          -u `id -u qmaild` -g `id -g nobody` -x /etc/tcp.smtp.cdb 0 smtp \
          $rblsmtpd /usr/sbin/qmail-smtpd 2>&1 \
          | $logger &"

by those ones :

          sh -c "start-stop-daemon --start --quiet --user qmaild \
          --pidfile /var/run/tcpserver_smtpd.pid --make-pidfile \
          --exec /usr/bin/tcpserver -- -H -P -R -l 0 \
          -u `id -u qmaild` -g `id -g nobody` -x /etc/tcp.smtp.cdb xxx.xxx.xxx.xxx smtp \
          $rblsmtpd /usr/sbin/qmail-smtpd 2>&1 \
          | $logger &"

(Just replace xxx.xxx.xxx.xxx by your IP address)

Now, when you list your listening dameon, you should see tcpserver listening to xxx.xxx.xxx.xxx:25 instead of 0.0.0.0:25.

machine# netstat -apn
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 xxx.xxx.xxx.xxx:25      0.0.0.0:*               LISTEN      21175/tcpserver
Categories
Documentation MySQL

Using the Google Perftools to speed up your MySQL server

The Google Perftools, especially tcmalloc (Thread Caching Malloc), can be very useful to speed up your applications, depending on your environment :

TCMalloc is faster than the glibc 2.3 malloc (available as a separate library called ptmalloc2) and other mallocs that I have tested. ptmalloc2 takes approximately 300 nanoseconds to execute a malloc/free pair on a 2.8 GHz P4 (for small objects). The TCMalloc implementation takes approximately 50 nanoseconds for the same operation pair.

Its deployment in your LAMP stack can speed up your MySQL servers, since it enhances memory allocation on threaded applications with the downside of larger memory footprints.

Here is how to use it easily…

Installation

First of all, be sure you use Debian 5.0 (a.k.a. “Lenny”) or later. Then install the minimal tcmalloc library from Dotdeb :

apt-get install libtcmalloc-minimal0

Then, since the mainstream MySQL packages are not compiled against tcmalloc, you’ll have to trick your OS’ dynamic linker by adding the following line at the top of your /etc/init.d/mysql init script :

export LD_PRELOAD="/usr/lib/libtcmalloc_minimal.so.0"

/etc/init.d/mysql modification

After relauching your MySQL server using the modified init script, you’ll take profit from tcmalloc’s faster memory allocation.

Results

At the time of writing this article, I didn’t make benchmarks, but some reported that they had a ~15-20% performance gain. Please leave comments about your experience.

Categories
Documentation

Mirroring Dotdeb

If Dotdeb is useful for you and if you want to mirror it, just add a cron job to periodically fetch the packages :

rsync -a --delete rsync.dotdeb.org::packages/ /your/local/path/

Then, if you wish your mirror to appear on the mirrors list, send me its full URL.

Note 1 : mirroring once a day should be enough. More frequent updates could lead to a ban without any warning.
Note 2 : Using anything else than Rsync (Wget, HTTrack…) could lead to a ban without any warning.

Categories
Documentation PHP

Storing your PHP sessions using memcached

Using PHP sessions can be a problem when your PHP applications are load-balanced on many web servers. You can store them on a NFS export or recode the session_set_save_handler using a SQL backend for example. But there is no solution more efficient, more scalable, more performant and easier to deploy than using memcached…

Memcached is a high-performance, distributed memory object caching system, generic in nature, but intended for use in speeding up dynamic web applications by alleviating database load…

Many well-known huge architecture (Facebook, Livejournal, Youtube…) are using it as memory caching to reduce the load on their servers. It can also be used to share PHP sessions among several servers. Let’s see how…

Installation

The first thing is to install the memcached server on your Debian server :

apt-get install memcached

Then, since the memcache PECL extension now provides its own session handler, it’s easy to plug PHP and memcached servers. Just install the appropriate extension (from Dotdeb) :

apt-get install php5-memcache

and change some of your PHP settings :

session.save_handler = files
; session.save_path = "N;/path"

to :

session.save_handler = memcache
; change server:port to fit your needs...
session.save_path="tcp://server:port?persistent=1&weight=1&timeout=1&retry_interval=15"

That’s all! After relaunching your Apache2 server, your PHP sessions will be stored on the memcached server.

Categories
Documentation Mail

How to enable greylisting on your Qmail server

Greylisting is very useful to avoid most of the incoming spam on your mail server. The Qmail packages shipped on Dotdeb have built-in MySQL-based greylisting. Here is how to enable it…

Installation

First of all, be sure to have Qmail installed from Dotdeb with a version number greater than 1.03-37.dotdeb.1.
Then, since we’ll use a MySQL backend to share the greylisting database between several servers, be sure to have MySQL installed somewhere on your boxes.

In our example, we’ll create the database (we’ll name it relaydelay) and grant access to user@host with the password “password“. You’re free to change these values to fit your needs…

CREATE DATABASE relaydelay;
GRANT ALL PRIVILEGES ON `relaydelay`.* TO 'user'@'localhost' IDENTIFIED BY 'password';

Then, we create the needed tables :

CREATE TABLE relaytofrom
(
  id              bigint          NOT NULL        auto_increment,
  relay_ip        char(16),
  mail_from       varchar(255),
  rcpt_to         varchar(255),
  block_expires   datetime        NOT NULL,
 
  record_expires  datetime        NOT NULL,
  blocked_count   bigint          default 0 NOT NULL,
  passed_count    bigint          default 0 NOT NULL,
  aborted_count   bigint          default 0 NOT NULL,
  origin_type     enum("MANUAL","AUTO") NOT NULL,
  create_time     datetime        NOT NULL,
  last_update     timestamp       NOT NULL,
 
  primary key(id),
  key(relay_ip),
  key(mail_from(20)),
  key(rcpt_to(20))
);
 
CREATE TABLE dns_name
(
  relay_ip      varchar(18)       NOT NULL,
  relay_name    varchar(255)      NOT NULL,
  last_update   timestamp         NOT NULL,
  primary key(relay_ip),
  key(relay_name(20))
);
 
CREATE TABLE mail_log
(
  id              bigint          NOT NULL        auto_increment,
  relay_ip        varchar(16)     NOT NULL,
  relay_name      varchar(255),
  dns_mismatch    bool            NOT NULL,
  mail_from       varchar(255)    NOT NULL,
  rcpt_to         varchar(255)    NOT NULL,
  rcpt_host       varchar(80)     NOT NULL,
  create_time     datetime        NOT NULL,
 
  primary key(id),
  key(relay_ip),
  key(mail_from(20)),
  key(rcpt_to(20))
);

We now have to put the appropriate settings in the /etc/init.d/qmail init script…

export MYSQLHOST="localhost"
export MYSQLUSER="user"
export MYSQLPASS="password"
export MYSQLDB="relaydelay"
export BLOCK_EXPIRE=5           # minutes until email is accepted
export RECORD_EXPIRE=600        # minutes until record expires
export RECORD_EXPIRE_GOOD=36
export LOCAL_SCAN_DEBUG=0

and in the /usr/sbin/greylisting-delete-expired :

MYSQLHOST="localhost"
MYSQLUSER="user"
MYSQLPASS="password"
MYSQLDB="relaydelay"

We can now relaunch Qmail and enable or disable the greylisting with a simple symbolic link :

/etc/init.d/qmail
ln -s /usr/bin/qmail-envelope-scanner /usr/sbin/qmail-envelope-scanner

Usage

Example wildcard whitelists for subnets :

INSERT INTO relaytofrom VALUES (0,"127.0.0.1"   ,NULL,NULL,"0000-00-00 00:00:00","9999-12-31 23:59:59",0,0,0,"MANUAL",NOW(),NOW());
INSERT INTO relaytofrom VALUES (0,"192.168"     ,NULL,NULL,"0000-00-00 00:00:00","9999-12-31 23:59:59",0,0,0,"MANUAL",NOW(),NOW());

Example wildcard whitelist entry for a recieved domain or subdomain

INSERT INTO relaytofrom VALUES (0,NULL,NULL,"sub.domain.com","0000-00-00 00:00:00","9999-12-31 23:59:59",0,0,0,"MANUAL",NOW(),NOW());
Categories
Documentation PHP

How to use PHP5 and PHP4 on the same Apache2 server

Since PHP4 is officially dead, it is now urgent to migrate your applications to PHP5, but it’s not that easy and you’ll have to test them before replacing PHP4 by PHP5 on your server. This article will help you to install both PHP4 and PHP5 on your box and swicth easily between them…

The problem

It is not possible to enable both PHP4 and PHP5 Apache2 modules on the same webserver, it causes shared objects collision. Then, the idea is to enable PHP5 as Apache2 module on one side and PHP4 as CGI on the other side.

Please note that this solution is not a good idea anywhere else but on developpement servers and should be temporary because of the performance and security issues of PHP as CGI.

Installation

First, we have to install the appropriate PHP packages and their dependencies (from Dotdeb, of course) :

apt-get install libapache2-mod-php5 php4-cgi

Apache2 is now configured by default to parse .php files with the mod_php5 module.

But, using mod_actions, we can tell Apache2 to parse .php files with the php4 CGI. First, enable mod_actions :

a2enmod actions
/etc/init.d/apache2 force-reload

Now, by adding the following two lines in a Virtualhost or Directory (…) context or directly in the main Apache2 configuration file, we’ll switch from PHP5 to PHP4 :

AddHandler php-script .php
Action php-script /cgi-bin/php4

(It could be necessary to reload your apache configuration :

/etc/init.d/apache2 force-reload

)

The result
To check that all works fine, create a file, called phpinfo.php, containing the following code :

Depending on the presence (or not) of the two above magical lines, a HTTP request on the script will show a PHP4 (or PHP5)-typical output.

Categories
Documentation Mail

How to enable the SMTP authentification on your Qmail server

The Qmail package from Dotdeb has been built with SMTP-auth features (but disabled by default). Here is the way to use them…

Installation

First of all, we suppose that :

  • you installed the last Qmail and Vpopmail packages from Dotdeb
  • you’re launching them using the bundled init scripts (with tcpserver)

Configuration

To enable SMTP-auth, you just have to edit the /etc/init.d/qmail init script and replace these lines :

sh -c "start-stop-daemon --start --quiet --user qmaild
  --pidfile /var/run/tcpserver_smtpd.pid --make-pidfile
  --exec /usr/bin/tcpserver -- -R
  -u `id -u qmaild` -g `id -g nobody` -x /etc/tcp.smtp.cdb 0 smtp
  $rblsmtpd /usr/sbin/qmail-smtpd 2>&1
  | $logger &"

by these ones :

sh -c "start-stop-daemon --start --quiet --user qmaild
    --pidfile /var/run/tcpserver_smtpd.pid --make-pidfile
    --exec /usr/bin/tcpserver -- -R
    -u `id -u qmaild` -g `id -g nobody` -x /etc/tcp.smtp.cdb 0 smtp
    $rblsmtpd /usr/sbin/qmail-smtpd /usr/sbin/vchkpw /bin/true 2>&1
    | $logger &"

Then, we have to setuid /usr/sbin/vchkpw :

chmod u+s /usr/sbin/vchkpw

and restart Qmail :

/etc/init.d/qmail stop
/etc/init.d/qmail start

The result

You will now be able to use the SMTP PLAINTEXT authentification. You just have to configure your favorite mail transport agent…