Thursday, February 19, 2009

The ClamAV Active Malware Report that was introduced in ClamAV 0.94.1 uses freshclam to send summary data to our server about the malware that has been detected. This data is then used to generate real-time reports on active malware. These reports, along with geographical and historic trends, will be published on http://www.clamav.net/.

Initial reports, viewable on our website, have shown two interesting facts that we did not predict. Firstly, over 50% of the malware that we detect is phishing attacks. Most of these are found by our heuristic system, which analyses data for the techniques that phishers use, the rest are found by signatures in our database. Although many people in the security industry have been warning about phishing emails for some time, the malware authors must still think it worthwhile to attack. Sadly innocent victims are still being conned to hand over their bank and other personal details to criminals. Secondly, some very old viruses, such as SomeFool (also known as Netsky) are still very active in the field years after we first detected them. Either many PCs have no anti-virus protection, or that protection is inadequate.

Currently we publish a daily report that lists the top 10 active malware in the last 24 hours. We hope, by the end of February, to have enough data to launch a statistics page full of more information. The more data that we receive from ClamAV users, the more reports, and the better the quality of the reports, will be. To enable the submission of data to us for use in the Active Malware Report, enable SubmitDetectionStats in freshclam.conf, and LogTime and LogFile in clamd.conf.

The only private data that is transferred is an IP address, which is used to create the geographical data. The size of the data that is sent is small; it contains just the filename, malware name and time of detection. The data is sent in sets of 10 records, up to 50 records per session. For example, if you have 45 new records, then freshclam will submit 40; if 78 then it will submit the latest 50 entries; and if you have 9 records no statistics will be sent.

Tuesday, January 20, 2009

ClamAV 0.95, which is currently scheduled for release by Sourcefire in March 2009, will include a redesigned and completely rewritten clamav-milter.

Developers and keen users of ClamAV may have noticed that the version of clamav-milter within the SVN repository has changed a lot. We want to let you know what we’ve done and why.

The most notable difference in the new clamav-milter is that the internal mode has been dropped which means that now you will need to run clamd. This has not only allowed us to keep clamav-milter compact and readable, but also it avoids a lot of code duplication. With the old clamav-milter, internal mode was almost the same as having an outdated clamd with a milter interface because we were not keeping the code up-to-date with clamd’s API.

The second important difference is that now clamav-milter has its own configuration and log files that replace the large number of command line switches in the previous version. To ease the difficulty of learning another configuration file, the new clamav-milter comes with a program that will generate a configuration file from your existing command line options and clamd.conf file.

Some features are no longer supported:

  • Notifications
  • Black-listing
  • Phish false positive prevention by use of a subset of SPF
  • Scanning information is no longer added to the email headers by default
  • Scanning and other information can no longer be added to message bodies

So Why Has This Been Done?

Nigel Horne, the program’s previous author, is no longer a member of ClamAV’s engineering team – he is now ClamAV’s product manager. The milter program did not support many new features included within ClamAV and hence clamav-milter was starting to lag behind and bugs were not being addressed. The code was over 7500 lines and we felt it was a great opportunity to rewrite the code from scratch to be more closely coupled with the rest of ClamAV.

As a result we have been able to support new features, including:

  • Clamav-milter can now run as a completely unprivileged user (e.g. nobody)
  • Quarantine has been reworked to use the native milter interface on later versions of Sendmail and Postfix that support it
  • White-listing now uses regular expressions, replacing strict matching
  • Support for Postfix has been added in addition to Sendmail
  • Full IPv6 support.

The new milter’s configuration file is designed to be consistent with the configuration file for clamd, allowing you to fine-tune specific configurations and to route log messages to a dedicated file.

The new milter supports load balancing to copies of clamd in a round-robin fashion. Should one instance of clamd temporarily go down, clamav-milter will issue probe requests every few minutes and the instance will be re-entered into the pool as soon as it becomes available again. Scan requests to remote clamds are performed via the STREAM command, while requests to a local scanner are (preferably) sent via a FILDES command (file descriptor passing over a UNIX socket). This allows systems’ administrators to run clamav-milter and clamd as different users.

And last, but not least, if you prefer, you can continue to use the old version which is kept under …/contrib/old-clamav-milter.
The new clamav-milter will supersede the old one in ClamAV 0.95. Whilst the previous version will still be available, it will no longer be supported.

Installation Instructions

Run ./configure --enable-milter and make as usual.

There are two ways to configure the new clamav-milter:

  • Use the example clamav-milter.conf that we have provided as a template for your configuration. It's well documented, but if something's not clear please report it to bugs.clamav.net and we’ll fix it.
  • Run the make-clamav-milter-conf.pl script with the same run-time arguments you currently pass to clamav-milter and a configuration file will be generated automatically.

Known Issues

The white-list format now uses regular expressions; the old format was a list of strings wrapped in “<>”, the white-list file will therefore need to be edited. We plan to add automated conversion of the file to later versions of the configuration converter script, in the meantime the file will need to be edited by hand.

The round-robin clamd selection requires more work. It works well with either one or a high number of instances of clamd, but the round-robin strategy is limited when the number of scanners is as low as two or three.

And Finally…

The new milter is currently a work in progress. Although we’ve tested it with several hundred GBs of emails, real-life situations are usually more complex than inside the lab. We’re working hard to ensure that clamav-milter is portable to more operating systems.

Please send us your feedback on the new program by adding a comment to this blog; we’re really interested to know what you think!