Friday, May 20, 2016

ClamAV Community,

This notice is to inform you that effective June 1, 2016, ClamAV 0.97 (and all minor versions) is no longer supported in accordance with ClamAV's EOL policy which can be found here:

http://www.clamav.net/documents/end-of-life-policy-eol

EOL for ClamAV essentially means that we will no longer be testing against that version when we write signatures, and we may break something with a future release.  So, while signatures may work past June 1st, we are no longer testing that configuration, so you are on your own.

Please upgrade to the newest version of ClamAV, currently at 0.99.2, available for download.

Thank you for using ClamAV!

Tuesday, May 3, 2016

ClamAV 0.99.2 has been released, and is available for download at ClamAV.net's download site.

Note:  As previously discussed for the last three releases, we are no longer uploading ClamAV to SourceForge for release.  0.99.2 is the first release that is ONLY released on ClamAV.net

Below are the notes from the ChangeLog since 0.99.1:

Thu, 22 Apr 2016 12:45:00 -0500 (Steven Morgan)
------------------------------------------
 * ClamAV 0.99.2 release.

Thu, 31 Mar 2016 17:07:39 -0400 (Kevin Lin)
------------------------------------------
 * 7z: fix for FolderStartPackStreamIndex array index heck

Tue, 29 Mar 2016 16:18:51 -0400 (Steven Morgan)
------------------------------------------
 * bb11547 - print all CDBNAME entries for a zip file when using the
-z flag.

Tue, 2 Sep 2014 22:44:41 +0200 (Sebastian Andrzej Siewior)
------------------------------------------
 * try to minimize the err cleanup path

Tue, 2 Sep 2014 22:44:14 +0200 (Sebastian Andrzej Siewior)
------------------------------------------
 * clamunrar: notice if unpacking comment failed

Wed, 23 Mar 2016 16:39:52 -0400 (Steven Morgan)
------------------------------------------
 * bb9042 - signature manual update.

Wed, 23 Mar 2016 16:14:42 -0400 (Kevin Lin)
------------------------------------------
 * bb#11396 - use temp var for realloc to prevent pointer loss. Patch by
Bill Parker.

Wed, 23 Mar 2016 15:49:56 -0400 (Kevin Lin)
------------------------------------------
 * bb#11397 - fix debug VI hex truncation

Wed, 23 Mar 2016 15:38:21 -0400 (Kevin Lin)
------------------------------------------
 * bb#11398 - freshclam: avoid random data in mirrors.dat. Patch by
Tomasz Kojm.

Wed, 23 Mar 2016 15:28:51 -0400 (Kevin Lin)
------------------------------------------
 * libclamav: print raw certificate metadata

Wed, 23 Mar 2016 14:16:00 -0400 (Kevin Lin)
------------------------------------------
 * bb#11529 - freshclam manager check return code of strdup. Patch by
Sebastian A. Siewior.

Tue, 22 Mar 2016 16:21:59 -0400 (Kevin Lin)
------------------------------------------
 * bb#11261 - additional suppress IP notification when using proxy

Tue, 22 Mar 2016 12:54:52 -0400 (Kevin Lin)
------------------------------------------
 * bb#10983 - fix download and verification of *.cld through PrivateMirrors

Mon, 21 Mar 2016 11:21:08 -0400 (Kevin Lin)
------------------------------------------
 * bb#11261 - suppress IP notification when using proxy

Mon, 21 Mar 2016 11:20:01 -0400 (Kevin Lin)
------------------------------------------
 * bb#11543 - remove redundant mempool assignment

Thu, 17 Mar 2016 11:49:26 -0400 (Kevin Lin)
------------------------------------------
 * bb#11003 - divide out dumpcerts output for better readability

Wed, 16 Mar 2016 15:42:35 -0400 (Kevin Lin)
------------------------------------------
 * bb#11003 - fix dconf and option handling for nocert and dumpcert

Mon, 14 Mar 2016 16:07:45 -0400 (Mickey Sola)
------------------------------------------
 * bb11463 - patch by Jim Morris to increase clamd's soft file descriptor to
its potential maximum on 64-bit systems

Mon, 14 Mar 2016 17:12:20 -0400 (Steven Morgan)
------------------------------------------
 * Move libfreshclam config to m4/reorganization.

Fri, 11 Mar 2016 13:32:31 -0700 (andrey mirtchovski)
------------------------------------------
 * adding libfreshclam

Sun, 13 Mar 2016 23:27:23 -0400 (Tom Judge)
------------------------------------------
 * Add 'cdb' datafile to sigtools list of datafile types.

Fri, 11 Mar 2016 16:02:22 -0500 (Steven Morgan)
------------------------------------------
 * bb11526 - NULL pointer check. Patch by Bill Parker.

Fri, 11 Mar 2016 15:48:01 -0500 (Steven Morgan)
------------------------------------------
 * bb11524 - malloc() NULL pointer check. Patch by Bill Parker.

Thu, 10 Mar 2016 18:26:33 -0500 (Steven Morgan)
------------------------------------------
 * bb1436 - clamscan 'block-macros' option. Patch by Kai Risku.

Wed, 9 Mar 2016 17:07:06 -0500 (Kevin Lin)
------------------------------------------
 * bb#11514 - initialize cpio name buffer

Wed, 9 Mar 2016 16:43:03 -0500 (Kevin Lin)
------------------------------------------
 * bb#11514 - initialize mspack decompression buffers

Wed, 9 Mar 2016 12:15:16 -0500 (Kevin Lin)
------------------------------------------
 * bb#11514 - prevent memory allocations on used pointers (folder objects)

Tue, 8 Mar 2016 16:04:21 -0500 (Kevin Lin)
------------------------------------------
 * bb#11514 - prevent memory allocations on used pointers (boolvectors)

Tue, 8 Mar 2016 14:37:20 -0500 (Kevin Lin)
------------------------------------------
 * bb#11514 - initialize ARJ metadata structures

Tue, 8 Mar 2016 14:37:01 -0500 (Kevin Lin)
------------------------------------------
 * bb#11514 - change cli_malloc with cli_calloc

Mon, 7 Mar 2016 16:25:10 -0500 (Kevin Lin)
------------------------------------------
 * bb#11514 - check packSizes prior to dereference

Mon, 7 Mar 2016 16:10:09 -0500 (Kevin Lin)
------------------------------------------
 * bb#11514 - fixed inconsistent folder state on failure

Mon, 7 Mar 2016 15:11:08 -0500 (Kevin Lin)
------------------------------------------
 * bb#11514 - pre-check on (*unpackSizes) dereference

Mon, 7 Mar 2016 13:56:42 -0500 (Kevin Lin)
------------------------------------------
 * bb11514 - fix on pre-checks on dereferenced array

Fri, 4 Mar 2016 16:57:14 -0500 (Kevin Lin)
------------------------------------------
 * bb11514 - pre-checks on dereferenced array size values (not =0)

Wed, 2 Mar 2016 13:57:03 -0500 (Mickey Sola)
------------------------------------------
 * bb-11514 - adding sanity checks to 7z header parsing

Tue, 1 Mar 2016 12:43:01 -0500 (Kevin Lin)
------------------------------------------
 * bb#11514 - fixed mew source read issue

Fri, 4 Mar 2016 17:05:01 -0500 (Steven Morgan)
------------------------------------------
 * bb11188 - Upgrade to use libtool 2.4.6 for ClamAV building: fixes issues
with MacOSX 10.10 and 10.11.

Tue, 1 Mar 2016 12:34:48 -0500 (Kevin Lin)
------------------------------------------
 * bb#11513 - documentation update on targets

Mon, 29 Feb 2016 16:58:19 -0500 (Kevin Lin)
------------------------------------------
 * filetype consistency

Mon, 29 Feb 2016 11:34:25 -0500 (Kevin Lin)
------------------------------------------
 * move llvm option flag handling to new m4 file

Wed, 24 Feb 2016 13:29:42 -0500 (Kevin Lin)
------------------------------------------
 * hwp5.x: fix for streams without names

Please go and grab the newest ClamAV and begin your download cycles, as always, we encourage bug submissions at https://bugzilla.clamav.net and your feedback on the ClamAV lists.


Wednesday, April 6, 2016

We'd like to congratulate our first winner of the monthly ClamAV Community Signature contest:  Samuel Borell!

Congratulations!  Thank you for your contributions!

For more information on how you can get involved in the monthly contest and try and win your ClamAV swag, please see my post on the blog.

Once again, thank you, and congratulations to Samuel!

Wednesday, March 30, 2016

Here at ClamAV, we continue to welcome signature submissions to improve community detection. As a thanks to our community, we like to reward individuals with some cool “ClamAV swag” items such as our ClamAV t-shirts, battery chargers, Snort calendars, and other goodies for signature submissions accepted.

Standard rules for submission criteria:

We are accepting signatures into ClamAV via the Community-Sigs mailing list, which anyone may join here:  http://lists.clamav.net/cgi-bin/mailman/listinfo/community-sigs

When we receive a signature, we will follow our standard internal procedures (which involves heavy QA of the signature, testing, optimization for performance, and running it against our bank of clean files).

You may reference the ClamAV Signatures Manual for general signature questions, as well as discussing it among fellow ClamAV Signature writers on the above list.

The signatures are released in the Official ClamAV Signature Set and are available to the ClamAV community via our normal mirror rule distribution process, published approximately every four hours!

We will provide you feedback about how to improve your signatures such as what you should or should not do, tips and tricks involved with the latest versions of ClamAV, as well as giving the author full attribution for their submissions.

If you’d like to submit to the ClamAV signature set, please email the above list with your signatures and research behind it (pcap, ascii dump, references, anything!)

As always False positive reports belong here: http://www.clamav.net/reports/fp

The highest submitter for accepted signatures for each month will receive some goodies.  Keep in mind that we must accept the signatures.

We thank the community in advance for community submissions, as well as continued submission of false positive reports.

Tuesday, March 29, 2016

A Brief History

With the release of ClamAV 0.99, users saw a slew of new features aimed at enhancing ClamAV’s core functionalities. Included in these features were the improvements to ClamAV’s long-neglected on-access scanning capabilities. Ages ago, ClamAV supported on-access scanning via the now-defunct Dazuko kernel module. When development on Dazuko ended in 2011, the ClamAV team began searching for a suitable replacement. The team soon landed on the recently stable fanotify API that first shipped with the 2.6.37 Linux kernel. By November of 2011 a barebones fanotify-based on-access scanner had been completed, but it wouldn’t see public release until ClamAV 0.98.

Fast forward to ClamAV version 0.99four years after the creation of that first prototypeand users found themselves presented with a completely reworked on-access scanning system that now leveraged not only fanotify, but inotify as well. While ClamAV’s documentation had been updated to help navigate users through the transition, a number of people reported problems they were having with ClamAV’s on-access scanner. Almost invariably, these issues came down to either improper configuration or system-level limitations. This guide seeks to provide users interested in ClamAV’s on-access scanning with a definitive resource on the topicstraight from the horse’s mouth, so to speak.

System Requirements

fanotify
On-access scanning, requires a system running a Linux kernel (version >= 3.8) with fanotify compiled in. Check the kernel configuration for fanotify’s presence by running:
$ cat /boot/config-<kernel_version> | grep FANOTIFY

Results if fanotify is available:
CONFIG_FANOTIFY=y

Results if fanotify can prevent malicious file access attempts:
CONFIG_FANOTIFY_ACCESS_PERMISSIONS=y

If this is seen:
CONFIG_FANOTIFY=y
# CONFIG_FANOTIFY_ACCESS_PERMISSIONS is not set

Then on-access scanning will be constrained to notify-only mode and will be unable to prevent access to malicious files, since fanotify lacks the ability to block events on the system.

inotify
To use the dynamic directory determination (DDD) system introduced in 0.99, ensure inotify is configured with enough watchpoints. By default, inotify can watch for changes in 8192 directories at any one time. If more directories than that need protection, run the following commandreplacing <NUM> with the maximum number of directories to be watched:
$ echo <NUM> | sudo tee -a /proc/sys/fs/inotify/max_user_watches

Overview of Configuration Options

The clamd.conf configuration file contains a number of options which modify the behaviour of ClamAV’s on-access scanner. Each configuration option is listed below alongside its intended purpose and any caveats of which users should remain cognizant.

ScanOnAccess
Purpose: Enables on-access scanning under clamd if set to “yes”. If set to ”no”, all other on-access configurations options will be ignored.

Caveats: clamd must be run as root.

OnAccessMountPath
Purpose: Watches the entirety of one or more specified mount points and notifies the user of any malicious events caught by fanotify.

Caveats: Users must specify a valid mount point. If this option is enabled, clamd will disable fanotify prevention to keep users from accidentally locking up their system. Enabling this option will also disable the DDD system and all configuration options which rely on that system.

OnAccessMaxFileSize
Purpose: Allows the user to stop clamd from scanning files larger than the specified size. If set to “0”, clamd will scan all files regardless of size.

Caveats: On some kernel versions and architectures, fanotify is known to fail when blocking access to files larger than 4GB. ClamAV will alert users if this problem occurs.

OnAccessIncludePath
Purpose: The files within the any specified path(s) will be monitored for access attempts. If the DDD system is enabled, clamd will watch this path recursively, i.e., the specified directory and all subdirectories will be monitored. If the DDD system is disabled, only the topmost directory will be watched and all subdirectories will be ignored.

Caveats: This option will be ignored if OnAccessMountPath is enabled. As an additional precaution, if DDD is enabled, clamd will prevent users from specifying “/” as an include path.

OnAccessExlcudePath
Purpose: Recursively removes the given directory from the set of directories monitored by clamd.

Caveats: If the DDD system is disabled, this option is ignored.

OnAccessExlcudeUID
Purpose: When fanotify catches an event, clamd will check given UIDs against the captured pid. If a match is found, the event is ignored and no scanning is performed.

Caveats: There aren't any known caveats with this option.

OnAccessDisableDDD
Purpose: This option allows users to turn off the DDD system if they prefer the legacy on-access scanning behaviour that was available in 0.98. If DDD is disabled OnAccessIncludePath will not recursively watch paths, making OnAccessExcludePath a redundant option.

Caveats: If OnAccessMountPath is enabled, clamd will act as though DDD is disabled.

OnAccessPrevention
Purpose: Flags fanotify to block any triggered events on monitored files, which allows ClamAV to scan affected files to determine if those events should be allowed to proceed.

Caveats: The kernel must be compiled with “CONFIG_FANOTIFY_ACCESS_PERMISSIONS=y” for this option to function. As a precaution, if OnAccessMountPath is enabled, clamd automatically disables this option to stop the user from accidentally locking up their system. Only access and open events can be blocked by fanotify.

OnAccessExtraScanning
Purpose: Provides DDD users with inotify event coverage, which allows for scans on newly created, moved, or renamed files.

Caveats: Since this option relies on the inotify framework maintained by the DDD system, it is ignored if OnAccessDisableDDD is “yes”.

Configuration Examples

This guide ends with a few simple clamd.conf configuration examples which will satisfy a number of general use cases. Though there are only four, these examples can readily be expanded upon or adapted to fit more targeted use cases.

Watch entire filesystem
ScanOnAccess yes
OnAccessMountPath /
OnAccessExcludeUID 0

Details: This will set fanotify to watch the entire filesystem in real-time and trigger ClamAV to run scans on any files opened, accessed, or closed except by the root user. While clamd will report any viruses found during this scanning, fanotify will not perform any blocking or prevention.

Watching select users’ home directories
ScanOnAccess yes
OnAccessIncludePath /home
OnAccessExlcudePath /home/user2
OnAccessExcludePath /home/user4
OnAccessExtraScanning yes

Details: Like the previous example, fanotify is not flagged to prevent events which occur on malicious files. However, further scanning coverage is provided thanks to OnAccessExtraScanning leveraging the inotify events caught by the DDD system.

Protecting a directory recursively
ScanOnAccess yes
OnAccessIncludePath /home/user/Downloads
OnAccessPrevention yes
OnAccessExtraScanning yes

Details: Here fanotify will block any triggered events, allowing only clean files to be opened, or otherwise accessed. This configuration will also immediately alert the user if a malicious file is inadvertently downloaded.

Protecting multiple dirs non-recursively
ScanOnAccess yes
OnAccessIncludePath /home/user/dir1
OnAccessIncludePath /home/user/dir2
OnAccessPrevention yes
OnAccessDisableDDD yes

Details: Similarly to legacy behaviour, only the top level of dir1 and dir2 will be protected in this case. The contents of all subdirectories will be ignored and extra scanning via inotify events is not possible since DDD is disabled.

Wednesday, March 16, 2016

Our ClamAV Signature Interface maintenance is now complete.  While we apologize for the delay, the rollout of the the new Signature Interface inside of ClamAV will result in several new features for the community, and I wanted to tell you about some of them:

First, the first new “main.cvd” in about two years.  This main.cvd has been completely re-written from scratch, and while the function of the “main” is largely the same, it’s been rewritten to not only enforce order to the signatures, but naming convention as well.  For example:

W97M.Ethan.AK-1 has moved to Doc.Trojan.Ethan
Worm.Padowor.A-zippwd has moved to Win.Worm.Padowor
Adware.Smshoax has moved to Win.Adware.Smshoax

Re-naming of the signatures may affect a local user’s whitelist.  If you have excluded certain signatures in the past that are now firing, we ask that you both submit the file to us for false positive remediation (if you believe it to be a false positive), and rename the signature whitelist on your side.

This new main is 109Mb in size, and contains 4 million signatures for ClamAV.  Now that the main.cvd has been rewritten, it is now easier for us to create diffs, which means upgrading the main more often, and making the “daily.cvd” smaller more often.

Second,  we now have the ability to offer different types of CVDs.  For instance, we now have the ability to distribute 3rd party signatures that are officially signed by ClamAV, but updated through the ClamAV global mirror network.  If we wanted to separate out “policy” type signatures from the daily.cvd into their own cvd, we can now do that.

Third, while we have not removed some of the older signature formats, we did convert those older signatures to the newer formats to empty those older “cvd”s out.

For example:
“db" signatures were consolidated into “ndb" signatures
“zmd" and “rmd"  archive signatures we moved to the “cdb" container signature format

These formats are not new, they simply have never been published before. This includes other formats such as “hsb", “msb", “sfp", and “crb".  The older formats are supported for now, we are simply no longer publishing them.

Fourth, newer features, like the ability to write signatures based on the SHA256 of a file have been added to the system, and we can now publish that type of detection.

We’d like to thank you for your patience.

Wednesday, March 9, 2016

ClamAV added support for a new family, or perhaps old family of documents, in the 0.99.1 release: Hangul Word Processor (HWP) documents. HWP documents are document files specialized in the Korean language and developed by Hancom Inc.. The documents’ specialization in Korean make it a highly popular format used in South Korea with the government being a notable example. As popular format, it is subjected to possibly malicious content.

For this release, we primarily targeted the word-specific documents: HWP 2.x, HWP 3.x, HWP 5.x, and HWPX. The other file formats developed by Hancom Inc. which cover the spreadsheet and presentation formats are already handled by pre-existing methods in ClamAV; the only exception is HPT (an old presentation format) which will not be in this release.
  • HWP 2.x, also known as HWPML
    • XML-based document format similar to Microsoft’s older XML document format
    • Contents of the document is stored in the XML including all embedded content
    •  Embedded content is usually base64-encoded and normally uses zlib compression
    • General embedded content is stored in OLE2 containers
    • File  property collection: document’s attributes and metadata fields
  • HWP 3.x, also known as HWP
    • Custom binary file format. For additional information on the format, the documentation can be retrieved from Hancom’s website (note that it’s in Korean)
    • Contents of HWP 3.x are stored in a file segment that uses optional password encryption and normally uses zlib compression
    • Embedded content is stored in the content stream with general embedded content stored in HWP-styled* OLE2 containers
    • File property collection: data from various file headers
  • HWP 5.x, also known as HWP
    • OLE2-based document format similar to Microsoft’s 97-2003 document formats
    • Contents of HWP 5.x are stored in individual streams with zlib compression normally used on specific content streams including embedded content
    • Embedded content is stored in individual streams under the BinData directory with general embedded content stored as HWP-style* OLE2 containers
    • File property collection: data from the fileheader stream which appears to be the HWP legacy header and the /x005HwpSummaryInformation stream which uses the same property method as 97-2003 Microsoft documents
  • HWPX
    • OOXML-compliant document format
    • Contents of the document are stored in XML documents within a ZIP archive
    • Embedded content is stored in the BinData directory with general embedded content stored as HWP-style* OLE2 containers
    • File property collection: data from the content.hpf document which is an xml document with the legacy file header.

*Hwp-styled OLE2 container are identical to normal OLE2 container with the exception that a 32-bit value is prepended to file/stream/data segment