Friday, July 1, 2022

ClamAV is releasing an update today that completely overhauls the ClamAV Bytecode Compiler (ClamBC-Compiler). 

A bytecode signature is a form of cross-platform ClamAV plugin. They are distributed and loaded alongside the other ClamAV databases. These signatures are executed as supplemental parsers for specific files, or are triggered by traditional ClamAV signature matches to provide advanced analysis before providing a scan verdict. These serve as a mechanism for writing advanced detection rules and executable unpackers without having to wait to integrate and publish the logic in a full ClamAV feature release. And, more importantly, that means such features can be rapidly upgraded or disabled should a problem arise, without the need to publish a ClamAV critical patch version. They provide an option for the Cisco Talos Malware Research Team to deliver malware detection capabilities that would not otherwise be possible with ClamAV's content-match, hash, and file metadata-based signatures. 

This release, version 0.105.0, mirrors that of the latest ClamAV release. ClamBC-Compiler 0.105.0 carries the source code headers from ClamAV 0.105.0 necessary to build bytecode signatures with all the latest features and fixes found in in ClamAV 0.105.0 and prior.  

But the real milestone here is a celebration of a major ClamBC-Compiler architecture change. In prior versions, the compiler project vendored a sizeable portion of the LLVM project source code, modified so it can build ClamAV bytecode signatures from C (-ish) source. As the LLVM project rapidly evolved, this design made it more difficult to upgrade the ClamBC-Compiler project. Now, ClamBC-Compiler sports a new command-line interface and leverages OS distribution-provided LLVM libraries and the Clang compiler alongside our custom compiler modules. 

This upgrade also includes several improvements to ClamBC-Compiler code quality, quality assurance tests, and test infrastructure. We also fixed a handful of existing compiler bugs.  

ClamBC-Compiler is available as source code on GitHub.com/Cisco-Talos/clamav-bytecode-compiler and as a ready-to-go container on Docker Hub under clamav/clambc-compiler. We look forward to leveraging this new release to deliver new and improved malware detection content.

 

ClamBC-Compiler 0.105.0 Release Notes 

➕ Complete overhaul of the bytecode compiler project. 

  • The clamav compiler passes now build against a system-installed LLVM library. This builds a single shared library (i.e., libclambcc.so).  
  • The compiler application is now a Python script that uses Clang with the bytecode compiler library to run the compiler passes. 

🐛 Fixed bug causing signatures with malloc to fail to run when using system installed LLVM. 

🐛 Fixed the peinfo.c example bytecode source. 

❌ Removed vendored/modified LLVM/Clang 2.7-ish source. 

🌌 Upgraded build system from Autotools to CMake. 

🌌 New Requirements:  

  • LLVM 8  
  • Clang 8  
  • Python 3.6+ 

➕ Support for compiling bytecode signatures from multiple source file. 

🐛 Many assorted bug fixes. 


Monday, May 9, 2022



ClamAV has come a long way from being the reason a small Polish college was the target of a distributed denial-of-service attack. Today, 20 years after our first release, we’re proud that ClamAV is instead known for preventing these types of attacks on thousands of devices around the world. 

After many releases, updates, bugs and late nights from our developers, ClamAV is proud to celebrate its 20th anniversary this week. We’re celebrating by inching closer to the long-anticipated 1.0 release and continuing to test our release candidate for 0.105.0. 

Tomasz Kojm, the original creator of ClamAV, released the first 0.10 version of the open-source anti-virus engine on May 8, 2002, with the goal of making the internet safer at large and helping users filter out spam from their email inboxes. At the time, he was hosting the virus databases on his college and previous high school’s servers in Poland. Unfortunately, this led to a DoS condition on these pages because it was so popular, which Kojm told us via email he wishes he could have avoided. But if nothing else, this at least was an important lesson for Kojm and his co-creators to build off. 

Wednesday, May 4, 2022

The ClamAV 0.105.0 feature release is now stable and available for download on ClamAV.net or through Docker Hub.

Today, we're also publishing the 0.104.3 and 0.103.6 security patch versions, including several CVE fixes. These fixes have also been included in the 0.105 feature release. The source package and installer packages for these versions can be found under the "Previous Stable Releases" section of our Downloads page.

As a reminder, the 0.105 and 0.104 release files now include the following install packages:

  • x86_64 and i686 RPM packages are compatible with RPM-based Linux distributions running glibc version 2.17 or newer.

  • x86_64 and i686 DEB packages are compatible with Debian-based Linux distributions running glibc version 2.23 or newer.

  • An x86_64/ARM64 macOS installer package is compatible with Intel and Apple M1 systems.

  • x64 and win32 Windows packages are compatible with Windows 7 and newer.

Keep reading to find out what is in each version. 

Tuesday, April 5, 2022

Today, we are publishing a second release candidate for ClamAV 0.105.0.

Please help us validate this release by providing feedback via the ClamAV mailing list or on our Discord.

This release candidate phase is expected to last for another four weeks before the 0.105.0 Stable version is published. Please take this opportunity to verify that 0.105.0 can build and run in your environment. 

Here's a rundown of what's changed since our first release candidate for 0.105.0:

  • Fixed: Yara rules containing regex strings will load once more. This worked in previous ClamAV releases but was accidentally broken during 0.105 development.
  • Fixed: In the Docker image, ClamD will now listen for IPv4 and IPv6 connections. Previousl,y it was hard-coded to listen on only 0.0.0.0 which made connections fail on some systems that prefer IPv6.
  • Fixed: Assorted code quality issues.
  • Added: A SigTool --fuzzy-img option to generate the image fuzzy hash. With this feature, analysts will not need to rely on ClamScan's debug output or the generated metadata JSON temp-file in order to generate hashes for new signatures.

Please submit bug reports to the ClamAV project GitHub Issues.  

Refer to the initial 0.105.0 release candidate announcement blog post for the full list of improvements and changes in ClamAV 0.105.0.

Monday, March 14, 2022

We are excited to announce the ClamAV 0.105.0 release candidate.

Please help us validate this release by providing feedback via the ClamAV mailing list or on our Discord.

This release candidate phase is only expected to last about two to four weeks before the 0.105.0 Stable version will be published. Take this opportunity to verify that 0.105.0 can build and run in your environment. 

There is one known issue:

  • Yara rules containing regex strings will fail to load. The fix for this issue will be in the final release or next release candidate.

Please submit bug reports to the ClamAV project GitHub Issues.  

ClamAV 0.105.0 includes the following improvements and changes.

New Requirements

  • Starting with ClamAV v0.105, the Rust toolchain is required to compile ClamAV.

    You can install the Rust toolchain for your development environment by following the instructions on the rustup website. Some binary package distributions provide up-to-date packages of the Rust toolchain, but many do not. Using rustup ensures that you have the most up-to-date Rust compiler at the time of installation. Keep your toolchain updated for new features and bug/security fixes by periodically executing:

    rustup update 

    Building ClamAV requires, at a minimum, Rust compiler version 1.56, as it relies on features introduced in the Rust 2021 Edition.

    ClamAV's third-party Rust library dependencies are vendored into the release tarball (clamav-<version>.tar.gz) file that we publish on clamav.net/downloads. But, if you build from a Git clone or an unofficial tarball taken from GitHub.com, you will need the internet to download the Rust libraries during the build.

Major changes

  • Increased the default limits for file-size and scan-size:

    • MaxScanSize: 100M -> 400M
    • MaxFileSize: 25M -> 100M
    • StreamMaxLength: 25M -> 100M
    • PCREMaxFileSize: 25M -> 100M
    • MaxEmbeddedPE: 10M -> 40M
    • MaxHTMLNormalize: 10M -> 40M
    • MaxScriptNormalize: 5M -> 20M
    • MaxHTMLNoTags: 2M -> 8M

  • Added image fuzzy hash sub-signatures for logical signatures.

    Image fuzzy hash sub-signatures are a new feature for detecting images known to be used in phishing campaigns or otherwise used when distributing malware.

    Image fuzzy hash sub-signatures follow this format:

    fuzzy_img#<hash>

    For example:

    logo.png;Engine:150-255,Target:0;0;fuzzy_img#af2ad01ed42993c7
    logo.png-2;Engine:150-255,Target:0;0&1;49484452;fuzzy_img#af2ad01ed42993c7

    This initial implementation does not support matching with a hamming distance. Support for matching with a hamming distance may be added in a future release.

    ClamAV's image fuzzy hash is very close to, but not 100% identical to, the fuzzy hash generated by the Python imagehash package's phash() function. Note that these are only clean-room approximations of the pHash™️ algorithm. ClamAV's image fuzzy hashes are not expected to match the fuzzy hashes generated using other tools. Some images may match, while others do not.

    To generate the image fuzzy hash you can run this command:

    clamscan --gen-json --debug /path/to/file

    The hash will appear in the JSON above the "SCAN SUMMARY" under the object named "ImageFuzzyHash".

  • ClamScan & ClamDScan (Windows only):

    • Added a process memory scanning feature from ClamWin's ClamScan.

      This adds three new options to ClamScan and ClamDScan on Windows:

      • --memory
      • --kill
      • --unload

      Special thanks to:

      • Gianluigi Tiesi for allowing us to integrate the Windows process memory scanning feature from ClamWin into the ClamAV.
      • Grace Kang for integrating the ClamScan feature, and for extending it to work with ClamDScan in addition.

Notable changes

  • Updated the LLVM bytecode runtime support so that it can use LLVM versions 8 through 12 and removed support for earlier LLVM versions. Using LLVM JIT for the bytecode runtime may improve scan performance over the built-in bytecode interpreter runtime, which is the default. If you wish to build using LLVM, you must obtain a complete build of the LLVM libraries including the development headers and static libraries.

    There are some known issues both compiling and running the test suite with some LLVM installations. We are working to further stabilize LLVM bytecode runtime support, and document specific edge cases. Your feedback is welcome.

    For details about building ClamAV with the LLVM bytecode runtime, see the install reference documentation.

  • Added a GenerateMetadataJson option to ClamD. The functionality is equivalent to the clamscan --gen-json option. Scan metadata is useful for file analysis and for debugging scan behavior. If Debug is enabled, ClamD will print out the JSON after each scan. If LeaveTemporaryFiles is enabled, ClamD will drop a metadata.json file in the scan-temp directory. You can customize the scan-temp directory path using the TemporaryDirectory option.

  • The libclamunrar.so library's SO versions now matches that of libclamav.so. The upstream UnRAR library does not have an SO version that we should match. This change is to prevent a possible collision when multiple ClamAV versions are installed.

  • CMake: Added support for using an external TomsFastMath library (libtfm).

    To use an external TomsFastMath library, configure the build with the new option -D ENABLE_EXTERNAL_TOMSFASTMATH=ON. The following CMake variables may also be set as needed:

    • -D TomsFastMath_INCLUDE_DIR=<path> - The directory containing tfm.h.
    • -D TomsFastMath_LIBRARY=<path> - The path to the TomsFastMath library.

    Also updated the vendored TomsFastMath code to version 0.13.1.


Other improvements

  • Freshclam:

    • Improve ReceiveTimeout behavior so that will abort a download attempt if the download is not making significant progress. Previously this limit was an absolute time limit for the download and could abort prematurely for those on a slower connection. Special thanks to Simon Arlott for this improvement.
  • Rewrote the ClamAV database archive incremental-update feature (CDIFF) from scratch in Rust. The new implementation was our first module to be rewritten in Rust. It is significantly faster at applying updates that remove large numbers of signatures from a database, such as when migrating signatures from daily.cvd to main.cvd.

  • Freshclam & ClamD:

    • Increased the maximum line-length for freshclam.conf and clamd.conf from 512-characters to 1024-characters. This change was by request to accommodate very long DatabaseMirror options when using access tokens in the URI.
  • Removed the Heuristics.PNG.CVE-2010-1205 detection. This alert had been placed behind the --alert-broken-media (SCAN_HEURISTIC_BROKEN_MEDIA) option in 0.103.3 and 0.104 because of excessive alerts on slightly malformed but non- malicious files. Now it is completely removed.

  • Added support for building ClamDTop using ncursesw if ncurses can not be found. Patch courtesy of Carlos Velasco.

Bug fixes

  • ClamOnAcc: Fixed a number of assorted stability issues and added niceties for debugging ClamOnAcc. Patches courtesy of Frank Fegert.

  • Fixed an issue causing byte-compare sub-signatures to cause an alert when they match even if other conditions of the given logical signatures were not met.

  • Fixed an issue causing XLM macro false positives when scanning XLS documents containing images if the --alert-macros (AlertOLE2Macros) option was enabled.

  • Fixed an issue preventing multiple matches when scanning in all-match mode.

  • Docker:

    • Fixed an issue exposing the health check port. Patch courtesy of Sammy Chu.
    • Fixed an issue with health check failure false positives during container startup. Patch courtesy of Olliver Schinagl.
    • Set the default time zone to Etc/UTC. The --env parameter can be used to customize the time zone by setting TZ the environment variables. Patch courtesy of Olliver Schinagl.
  • Added support for detecting the curses library dependency even when the associated pkg-config file is not present. This resolves a build issue on some BSD distributions. Patch courtesy of Stuart Henderson.

  • Assorted bug fixes and improvements.

Acknowledgments

The ClamAV team thanks the following individuals for their code submissions:

  • Ahmon Dancy
  • Alexander Sulfrian
  • Carlos Velasco
  • Bernd Kuhls
  • David Korczynski
  • Fabrice Fontaine
  • Frank Fegert
  • Gianluigi Tiesi
  • Giovanni Bechis
  • Grace Kang
  • John Humlick
  • Jordan Ernst
  • JunWei Song
  • mko-x
  • Olliver Schinagl
  • Sammy Chu
  • Sergey Valentey
  • Simon Arlott
  • Stuart Henderson
  • Yann E. Morin

Wednesday, January 12, 2022

ClamAV versions 0.103.5 and 0.104.2 are now available for download on the clamav.net Downloads page

We would also like to take this opportunity to remind users that versions 0.102 and 0.101 have reached their end-of-life period. These versions exceeded our EOL dates on Jan. 3, 2022 and will soon be actively blocked from downloading signature database updates.

For additional details about ClamAV's end-of-life policy, please see our online documentation.

Friday, December 17, 2021

In early 2021, we made a decision to switch to using GitHub directly. Before then, our primary Git repository was sequestered to the Cisco corporate network. The public GitHub repository was just a mirror. That made it tougher for us to connect to and work with contributors from the ClamAV community. Our daily routines drew our attention away from GitHub.

At the same time, we also settled on a plan for how we could test using GitHub Issues for ClamAV bug reports. Initially, this would supplement our aging Bugzilla server. While Bugzilla requires users to make an account on our server, enabling GitHub Issues would open up bug reporting to the wider GitHub community that already has an account.

Since enabling GitHub Issues at the start of June, we have found that using GitHub Issues has been a very positive experience. At this point, most of our reports come in through GitHub Issues. With the aid of issue templates, the quality of bug reports coming through GitHub Issues has also been of a higher caliber than those coming through Bugzilla. With users following guidance in the template, the bug reports are generally easier to triage and respond to intelligently. 

All that being said, we will disable Bugzilla ticket creation so we can focus all new bug reports towards GitHub Issues. Effective immediately, the new ticket creation form on bugzilla.clamav.net will redirect users to the new issue creation page on our primary ClamAV repository on GitHub.com.

Existing open tickets on Bugzilla will remain open for now, and the Bugzilla server is expected to remain available through 2022 as we complete this transition. 

If you have an open ticket on Bugzilla and would like to help migrate it to GitHub Issues, we would appreciate the help. If you choose to migrate any tickets from Bugzilla to GitHub Issues, please add references in each ticket and close the original Bugzilla ticket(s) to assist our team with the ticket queue maintenance.


Thank you!

Micah

Tuesday, December 14, 2021

As a reminder, the ClamAV 0.102 and 0.101 releases will reach its end of life (EOL) in less than one month's time on Jan. 3, 2022. After this date, ClamAV 0.102, and 0.101 and all patch versions will be blocked from downloading new signatures from database.clamav.net. 

For information about the EOL policy, visit the ClamAV online documentation.

Wednesday, November 3, 2021

ClamAV 0.103.4 LTS and 0.104.1 patch versions are out now. Both of these can be found on clamav.net/downloads, with 0.104.1 as the main release and 0.103.4 under "Previous Stable Releases."


0.103.4

ClamAV 0.103.4 is a critical patch release with the following fixes:

  • FreshClam:

    • Add a 24-hour cool-down for FreshClam clients that have received an HTTP 403 (Forbidden) response from the CDN. This is to reduce the volume of 403-response data served to blocked FreshClam clients that are configured with a tight update-loop.
    • Fixed a bug where FreshClam treats an empty CDIFF as an incremental update failure instead of as an intentional request to download the whole CVD.
  • ClamDScan: Fix a scan error when broken symlinks are encountered on macOS with "FollowDirectorySymlinks" and "FollowFileSymlinks" options disabled.

  • Overhauled the scan recursion / nested archive extraction logic and added new limits on embedded file-type recognition performed during the "raw" scan of each file. This limits embedded file-type misidentification and prevents detecting embedded file content that is found/extracted and scanned at other layers in the scanning process.

  • Fix an issue with the FMap module that failed to read from some nested files.

  • Fixed an issue where failing to load some rules from a Yara file containing multiple rules may cause a crash.

  • Fixed assorted compiler warnings.

  • Fixed assorted Coverity static code analysis issues.

  • Scan limits:

    • Added virus-name suffixes to the alerts that trigger when a scan limit has been exceeded. Rather than simply Heuristics.Limits.Exceeded, you may now see limit-specific virus-names, to include:
      • Heuristics.Limits.Exceeded.MaxFileSize
      • Heuristics.Limits.Exceeded.MaxScanSize
      • Heuristics.Limits.Exceeded.MaxFiles
      • Heuristics.Limits.Exceeded.MaxRecursion
      • Heuristics.Limits.Exceeded.MaxScanTime
    • Renamed the Heuristics.Email.ExceedsMax.* alerts to align with the other limit alerts names. These alerts include:
      • Heuristics.Limits.Exceeded.EmailLineFoldcnt
      • Heuristics.Limits.Exceeded.EmailHeaderBytes
      • Heuristics.Limits.Exceeded.EmailHeaders
      • Heuristics.Limits.Exceeded.EmailMIMEPartsPerMessage
      • Heuristics.Limits.Exceeded.EmailMIMEArguments
    • Fixed an issue where the Email-related scan limits would alert even when the "AlertExceedsMax" (--alert-exceeds-max) scan option is not enabled.
    • Fixes an issue in the Zip parser where exceeding the "MaxFiles" limit or the "MaxFileSize" limit would abort the scan but would fail to alert. The Zip scan limit issues were independently identified and reported by Aaron Leliaert and Max Allan.
  • Fixed a leak in the Email parser when using the --gen-json scan option.

  • Fixed an issue where a failure to record metadata in the Email parser when using the --gen-json scan option could cause the Email parser to abort the scan early and fail to extract and scan additional content.

  • Fixed a file name memory leak in the Zip parser.

  • Fixed an issue where certain signature patterns may cause a crash or cause unintended matches on some systems when converting characters to uppercase if a UTF-8 unicode single-byte grapheme becomes a multi-byte grapheme. Patch courtesy of Andrea De Pasquale.

Other fixes backported from 0.104.0:

  • Fixed a crash in programs that use libclamav when the programs don't set a callback for the "virus found" event. Patch courtesy of Markus Strehle.

  • Added checks to the the SIS archive parser to prevent an SIS file entry from pointing to the archive, which would result in a loop. This was not an actual infinite loop, as ClamAV's scan recursion limit limits the depth of nested archive extraction.

  • ClamOnAcc: Fixed a socket file descriptor leak that could result in a crash when all available file descriptors are exhausted.

  • FreshClam: Fixed an issue where FreshClam would download a CVD repeatedly if a zero-byte CDIFF is downloaded or if the incremental update failed and if the CVD downloaded after that is older than advertised. Patch courtesy of Andrew Williams.

  • ClamDScan:

    • Fixed a memory leak of the scan target filename when using the --fdpass or --stream options.
    • Fixed an issue where ClamDScan would fail to scan any file after excluding a file with the "ExcludePath" option when using when using the --multiscan (-m) option along with either --fdpass or --stream. Also fixed a memory leak of the accidentally-excluded paths in this case.
    • Fixed a single file path memory leak when using --fdpass.
    • Fixed an issue where the "ExcludePath" regex may fail to exclude absolute paths when the scan is invoked with a relative path.

Special thanks to the following for code contributions and bug reports:

  • Aaron Leliaert
  • Andrea De Pasquale
  • Andrew Williams
  • Markus Strehle
  • Max Allan

 

0.104.1

ClamAV 0.104.1 is a critical patch release with the following fixes:

  • FreshClam:

    • Add a 24-hour cool-down for FreshClam clients that have received an HTTP 403 (Forbidden) response from the CDN. This is to reduce the volume of 403-response data served to blocked FreshClam clients that are configured with a tight update-loop.
    • Fixed a bug where FreshClam treats an empty CDIFF as an incremental update failure instead of as an intentional request to download the whole CVD.
  • ClamDScan: Fix a scan error when broken symlinks are encountered on macOS with "FollowDirectorySymlinks" and "FollowFileSymlinks" options disabled.

  • Overhauled the scan recursion / nested archive extraction logic and added new limits on embedded file-type recognition performed during the "raw" scan of each file. This limits embedded file-type misidentification and prevents detecting embedded file content that is found/extracted and scanned at other layers in the scanning process.

  • Fix an issue with the FMap module that failed to read from some nested files.

  • Fixed an issue where failing to load some rules from a Yara file containing multiple rules may cause a crash.

  • Fixed assorted compiler warnings.

  • Fixed assorted Coverity static code analysis issues.

  • Scan limits:

    • Added virus-name suffixes to the alerts that trigger when a scan limit has been exceeded. Rather than simply Heuristics.Limits.Exceeded, you may now see limit-specific virus-names, to include:
      • Heuristics.Limits.Exceeded.MaxFileSize
      • Heuristics.Limits.Exceeded.MaxScanSize
      • Heuristics.Limits.Exceeded.MaxFiles
      • Heuristics.Limits.Exceeded.MaxRecursion
      • Heuristics.Limits.Exceeded.MaxScanTime
    • Renamed the Heuristics.Email.ExceedsMax.* alerts to align with the other limit alerts names. These alerts include:
      • Heuristics.Limits.Exceeded.EmailLineFoldcnt
      • Heuristics.Limits.Exceeded.EmailHeaderBytes
      • Heuristics.Limits.Exceeded.EmailHeaders
      • Heuristics.Limits.Exceeded.EmailMIMEPartsPerMessage
      • Heuristics.Limits.Exceeded.EmailMIMEArguments
    • Fixed an issue where the Email-related scan limits would alert even when the "AlertExceedsMax" (--alert-exceeds-max) scan option is not enabled.
    • Fixes an issue in the Zip parser where exceeding the "MaxFiles" limit or the "MaxFileSize" limit would abort the scan but would fail to alert. The Zip scan limit issues were independently identified and reported by Aaron Leliaert and Max Allan.
  • Fixed a leak in the Email parser when using the --gen-json scan option.

  • Fixed an issue where a failure to record metadata in the Email parser when using the --gen-json scan option could cause the Email parser to abort the scan early and fail to extract and scan additional content.

  • Fixed a file name memory leak in the Zip parser.

  • Fixed an issue where certain signature patterns may cause a crash or cause unintended matches on some systems when converting characters to uppercase if a UTF-8 unicode single-byte grapheme becomes a multi-byte grapheme. Patch courtesy of Andrea De Pasquale.

  • CMake:

    • Fix a packaging issue with the Windows *.msi installer so that it will include all of the required files.
    • Add support for developer code-signing on macOS during the build.
    • Fix an issue finding and linking with the tinfo library on systems where tinfo is separate from ncurses. Patch courtesy of Luca Barbato.
  • Tests: Improved the Freshclam incremental update tests to verify correct behavior when a zero-byte CDIFF is downloaded and the CVD served to FreshClam is older than advertised.

  • Docker: Remove the freshclam.dat file when building the Docker image with the databases-included so FreshClam agents running in the container will have a unique ID in the HTTP User-Agent.

Special thanks to the following for code contributions and bug reports:

  • Aaron Leliaert
  • Andrea De Pasquale
  • Luca Barbato
  • Max Allan