Showing posts with label beta. Show all posts
Showing posts with label beta. Show all posts

Monday, March 31, 2025

The ClamAV 1.5.0 beta is now available. You may find the source code and installers for this release at clamav.net/downloads or on the ClamAV GitHub release page.

The beta phase is expected to last two to four weeks before we publish the stable release or else publish a release candidate. This will depend on how many changes are required to stabilize this version. Please take this time to evaluate ClamAV 1.5.0.

Please help us validate this release by providing feedback via GitHub issues, via the ClamAV mailing list or on our Discord.

IMPORTANT: A major feature of the 1.5 release is a FIPS-compliant method for verifying the authenticity of CVD signature database archives and CDIFF signature database patch files. The feature is ready to test in this beta, but we are not yet distributing the associated “.cvd.sign” signature files for the daily, main, and bytecode databases. Because these files aren’t available, ClamAV will fall back to using the legacy MD5-based RSA signature check. In other words, Freshclam will continue to fail on FIPS-enabled systems, for now. However, the unit tests do include a test signing key and certificate pair along with tests to exercise signing and verification using the FIPS-compliant method.

Note: Windows builds on GitHub Actions, which use VCPkg to provide C library dependencies, are failing at this time.

Tip: If you are downloading the source from the GitHub release page, the package labeled "clamav-1.5.0-beta.tar.gz" does not require an internet connection to build. All dependencies are included in this package. However, if you download the ZIP or TAR.GZ generated by GitHub, located at the very bottom, then an internet connection will be required during the build to download additional Rust dependencies.

For Docker users, there is no specific Docker tag for the beta, but you can use the clamav:unstable or clamav:unstable_base tags.

ClamAV 1.5.0 includes the following improvements and changes:

Major changes

  • Added checks to determine if an OLE2-based Microsoft Office document is encrypted.
  • Added the ability to record URLs found in HTML if the generate-JSON-metadata feature is enabled. Also adds an option to disable this in case you want the JSON metadata feature but don't want to record HTML URL's. The ClamScan command-line option is “--json-store-html-urls=no”. The clamd.conf config option is “JsonStoreHTMLUrls no”.
  • Added regex support for the clamd.conf OnAccessExcludePath config option. This change courtesy of GitHub user b1tg.
  • Added FIPS-compliant CVD signing/verification with external .sign files.
    Freshclam will now attempt to download external signature files to accompany existing .cvd databases and .cdiff patch files. Sigtool now has commands to sign and verify using the external signatures.

    ClamAV now installs a 'certs' directory in the app config directory (e.g. <prefix>/etc/certs). The install path is configurable. The CMake option to configure the CVD certs directory is “-D CVD_CERTS_DIRECTORY=PATH”. 

    Added sign/verify commands to Sigtool.

    New options to set an alternative CVD certs directory:
    - The command-line option for Freshclam, ClamD, ClamScan, and Sigtool is: --cvdcertsdir PATH
    - The environment variable for Freshclam, ClamD, ClamScan, and Sigtool is: CVD_CERTS_DIR

    The config option for Freshclam and ClamD is: CVDCertsDirectory PATH

    Added two new APIs to the public clamav.h header:
    extern cl_error_t cl_cvdverify_ex(const char *file, 
                                      const char *certs_directory);
    extern cl_error_t cl_cvdunpack_ex(const char *file, 
                                      const char *dir, 
                                      bool dont_verify, 
                                      const char *certs_directory); 


    The original cl_cvdverify and cl_cvdunpack are deprecated. 

    Added a cl_engine_field enum option CL_ENGINE_CVDCERTSDIR. You may set this option with cl_engine_set_str and get it with cl_engine_get_str to override the compiled in default CVD certs directory.

    Thank you to Mark Carey at SAP for inspiring work on this feature with an initial proof of concept for external-signature FIPS compliant CVD signing. 

Other improvements

  • Set a limit on the max-recursion config option. Users will no longer be able to set max-recursion higher than 100. This change prevents errors on start up or possible crashes if encountering a file with that many layers of recursion.
  • Build system: CMake improvements to support compiling for the AIX platform. This change is courtesy of GitHub user KamathForAIX.
  • Improve support for extracting malformed .zip archives. This change is courtesy of Frederick Sell.
  • Windows: Code quality improvement for the ClamScan and ClamDScan --move and --remove options. This change is the courtesy of Maxim Suhanov.
  • Added file type recognition for some kinds of AI model files.

    The file type appears as a string parameter for these callback functions:
      - clcb_pre_cache
      - clcb_pre_scan
      - clcb_file_inspection

    When scanning these files, the `type` parameter will now show "CL_TYPE_AI_MODEL" instead of "CL_TYPE_BINARY_DATA".

Bug fixes

  • Technical debt: Reduced email multipart message parser complexity.
  • Fixed possible undefined behavior in inflate64 module. The inflate64 module is a modified version of the zlib library, taken from version 1.2.3 with some customization and with some cherry-picked fixes. This adds one additional fix from zlib 1.2.9. Thank you to TITAN Team for reporting this issue.
  • Fixed a bug in ClamD that broke reporting of memory usage on Linux. The STATS command can be used to monitor ClamD directly or through ClamDTOP. The memory stats feature does not work on all platforms (e.g. Windows).
  • Windows: Fix a build issue when the same library dependency is found in two different locations.
  • Fix an infinite loop when scanning some email files in debug-mode. This fix is courtesy of Yoann Lecuyer

Acknowledgments

Special thanks to the following people for code contributions and bug reports:

  • b1tg
  • Frederick Sell
  • KamathForAIX
  • Mark Carey at SAP
  • Maxim Suhanov
  • TITAN Team
  • Yoann Lecuyer


Wednesday, January 24, 2024

We are excited to announce the ClamAV 1.3.0 release candidate.

You can find the source code and installers for this release on the clamav.net/downloads page or the ClamAV GitHub release page.

Tip: If you are downloading the source from the GitHub release page, the package labeled "clamav-1.3.0-rc2.tar.gz" does not require an internet connection to build. All dependencies are included in this package. But if you download the ZIP or TAR.GZ generated by GitHub, located at the very bottom, then an internet connection will be required during the build to download additional Rust dependencies.

For Docker users, there is no specific Docker tag for the release candidate, but you can use these tags:

  • clamav/clamav:unstable
  • clamav/clamav:unstable_base 
  • clamav/clamav-debian:unstable
  • clamav/clamav-debian:unstable_base 

This release candidate phase is expected to last two weeks before we publish the stable release. Please take this time to evaluate ClamAV 1.3.0.  

Please help us validate this release by providing feedback via the ClamAV mailing list or our Discord.

Changes since the first release candidate include:

  • ClamOnAcc: Fixed an infinite loop when a file has been deleted before a scan.
    Patch courtesy of gsuehiro.
  • ClamConf: Fixed an issue printing `MaxScanSize` introduced with the change to allow a `MaxScanSize` greater than 4 GB.
    Fix courtesy of teoberi.
  • Fixed an issue building a ClamAV RPM in some configurations.
    The issue was caused by faulty CMake logic that intended to create an empty database directory during the installation.

You can find the full list of changes and improvements in the blog post for the first release candidate.

Monday, August 5, 2019

We are pleased to introduce the ClamAV 0.101.3 security patch release and a beta for the upcoming 0.102 feature release.

Both of these can be found on ClamAV's downloads site, with 0.101.3 in the "latest stable release" section and 0.102.0-beta in the beta section.

0.101.3

ClamAV 0.101.3 is a patch release to address a vulnerability to non-recursive zip bombs.

A Denial-of-Service (DoS) vulnerability may occur when scanning a zip bomb as a result of excessively long scan times. The issue is resolved by detecting the overlapping local file headers which characterize the non-recursive zip bomb described by David Fifield.

Thank you to Hanno Böck for reporting the issue as it relates to ClamAV, here.

Also included in 0.101.3:
  • Update of bundled the libmspack library from 0.8alpha to 0.10alpha, to address a buffer overflow vulnerability in libmspack < 0.9.1α.


0.102-beta

ClamAV 0.102.0 includes an assortment of improvements and a couple of significant changes.

Major changes

  • The On-Access Scanning feature has been migrated out of clamd and into a brand new utility named clamonacc. This utility is similar to clamdscan and clamav-milter in that it acts as a client to clamd. This separation from clamd means that clamd no longer needs to run with root privileges while scanning potentially malicious files. Instead, clamd may drop privileges to run under an account that does not have super-user. In addition to improving the security posture of running clamd with On-Access enabled, this update fixed a few outstanding defects:
    • On-Access scanning for created and moved files (Extra-Scanning) is fixed.
    • VirusEvent for On-Access scans is fixed.
    • With clamonacc, it is now possible to copy, move, or remove a file if the scan triggered an alert, just like with clamdscan. For details on how to use the new clamonacc On-Access scanner, please refer to the user manual on ClamAV.net, and keep an eye out for a new blog post on the topic
  • The freshclam database update utility has undergone a significant update. This includes:
    • Added support for HTTPS.
    • Support for database mirrors hosted on ports other than 80.
    • Removal of the mirror management feature (mirrors.dat).
    • An all new libfreshclam library API.

Notable changes

  • Added support for extracting ESTsoft .egg archives. This feature is new code developed from scratch using ESTsoft's Egg-archive specification and without referencing the UnEgg library provided by ESTsoft. This was necessary because the UnEgg library's license includes restrictions limiting the commercial use of the UnEgg library.
  • The documentation has moved!
    • Users should navigate to ClamAV.net to view the documentation online.
    • The documentation will continue to be provided in HTML format with each release for offline viewing in the docs/html directory.
    • The new home for the documentation markdown is in our ClamAV FAQ Github repository.

Other improvements

  • Improved Windows executable Authenticode handling, enabling both whitelisting and blacklisting of files based on code-signing certificates. Additional improvements to Windows executable (PE file) parsing. Work courtesy of Andrew Williams.
  • Added support for creating bytecode signatures for Mach-O and ELF executable unpacking. Work courtesy of Jonas Zaddach.
  • Re-formatted the entire ClamAV code-base using clang-format in conjunction with our new ClamAV code style specification. See the clamav.net blog post for details.
  • Integrated ClamAV with Google's OSS-Fuzz automated fuzzing service with the help of Alex Gaynor. This work has already proven beneficial, enabling us to identify and fix subtle bugs in both legacy code and newly developed code.
  • The clamsubmit tool is now available on Windows.
  • The clamscan metadata feature (--gen-json) is now available on Windows.
  • Significantly reduced number of warnings generated when compiling ClamAV with "-Wall" and "-Wextra" compiler flags and made many subtle improvements to the consistency of variable types throughout the code.
  • Updated the majority of third-party dependencies for ClamAV on Windows. The source code for each has been removed from the clamav-devel repository. This means that these dependencies have to be compiled independently of ClamAV. The added build process complexity is offset by significantly reducing the difficulty of releasing ClamAV with newer versions of those dependencies.
  • During the 0.102 development period, we've also improved our Continuous Integration (CI) processes. Most recently, we added a CI pipeline definition to the ClamAV Git repository. This chains together our build and quality assurance test suites and enables automatic testing of all proposed changes to ClamAV, with customizable parameters to suit the testing needs of any given code change.

Bug fixes

  • Fix to prevent a possible crash when loading LDB type signature databases and PCRE is not available. Patch courtesy of Tomasz Kojm.
  • Fixes to the PDF parser that will improve PDF malware detection efficacy. Patch courtesy of Clement Lecigne.
  • Fix for regular expression phishing signatures (PDB R-type signatures).
  • Various other bug fixes.

New Requirements

  • Libcurl has become a hard-dependency. Libcurl enables HTTPS support for freshclam and clamsubmit as well as communication between clamonacc and clamd.
  • Libcurl version >= 7.45 is required when building ClamAV from source with the new On-Access Scanning application (clamonacc). Users on Linux operating systems that package older versions of libcurl (e.g. all versions of CentOS and Debian versions <= 8) have a number of options:
    • Wait for your package maintainer to provide a newer version of libcurl.
    • Install a newer version of libcurl from source.
    • Disable installation of clamonacc and On-Access Scanning capabilities with the ./configure flag --disable-clamonacc.

      Non-Linux users will need to take no actions as they are unaffected by this new requirement.

Acknowledgements

The ClamAV team thanks the following individuals for their code submissions:
  • Alex Gaynor
  • Andrew Williams
  • Carlo Landmeter
  • Chips
  • Clement Lecigne
  • Paul Arthur
  • Jonas Zaddach
  • Ørjan Malde
  • Rick Wang
  • Rosen Penev
  • Thomas Jarosch
  • Tomasz Kojm

Finally, we'd like to thank Joe McGrath for building our quality assurance test suite and for working diligently to ensure knowledge transfer up until his last day on the team. Working with you was a pleasure, Joe, and we wish you the best of luck in your next adventure!

Tuesday, October 30, 2018

Welcome to the ClamAV 0.101.0 beta!

Important notes about this release:


  • Changes to the libclamav API:
    • Those who build applications around our shared library will need to change how they declare and pass scanning options to libclamav. Please take a look at the change to our example code for details. 
    • Many of the scanning option #defines have changed. These can be found in our clamav.h header.
    • The libclamav version number has changed.
  • Some of the clamd config and clamscan command line option names have changed. The original versions will still work for a time, but eventually they will be deprecated. These options in question are detailed in the NEWS document.
  • A new sub-signature type called "Byte Compare". Byte Compare sub-signatures can be used to evaluate a numeric value at a given offset from the start of another (matched) sub-signature within the same logical signature. That numerical value may be interpreted from signed ascii decimal, unsigned ascii hex, or unsigned binary data relative to a match offset. Your participation in testing this feature during our beta period will be greatly appreciated. For details, see the signature writing documentation.
  • Changes to our documentation. Documentation is now in Markdown and should be easier to navigate and easier to contribute to.
  • Support for extraction/scanning of RAR v5.x archives.


For additional details on changes in 0.101, please read the notes in our NEWS document.

Thank you in advance for participating in the ClamAV 0.101 beta! Bugs should be brought to our attention via the clamav-devel mailing list or via bugzilla.

You may download the ClamAV 0.101.0 beta at the usual location on ClamAV.net.

Monday, February 5, 2018

ClamAV 0.100.0-beta is the successor to the previous 0.99.3-beta2.  The 0.99.3 patch release on January 25th was required to address vulnerability fixes in a timely manner, so the features previously found in 0.99.3 betas have been bumped to this new version.  If you haven’t read it, please read the announcement regarding the version number change. 

The 0.100.0-beta includes all of the feature improvements and bug fixes that were in the previous 0.99.3-beta2, plus some additional bug fixes and requested improvements that were found by users of the previous beta.  These include:

  • Eliminating warnings regarding with variables being used before being initialized. 
  • Correcting an issue for those using private mirrors where freshclam attempts to pull down the CVD file if the CLD is up-to-date. 
  • Fixed a bug in the filtering system that caused unexpected behavior for signatures that use the case insensitive signatures (:i). 
  • Increased the max stack size when building ClamAV for non-glibc Linux machines (i.e. musl). 
  • Deprecated the AllowSupplementaryGroups config option in a more graceful way.
  • Bug fixes to on-access scanning.
  • A few other bug fixes.


We could use community support testing these fixes, of course.  That said, our main goal of 0.100.0-beta is to get the community ready for the version string change.  Mirror maintainers have been asked to verify that ClamAV clients using the 0.100.0 version number in the HTTP user agent are not blacklisted by regexes intended to drop support for older versions of ClamAV.

As a disclaimer, 0.100.0-beta isn’t a release candidate because we have a few outstanding known issues that we must address prior to the 0.100.0 release, and because once the fixes are made we will have to complete regression testing.  The known issues blocking release include the following:

  • The libmspack library install location, name. Bug
  • BlockMax config option may differ slightly from —block-max command line option. Bug
  • Using the ./configure --disable-static will still require llvm-static. Bug
  • Improvements in PDF object parsing (in progress). 
  • Messages when clamscan skips a file due to max file size settings, along with corrections to the —help string. Bug
  • Warnings when building on macOS.  Bugs, Bugs 
Bugs should be brought to our attention via the clamav-devel mailing list or via bugzilla

Monday, December 18, 2017

Welcome to ClamAV 0.99.3's beta2 release. In this release, we have included many code
submissions from the ClamAV community:

  • Interfaces to the Prelude SIEM open source package for collecting ClamAV virus events.
  • Visual Studio 2015 for building Microsoft Windows binaries.
  • Support libmspack internal code or as a shared object library. The internal library is the default and contains additional integrity checks.
  • Linking with openssl 1.1.0.
  • Numerous code patches, typos, and compiler warning fixes.

Additionally, we have introduced important changes and new features in
ClamAV 0.99.3, including:

  • Deprecating internal LLVM code support. The configure script has change to search the system for an installed instance of the LLVM development libraries, and to otherwise use the bytecode interpreter for ClamAV bytecode signatures. To use the LLVM Just-In-Time compiler for executing bytecode signatures, please ensure that the LLVM development package at version 3.6 or lower is installed. Using the deprecated LLVM code is possible with the command: './configure --with-system-llvm=no', but it no longer compiles on all platforms.
  • Compute and check PE import table hash (a.k.a. "imphash") signatures.
  • Support file property collection and analysis for MHTML files.
  • Raw scanning of PostScript files.
  • Fix clamsubmit to use the new virus and false positive submission web interface.
  • Optionally, flag files with the virus "Heuristic.Limits.Exceeded" when size limitations are exceeded.
  • Improve decoders for PDF files.


The ClamAV community thanks the following individuals for their ClamAV 0.99.3
code submissions:

Sebastian Andrzej Siewior
Keith Jones
Bill Parker
Chris Miserva
Daniel J. Luke
Matthew Boedicker
Ningirsu
Michael Pelletier
Anthony Chan
Stephen Welker
Marc Deslauriers
Mark Allan
Andreas Schulze
Jonas Zaddach
Georgy Salnikov

We are releasing beta2 for further testing while we resolve our small list of known issues in the background as we are prepping for "General Availability".  If you have the ability to download and use beta2 on your network, please do.  Thanks!

Thursday, August 3, 2017

Join us as we welcome ClamAV 0.99.3 beta for testing!  Be sure and grab the beta release on our official ClamAV download site.

Welcome to ClamAV 0.99.3. In this release, we have included many code
submissions from the ClamAV community:

  • Interfaces to the Prelude SIEM open source package for collecting ClamAV virus events.
  • Visual Studio 2015 for building Microsoft Windows binaries.
  • Support libmspack internal code or as a shared object library. The internal library is the default and contains additional integrity checks.
  • Linking with openssl 1.1.0.
  • Numerous code patches, typos, and compiler warning fixes.


Additionally, we have introduced important changes and new features in
ClamAV 0.99.3, including:

  • Deprecating internal LLVM code support. The configure script has changed to search the system for an installed instance of the LLVM development libraries, and to otherwise use the bytecode interpreter for ClamAV bytecode signatures. To use the LLVM Just-In-Time compiler for executing bytecode signatures, please ensure that the LLVM development package at version 3.6 or lower is installed. Using the deprecated LLVM code is possible with the command: './configure --with-system-llvm=no', but it no longer compile on all platforms.
  • Compute and check PE import table hash (a.k.a. "imphash") signatures.
  • Support file property collection and analysis for MHTML files.
  • Raw scanning of PostScript files.
  • Fix clamsubmit to use the new virus and false positive submission web interface.
  • Optionally, flag files with the virus "Heuristic.Limits.Exceeded" when size limitations are exceeded.
  • Improve decoders for PDF files.


The ClamAV community thanks the following individuals for their ClamAV 0.99.3 code submissions:

Sebastian Andrzej Siewior
Keith Jones
Bill Parker
Chris Miserva
Daniel J. Luke
Matthew Boedicker
Ningirsu
Michael Pelletier
Anthony Chan
Stephen Welker

Following are issues discovered during release testing. For additional information, please review the corresponding tickets on bugzilla.clamav.net:

11879 - cli_scanmscan() Failed to extract 4 in Windows beta when scanning cab files
11882 - ./configure does not automatically detect libxml2 on FreeBSD 10.3 and 11.0
11884 - 'sudo make install' on FreeBSD 10.3 and 11.0 leaves files owned by root, subsequent make command fails
11885 - clamsubmit not building on FreeBSD 10.3 and 11.0
11887 - Failures of 'make check VG=1' on FreeBSD 10.3 and 11.0

We ask that feedback be provided via the ClamAV mailing lists.

Thursday, February 11, 2016

Join us as we welcome ClamAV 0.99.1 beta for testing!  Be sure and grab the latest beta release on our official ClamAV download site.  Remember, as we've stated for the past two releases, 0.99.1 will be the last release we place on SourceForge.net, so please transition to using ClamAV.net!

First, a couple release notes:

ClamAV 0.99.1 contains a new feature for parsing Hancom Office files
including extracting and scanning embedded objects. ClamAV 0.99.1
also contains important bug fixes.

Thanks to the following community members for code submissions used in
ClamAV 0.99.1:

Jim Morris
Andreas Cadhalpun

ChangeLog:

Tue, 2 Feb 2016 14:23:13 -0500 (Kevin Lin)
------------------------------------------
 * add scanning options for scanning xml-based documents
(MSXML, OOXML, HWPML) and HWP3

Fri, 29 Jan 2016 16:17:40 -0500 (Kevin Lin)
------------------------------------------
 * add dconfs for XDP, MBR, GPT, APM, OOXML, MSXML, and HWP formats

Tue, 2 Feb 2016 12:38:27 -0500 (Kevin Lin)
------------------------------------------
 * hwp: scan decompressed data on limits exceeded

Sun, 31 Jan 2016 15:41:54 -0500 (Steven Morgan)
------------------------------------------
 * Fix for signature name length<3 in .ign & .ign2 CVD files.

Fri, 29 Jan 2016 11:21:46 -0500 (Steven Morgan)
------------------------------------------
 * Change RTF file magic from '{\rtf' to '{\rt'

Wed, 27 Jan 2016 14:17:47 -0500 (Mickey Sola)
------------------------------------------
 * bb11444 - zeroing out buffer at allocation to avoid writing uninitialized
bytes to a file

Wed, 20 Jan 2016 16:21:58 -0500 (Mickey Sola)
------------------------------------------
 * bb11446 - adding check for compressed input stream

Wed, 13 Jan 2016 14:56:46 -0500 (Kevin Lin)
------------------------------------------
 * clean up and boost accuracy to detecting OOXML documents

Thu, 14 Jan 2016 17:01:01 -0500 (Steven Morgan)
------------------------------------------
 * 11477 - Fix crash when using pcre statistics due to inconsistent memory
management function calls

Thu, 14 Jan 2016 15:31:02 -0500 (Mickey Sola)
------------------------------------------
 * bb11444: adding check to verify a valid number of rounds for rijndael aes
encryption/decryption

Wed, 13 Jan 2016 13:18:01 -0500 (Mickey Sola)
------------------------------------------
 * bb11441 - fixing edge case revealed by unit testing

Tue, 12 Jan 2016 17:20:26 -0500 (Steven Morgan)
------------------------------------------
 * bb11466 - let html be scanned raw when --scan-html=no is in effect.

Tue, 12 Jan 2016 16:14:40 -0500 (Steven Morgan)
------------------------------------------
 * bb11475: fix crash due to memory overlay with crafted 7z files.

Tue, 12 Jan 2016 14:30:41 -0500 (Mickey Sola)
------------------------------------------
 * bb11441 - fixing off by one OOB write in htmlnorm

Mon, 11 Jan 2016 14:47:30 -0500 (Mickey Sola)
------------------------------------------
 * bb11446 - fixing autoit OOB bufferread

Wed, 6 Jan 2016 14:57:48 -0500 (Mickey Sola)
------------------------------------------
 * sigtool: adding support for decoding cdb sigs

Tue, 5 Jan 2016 13:13:47 -0500 (Steven Morgan)
------------------------------------------
 * bb11467 - fix embedded http links.

Tue, 22 Dec 2015 19:04:42 -0500 (Steven Morgan)
------------------------------------------
 * bb11451 - fix guard macro name collision on AIX.

Tue, 22 Dec 2015 13:36:51 -0500 (Steven Morgan)
------------------------------------------
 * fix regression crash on sid 1005597703 due to uninitialzed
(i.e., garbage initialized) structure.

Mon, 21 Dec 2015 17:39:47 -0500 (Steven Morgan)
------------------------------------------
 * bb11456 - corrects mso stream prefix value for big endian processors.
Patch by Jim Morris.

Thu, 17 Dec 2015 16:16:55 -0500 (Kevin Lin)
------------------------------------------
 * ooxml_hwp: add support for filetyping and preclassification

Thu, 17 Dec 2015 11:04:00 -0500 (Kevin Lin)
------------------------------------------
 * hwpole2: new filetype and handler for hwp embedded ole2 files

Thu, 17 Dec 2015 12:46:38 -0500 (Mickey Sola)
------------------------------------------
 * bb11443 - fixing possible oob dereference when parsing mbox files

Tue, 15 Dec 2015 16:12:01 -0500 (Mickey Sola)
------------------------------------------
 * bb11442 - adding explicit wwunpack oob checks

Tue, 15 Dec 2015 10:53:05 -0500 (Mickey Sola)
------------------------------------------
 * bb11445 - fixing edge case where a null terminator was unintentionally
written into an adjacent buffer.

Wed, 16 Dec 2015 16:13:05 -0500 (Kevin Lin)
------------------------------------------
 * hwpml: use msxml_parser callback scanner for binary data

Wed, 16 Dec 2015 16:12:20 -0500 (Kevin Lin)
------------------------------------------
 * msxml_parser: add callback-based scanning mechanism

Tue, 15 Dec 2015 13:01:40 -0500 (Kevin Lin)
------------------------------------------
 * HWPML: added hwpml_keys for hwpml parsing

Tue, 15 Dec 2015 10:54:20 -0500 (Kevin Lin)
------------------------------------------
 * add HMPML filetype, tab fixes in filetype.c

Mon, 14 Dec 2015 16:34:11 -0500 (Kevin Lin)
------------------------------------------
 * Hwp3.x: inflate compressed segment and offset tracking

Fri, 11 Dec 2015 17:50:40 -0500 (Kevin Lin)
------------------------------------------
 * Hwp3.x file header parsing and preclass

Wed, 16 Dec 2015 14:56:56 -0500 (Kevin Lin)
------------------------------------------
 * pcre: fixed minimum pcre version check

Fri, 11 Dec 2015 17:41:38 -0500 (Steven Morgan)
------------------------------------------
 * bb11452 - endianess correction for swf files on sparc -
patch supplied by Jim Morris.

Wed, 9 Dec 2015 12:15:16 -0500 (Mickey Sola)
------------------------------------------
 * bb11424 - correcting buffer alignemnt for riff detection on sparc systems;
patch by Jim Morris

Wed, 9 Dec 2015 00:17:41 +0100 (Andreas Cadhalpun)
------------------------------------------
 * fix detection of libcheck without pkg-config file

Wed, 9 Dec 2015 11:43:35 -0500 (Kevin Lin)
------------------------------------------
 * add HWP5 filetype tracking to preclassification

Tue, 8 Dec 2015 15:05:30 -0500 (Kevin Lin)
------------------------------------------
 * add HWP 3.x internal filetypes

Tue, 8 Dec 2015 14:47:12 -0500 (Kevin Lin)
------------------------------------------
 * add hwp5 contents to preclass set

Fri, 4 Dec 2015 11:55:52 -0500 (Kevin Lin)
------------------------------------------
 * proper identification of HWP file format

Tue, 8 Dec 2015 18:36:48 -0500 (Steven Morgan)
------------------------------------------
 * bb11438 - strengthen file typing for OOXML.

Wed, 2 Dec 2015 14:06:32 -0500 (Mickey Sola)
------------------------------------------
 * bb-11099 - normalization of whitelist signatures

Wed, 2 Dec 2015 16:23:48 -0500 (Steven Morgan)
------------------------------------------
 * github issue #9 - wrong length & offset.

Monday, September 14, 2015

With ClamAV’s second 0.99 beta release comes a revamp of the on-access scanner for Linux. This is the first major update to on-access scanning following the move to fanotify from Dazuko back in the 0.98 release.  A quick overview of features and updates can be found in the table below.


on-access-table.png


The meatiest updates for 0.99 on-access scanning are the recursive directory watching and dynamic watchpoint determination features. Collectively these features make up 0.99’s new dynamic directory determination system (ddd or 3D for short), which allows users to recursively watch any directory and track it for changes, updating fanotify watchpoints on the fly. 


In previous versions, the on-access scanner would watch only immediate children of the specified directory, and recursive monitoring could only be accomplished with custom scripts or manually specified clamd.conf entries. This limitation was partially due to fanotify’s inability to recognize the creation of a subdirectory within a watched parent directory. As a result, potentially malicious files could be freely created and executed one level beneath the parent directory, leaving the on-access scanner none the wiser. What’s more, even if the on-access scanner caught a malicious file, it would fail to stop that file from being executed, read, or opened.


Now, the latest version of on-access scanning has recursive watching—on any directory— built-in by default. Additionally, we leverage inotify (a kernel-provided API commonly used to help keep track of filesystem changes) to update fanotify watchpoints as changes occur to all watched directories and their subdirectories. But most critically of all, fanotify will correctly prevent read, open, and execute attempts for any virus found within the watched hierarchy.


Touching on the 3D system for a moment, it’s important to note that special care has been taken to ensure the new feature operates both quickly and efficiently. While there’s still room for improvement, the 3D system manages to preserve ClamAV’s tradition of speedy scanning. To put that into more tangible terms, it’s capable of processing watchpoint updates at a rough rate of 18,000 directories per second. Keep in mind, these metrics were gathered on a VM with decidedly modest hardware, so YMMV.


Of course, being able to watch thousands upon thousands of directories at all necessities dealing with inotify’s default limitations on active watchpoints. Typically, inotify limits users to setting watchpoints on a maximum of 8192 directories. The actual value can be located via /proc/sys/fs/inotify/max_user_watches. If you plan on pointing the on-access scanner at a directory with more than 8192 subdirectories, you’ll need to temporarily update this value as follows:


$ sysctl fs.inotify.max_user_watches=<n>


Where <n> should be replaced with your new max watch limit. If a more permanent solution is necessary, you can pipe an echo command with the new watch limit into tee and directly modify the max_user_watches entry under /proc. This would look something like:


$ echo <n> | sudo tee -a /proc/sys/fs/inotify/max_user_watches


As previously mentioned, this beta release also features changes to how fanotify handles infected files. Previous versions did not leverage fanotify’s ability to block access events. We’ve tweaked things such that fanotify now prevents any infected files from being read or opened, while allowing operations on clean files to continue unimpeded. Obviously, this may be problematic in an environment with one or more false positives, thus the final 0.99 release will allow users to configure the granularity of this feature via clamd.conf.

Along these same lines, those not keen on using the new 3D system will be able to toggle it off in the final release. Doing so will have the on-access scanner fall back to setting non-recursive watchpoints. However, this beta is focused on testing the new 3D system alongside the fanotify blocking features. So for now, on-access scanning will be using the 3D system exclusively. Luckily, the configuration options for these new features are unchanged from the previous release, meaning your current clamd.conf file shouldn’t need updating, so you might as well give it a try and let us know what you think!

Wednesday, September 9, 2015

We are pleased to announce the beta 2 of ClamAV 0.99. In addition to the features we added to 0.99 in beta 1, we've added a few more for the second beta that we'd love for you to test and provide feedback!

0.99-beta2
------

 *Highlights* for the second beta of ClamAV 0.99

  • New and improved on-access scanning for Linux. We will be showcasing a blog post in the near future about these capabilities.
  • Improved support for YARA rules including private rules, referencing other rules, and YARA "include"
  • Configurable default password list to attempt zip file decryption.
  • TIFF support.
  • ./configure options for YARA.
  • Upgrade Windows pthread library to 2.9.1.
  • A new signature target type for uncategorized files.
  • Improved fidelity of the "data loss prevention" algorithm. Code supplied by Bill Parker.
ClamAV 0.99 beta 2 is available for download from our download site under "Development Releases".

Please provide feedback on our ClamAV-Development list, found on our "Contact" page, under Mailing lists.

Please note that we are intentionally pointing you towards ClamAV.net for downloads, as we are planning on moving out permanent download location to ClamAV.net instead of using SourceForge at some point in the future.  More on this transition will be published later.

Wednesday, June 3, 2015

The first beta release of ClamAV 0.99 is now on SourceForge! ClamAV 0.99 has some important new features to improve malware detection.

First, ClamAV 0.99 supports YARA rules. YARA is another popular open source project for malware detection, analysis, and classification. YARA rules, in a nutshell, contain a list of strings and a powerful logical expression called the YARA condition. A YARA condition is typically composed of logical operations upon the YARA rule’s strings, with many other condition operators available as well. YARA strings come in three flavors: literal text strings (with modifier keywords NOCASE, FULLWORD, WIDE and ASCII), hexadecimal (including wildcard and alternates, similar to substrings in ClamAV’s logical signatures), and regular expressions. The full documentation about YARA rules may be found at http://plusvic.github.io/yara/.

One of the key benefits ClamAV brings to YARA is leveraging ClamAV’s myriad file decomposition capabilities. This enables YARA rules to automatically match on malware residing in any of the compression, archive, document, or packer formats provided by ClamAV.

Using YARA rules with ClamAV is simple - just place your YARA rule files into the ClamAV virus database location. This is /usr/local/share/clamav by default. Alternatively, you can place them in other locations and reference them with the “–database” command line option for clamscan or the clamd.conf “DatabaseDirectory” parameter if you are using clamd and clamdscan.

Additionally in ClamAV 0.99, we have added regular expression support to ClamAV’s logical signatures. This will enable signature authors to more readily reuse regular expression constructs from the Snort rule collection, thus providing more powerful malware detection for ClamAV.

Regular expressions in both YARA rules and ClamAV logical signatures require the Perl Compatible Regular Expressions (PCRE) library. Please ensure PCRE is installed on your system when configuring ClamAV. ClamAV configuration will automatically look for PCRE in /usr/lib and /usr/local/lib. If you have PCRE installed in a different location, use ‘./configure –with-pcre=DIR’ to specify the resident PCRE directory.

There are currently a few limitations of YARA rules within ClamAV 0.99 beta1, due either to nonexistent ClamAV capabilities or to YARA features that did not fit well into the ClamAV processing model. We hope to further evaluate and include as much of this functionality as possible in subsequent releases. YARA rules using any of the following features will be flagged in error, and the respective rules will be disabled:
  • Single byte YARA string components – currently in the ClamAV matcher, all strings, as well as components of strings delimited by wild cards, must be at least two bytes in length
  • External variables – variables referenced in YARA conditions whose value may be set using the ‘yara –d’ command line option.
  • Private rules – YARA rules which do not trigger by themselves. They are intended to match only when referenced by other YARA rules. These use the private keyword.
  • Global rules – YARA rules whose conditions are intended to be in effect for all other YARA rules. These use the global keyword.
  • Modules – A YARA feature intended to provide modular extensions to the YARA core. Modules are normally activated using the import keyword.
  • Rules precompiled with the YARA compiler – ClamAV only reads YARA rules in the original source form.
  • Rule tags – intended to provide a results filtering mechanism by specifying tag names on YARA rules and then using the ‘yara –t’ command to name the matching rule tags of interest.
  • References to other rules – intended to use the results from other YARA rules in a YARA condition.
  • YARA rules containing only a YARA condition – YARA rules in ClamAV currently must contain at least one string.
So, give it a try, we would love to get your comments and feedback! Please download ClamAV 0.99 “beta 1” from http://sourceforge.net/projects/clamav/files/beta/0.99-beta1/. Some YARA rules may be found at http://yararules.com/.

Tuesday, July 8, 2014

The ClamAV team is proud to announce the availability of ClamAV 0.98.5 beta ready for testing!

0.98.5
------

Welcome to ClamAV 0.98.5 beta! ClamAV 0.98.5 will include important new features for collecting and analyzing file properties. Software developers and analysts may collect file properties using the ClamAV API and then analyze them with ClamAV bytecode programs. Using the new features will require that libjson-c is installed.

Look for our upcoming series of blog posts to learn more about using the ClamAV API and bytecode facilities for collecting and analyzing file properties.

The release can be downloaded from http://sourceforge.net/projects/clamav/files/beta/.  Please download, test and provide any feedback via the mailing lists.


--
The ClamAV team (http://www.clamav.net/team)