We are pleased to announce the release of ClamAV 0.101.0! Please take a look at the below release notes for further information.
0.101.0 Release Notes
ClamAV 0.101.0 is a feature release with an assortment of improvements that we've cooked up over the past 6 months.
Highlighted Changes:
- Our user manual has been converted from latex/pdf/html into Markdown! Markdown is easier to read & edit than latex, and is easier to contribute to as it eliminates the need to generate documents (the PDF, HTML). The user manual is now shipped in:docs/UserManual[.md]. However, the most up to date version at any time will be on ClamAV.net (This is not live right now, but will be shortly)
- Support for RAR v5 archive extraction! We replaced the legacy C-based unrar implementation with RarLabs UnRAR 5.6.5 library. Licensing is the same as before, although our
libclamunrar_iface
supporting library has changed from LGPL to the BSD 3-Clause license.
- Libclamav API changes:
- The following scanning functions now require a filename argument.
- This will enable ClamAV to report more details warning and error information in the future, and will also allow for more sensible temp file names. The filename argument may be
NULL
if a filename is not available.
cl_scandesc
cl_scandesc_callback
cl_scanmap_callback
- Scanning options have been converted from a single flag bit-field into a structure of multiple categorized flag bit-fields. This change enabled us to add new scanning options requested by the community. In addition, the name of each scan option has changed a little. As a result, the API changes will require libclamav users to modify how they initialize and pass scan options into calls such as
cl_scandesc()
.
- With our move to openssl versions >1.0.1, the
cl_cleanup_crypto()
function has been deprecated. This is because cleanup of open-ssl init functions is now handled by an auto-deinit procedure within the openssl library, meaning the call to EVP_cleanup()
may cause problems to processes external to Clam.
CL_SCAN_HEURISTIC_ENCRYPTED
scan option was replaced by 2 new scan options:
CL_SCAN_HEURISTIC_ENCRYPTED_ARCHIVE
CL_SCAN_HEURISTIC_ENCRYPTED_DOC
- clamd.conf and command line interface (CLI) changes:
- As in 0.100.2, the clamd.conf
OnAccessExtraScanning has been temporarily disabled in order to prevent resource cleanup issues from impacting clamd stability. As noted below, OnAccessExtraScanning
is an opt-in minor feature of on-access scanning on Linux systems and its loss does not significantly impact the effectiveness of on-access scanning. The option still exists, but the feature will not be enabled and a warning will show if LogVerbose
is enabled.
- For details, see: https://bugzilla.clamav.net/show_bug.cgi?id=12048
- "Heuristic Alerts" (aka "Algorithmic Detection") options have been changed to make the names more consistent. The original options are deprecated in 0.101, and will be removed in a future feature release.
- In addition, two new scan options were added to alert specifically on encrypted archives or encrypted docs. Previous functionality did both, even though it claimed to be specific to archives:
- Scan option details:
| Old `clamd.conf` option | *New* `clamd.conf` option |
| -------------------------------- | ---------------------------- |
| `AlgorithmicDetection` | `HeuristicAlerts` |
| `DetectBrokenExecutables` | `AlertBrokenExecutables` |
| `PhishingAlwaysBlockCloak` | `AlertPhishingCloak` |
| `PhishingAlwaysBlockSSLMismatch` | `AlertPhishingSSLMismatch` |
| `PartitionIntersection` | `AlertPartitionIntersection` |
| `BlockMax` | `AlertExceedsMax` |
| `OLE2BlockMacros` | `AlertOLE2Macros` |
| `ArchiveBlockEncrypted` | `AlertEncrypted` |
| | `AlertEncryptedArchive` |
| | `AlertEncryptedDoc` |
| Old `clamscan` option | *New* `clamscan` option |
| ---------------------------- | -------------------------------- |
| `--algorithmic-detection` | `--heuristic-alerts` |
| `--detect-broken` | `--alert-broken` |
| `--phishing-cloak` | `--alert-phishing-cloak` |
| `--phishing-ssl` | `--alert-phishing-ssl` |
| `--partition-intersection` | `--alert-partition-intersection` |
| `--block-max` | `--alert-exceeds-max` |
| `--block-macros` | `--alert-macros` |
| `--block-encrypted` | `--alert-encrypted` |
| | `--alert-encrypted-archive` |
| | `--alert-encrypted-doc` |
Some more subtle improvements:
- Logical signatures have been extended with a new sub-signature type which allows for numerical byte sequence comparison. For those familiar with Snort, this byte comparison feature works similarly to the byte_extract and byte_test feature, in that it allows signature writers to extract and compare a specified number of bytes (offset from a match) against another numeric value. You can read more about this feature, see how it works, and look over examples in our documentation.
- Backwards compatibility improvements for detecting the OpenSSL dependency.
- Freshclam updated to match exit codes defined in the freshclam.1 man page.
- Upgrade from libmspack 0.5alpha to libmspack 0.7.1alpha. As a reminder, we support system-installed versions of libmspack. However, at this time the ClamAV-provided version of libmspack provides additional abilities to parse broken or non-standard CAB files beyond what the stock libmspack 0.7.1alpha provides. We are working with the upstream project to incorporate our modifications, and hopefully these changes will appear in a future release of libmspack.
- Updated the bundled 3rd party library libxml2 included for Windows builds to version 2.9.8.
- Updated the bundled 3rd party library pcre included for Windows builds to pcre2 version 10.31.
- Upgraded Aspack PE unpacking capability with support up to version 2.42.
- Improvements to PDF parsing capability.
- Replaced the Windows installer with a new installer built using InnoSetup 5.
- Improved
curl-config
detection logic
- GitHub pull-request by Thomas Petazzoni.
- Added file type
CL_TYPE_LNK
to more easily identify Windows Shortcut files when writing signatures.
- Improved parsing of Windows executable (PE) Authenticode signatures.
- Pull request by Andrew Williams.
- Added support for Authenticode signature properties commonly used by Windows system files. These files are now much more likely to be whitelisted correctly.
- Signature parsing now works correctly on big endian systems.
- Some simplification to freshclam mirror management code, including changes to reduce timeout on ignoring mirrors after errors, and to make freshclam more tolerant when there is a delay between the time the new signature database content is announced and the time that the content-delivery-network has the content available for download.
- Email MIME Header parsing changes to accept argument values with unbalanced quotes. Improvement should improve detection of attachments on malformed emails.
- GitHub pull request by monnerat.
- Included the config filename when reporting errors parsing ClamAV configs.
- GitHub pull request by Josh Soref.
- Improvement to build scripts for clamav-milter.
- GitHub pull request by Renato Botelho.
Other changes:
- Removed option handler for
AllowSupplementaryGroups
from libfreshclam. This option was previously deprecated from freshclam in ClamAV 0.100.0 but remained in libfreshclam by mistake.
- In older versions of pcre2 and in pcre, a higher
PCRERecMatchLimit
may cause clamd
to crash on select files. We have lowered the default PCRERecMatchLimit
to 2000 to reduce the likelihood of a crash and have added warnings to recommend using pcre2 v10.30 or higher to eliminate the issue.
Supporting infrastructure:
As you might imagine, ClamAV is much more than just the tarball or EXE you download and install. Here at Cisco Talos, we've been working hard on the support infrastructure that's so easy to take for granted.
- Test Frameworks
- Feature Testing:
- Throughout the development of ClamAV 0.101, our quality assurance engineers have been hard at work rebuilding our QA automation framework in Python from the ground up to test ClamAV features on 32-and-64bit versions:
- Linux: Ubuntu, Debian, CentOS, Fedora
- FreeBSD 11
- Windows 10
- In addition to building out the framework, they've written over 260 individual feature tests to validate correctness of the new features going into 0.101 as well as to validate many existing features.
- Build Acceptance Testing:
- Another major task accomplished during the development of 0.101 was the creation of a build acceptance test framework that we run from our Jenkins CI server.
- Similar to the feature testing framework, our build acceptance framework tests across 64bit and 32bit (where available):
- macOS 10 (.10, .11, .13)
- Windows (7, 10)
- Debian (8, 9), Ubuntu (16.04, 18.04), CentOS (6, 7)
- FreeBSD (10, 11)
This pipeline creates our release materials including the Windows installers, and then validates that the basic install, update, start, scan, and stop procedures all work as expected each time commits are made to our development branches.
- Signature Database Distribution:
- During the course of ClamAV 0.101 development, our Community team has been able to migrate us from a network of third-party mirrors over to use the services of CloudFlare to provide a more unified content-delivery-network.
- With CloudFlare, users in geographic regions that had few mirrors will notice much improved signature update speeds and reliability. In addition, we're excited to be able to finally see user metrics that will help us continue to improve ClamAV. We've observed a 57% improvement in download speeds (on average) across the globe. We are of course grateful to all of the community members who have donated their server bandwidth to mirror the ClamAV signature databases over the years. Thank you so much! We have not called for an official shutdown of the old mirror network yet.
- Development Processes:
- As many of you know, ClamAV 0.100 was in development for a good two years. Not only was this frustrating for users awaiting new features and bug-fixes, it also made for a difficult transition for users that weren't expecting two years worth of change when 0.100 landed.
- We have learned from the experience and are committed to providing shorter and more responsive ClamAV development cycles.
ClamAV 0.101 is the first of many smaller feature releases where we created a roadmap with distinct deadlines and with specific planned features. We based the feature list on both community requests and our own needs and then executed that plan.
We're very proud of ClamAV 0.101 and we hope you enjoy it.
Acknowledgements:
The ClamAV team thanks the following individuals for their code submissions:
- Andrew Williams, Cisco Talos
- Craig Andrews
- Josh Soref
- monnerat
- Renato Botelho
- tchernomax
- Thomas Petazzoni
As always, we welcome discussion about ClamAV on our
mailing lists.
