Certificate freshness

The fetch-crl utility will retrieve certificate revocation lists (CRLs) for a set of installed trust anchors, based on crl_url files or IGTF-style info files. It will install these for use with OpenSSL, NSS or third-party tools.

The tool, evolved (but completely rewritten) from the EU DataGrid fetch-crl2 utility, is maintained on Github and documented on the PDP Wiki.

Availability

Fetch-crl is available from the usual reposities for most Linux distributions: EPEL and Fedora for RH Enterprise derivatives such a Rocky Linux or CentOS, maintained by Steve Traylen, and for Debian net/fetch-crl maintained by Mattias Ellert.

The latest version is 3.0.23. The latest version in a generic tar-ball and RPM format is always available from the IGTF distribution web site.

Sources are on Github, and can be built with a simple make command. Fetch-CRL has been tested on GNU/Linux and Sun’s Solaris 10 platforms.

Changelog:

  • most important changes in 3.0.23:
    • Add support for explicitly setting https_proxy for retrieval, while making the earlier http_proxy setting also affect https retrievals. The new https_proxy setting is mutually exclusie with “http_proxy=ENV”
  • previous significant changes:
    • Add option to override the UserAgent string sent in the LWP web requests (both HEAD and GET)
    • Add option for postexec hook to execute a command after running fetch-crl (or after each CRL installation)
    • fix superfluous newline in DER formatted CRL files
    • Re-set cache expiry of state data if CRL nextUpdate is within or beyond 7 hrs (config “expirestolerance”) claimed URL Expiry or Cache-control max-age

Features in Fetch-CRL version 3.0

  • parallel downloading for multiple trust anchors
  • explicit web proxy support (using LWP http proxies)
  • support for multiple output formats: OpenSSL 1 in dual-hash mode, specific DER and PEM outputs, and NSS databases
  • support for multiple CRLs for a single CA, allowing more than one CA with the same subject name but different CLRs. Review your client software to see if and how these CRLs are used.
  • stateful retrieval helps reduce bandwidth usage by caching the CRLs locally and respecting the Cache Control headers sent by the web server hosting the CRL. This can reduce the number of downloads
  • support for HEAD-only requests when state preservation is used (initially only retrieve HTTP headers, and only if the CRL actually changed to a full download)
  • support for more CRL retrieval protocols (file:// and ftp://)
  • ability to try site-local URLs first, before relying on the URLs shipped with the trust anchor. This allows building an explicit local caching (web) server.
  • ability to specify additional URLs to try in case the URLs shipped with the trust anchor were not responsive. This allows for automatic fall-back to (local or global) mirror services for CRL downloads
  • warnings and errors can be suppressed on a per-trust anchor basis, to allow silencing for particularly unstable trust anchors

  • aging tolerance (the delay time before errors are generated in case downloads consistently fail) can be configured on a per-trust anchor basis

  • only requires perl5 to be installed (tested with perl 5.8.0 and higher) with libwww-perl, and only the basic modules (such as POSIX)
  • requires a version of OpenSSL (0.9.5a or better) to be installed. Needs OpenSSL 1.0.0 (at least beta5) for dual-hash support, and needs OpenSSL even when only the NSS database output format has been selected
  • when using parallel downloads, it can only run on pure-POSIX systems (parallelism in combination with the NSS database output format has not been tested)

Configuration file

Fetch CRL comes with some built-in default, but virtually all of its behaviour can be controlled through configuration files and command-line arguments. The command-line arguments are a subset of the directives, but override those directives when specified at run-time. The list of valid configuration options is given below - but note that directives that are not recognised will be silently ignored (this does not apply to command-line arguments).

By default, the configuration file “/etc/fetch-crl.conf” is read on start up. If this does not exist, but “/etc/fetch-crl.cnf” does exist, the latter is read. If an explicit configuration file is specified on the command-line (“-c file”), this file is used. Additional configuration has be given through separate configuration files in a configuration directory (by default: “/etc/fetch-crl.d”).

CRL URL and INFO formats

The CRL URL(s) come from a per-trustanchor “.crl_url” or “.info” file. Where both are available, the “.info” file is preferred and the crl_url file ignored. The URLs must be resolvable by libwww-perl, so http, ftp, file, or gopher URLs are allowed, as well as any other LWP::Protocols that have been explicitly added (e.g. LWP::Protocols::ldap). Support for https (and ldaps) requires perl’s IO::Socket::SSL or Net::SSLeay.

Known issues

  • although fetch-crl3 will install multiple CRLs in the CRl stores (called “.r0”, “.r1”, or labelled appropriately in an NSS store), if the number of CRLs decreases the left-overs are not automatically removed. So if the number of CRLs for a particular CA does down from n to n-1, the file “.rn” must be removed manually.
  • to remove “.rn” files that have no corresponding CA file anymore, use the clean-crl script. See clean-crl(8) for details.

Source code management and issue tracking

Sources are kept on Github

  • https://github.com/dlgroep/fetch-crl

You can file issues and bugs at

  • https://github.com/dlgroep/fetch-crl/issues

but please be advised that this product is supported on a best-effort basis. Supplying patches that fix the issue is encouraged and appropriate patches will be reviewed and included with attribution (unless you want to remain anonymous). Submitted patches and code shall be governed by the Apache 2.0 license as well. Expedited issue processing can be obtained by sending the appropriate amount of apple pie (preferably fresh so no sea freight shipping please).

License, copyrights and acknowledgments

Fetch-crl3 is a complete re-write of the utility, inspired by the previous versions but no code has been re-used from there.

Copyright 2010-2022 David Groep, Nationaal instituut voor subatomaire fysica Nikhef

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0. Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

This work is part of the research programme of the Dutch Foundation for Fundamental Research on Matter (FOM), which is financially supported by the Netherlands Organisation for Scientific Research (NWO).

This work is part of the programme of BiG Grid, the Dutch e-Science Grid, which is financially supported by the Nederlandse Organisatie voor Wetenschappelijk Onderzoek (Netherlands Organisation for Scientific Research, NWO).

Tags: security