Fedora evicts WolfSSL
The Fedora Engineering Steering Committee (FESCo) has voted to immediately remove the WolfSSL package from all of Fedora's repositories due to its maintainer failing to gain approval to package a new cryptography library for Fedora. WolfSSL's brief travels through Fedora's package system highlights gaps in documentation, as well as in the package‑review process. The good news is that this may stir Fedora to improve its documentation and revive a formal security team.
Fedora and cryptography
Fedora's packaging
guidelines require that every application entering Fedora be
checked for compliance with the policies
on cryptography, but those policies could be written more
clearly and are in need of an update. For example, the crypto
policies say that new libraries "must comply with the crypto policies to enter Fedora
" which
seems oddly circular since the reader would likely think that is what they
are reading. However, that is meant to be a reference to Fedora's crypto‑policies
project, and that crypto libraries must have full integration with
this system.
The crypto-policies project, maintained by Alexander Sosedkin, is an effort to unify the crypto policies for the whole distribution and also simplify the management of crypto applications and libraries. This means, in part, that Fedora has a limited set of approved crypto "back-ends" such as OpenSSL, GnuTLS, and Libreswan.
Fedora users can set system-wide crypto policies, such as a legacy policy when older encryption algorithms are needed for compatibility or the FIPS policy for conformance with FIPS 140 requirements. This system was adopted with the Fedora 21 release in 2014. The change proposal has a description of the system, and the crypto-policies man page describes the available policies and tools. A new crypto library would need to integrate with this system, but first it would have to be accepted in the first place.
Crypto libraries new to Fedora are required to get approval before being added, though the documentation does not do a great job of describing that process. Even though the packaging guidelines are a bit confusing, it should be clear enough that packagers need to consult with the Fedora security team, and then gain an exception from the Fedora packaging committee before being added to the repositories. There is one minor problem with this, though: the Fedora security team has been defunct for a while, and the policies have not been updated to reflect this.
WolfSSL
This posed difficulties for Andrew Bauer, who wanted to package WolfSSL, an SSL/TLS library written in C, as a dependency for another package he maintains, Netatalk. That software is an open-source implementation of the Apple Filing Protocol, which allows Linux (and other) systems to act as AppleShare file servers for new and old Apple systems. Really old, in fact—Netatalk version 2 supports systems as far back as the Apple II and version 3 supports Macs running Mac OS 7.5 (released in 1994) through today's Macs.
The Netatalk project had dropped OpenSSL in favor of WolfSSL with its 3.2.0 release in June because OpenSSL 3.0 had deprecated encryption algorithms that it needed to use for authentication with ancient Mac OS clients. This meant that Bauer could no longer rely on OpenSSL to fill the needs of the Netatalk package in Fedora, and needed to package WolfSSL. However, WolfSSL would need special approval to enter Fedora beyond simple package approval.
With Netatalk requiring WolfSSL, Bauer created a package and filed a review request on August 3. As part of the request, he included the rpmlint output that had flagged the package as non-compliant with Fedora's packaging policy. Bauer said:
Grepping the source code, one can see that wolfssl calls wolfSSL_CTX_set_cipher_list() rather than SSL_CTX_set_cipher_list(). Thus, this is a false positive and can be ignored.
And ignore it he did. So did Felix Wang, who reviewed the package and approved it on August 17. It might have helped if the error had been more explicit, or if it were listed among the common rpmlint issues in the package documentation. An issue was opened about the lack of description in 2023. The error is meant to alert the user that the application may not be using the system-wide crypto policy. FESCo member Fabio Valentini replied to the WolfSSL review‑request ticket to say that it would require further review and pointed to the crypto policies.
Security team snafu
So, Bauer sought
help via the fedora‑devel list on August 18 to figure out how to
contact the non-existent Fedora security team per the packaging
guidelines. Neal Gompa responded
that the "formal security team disbanded many years ago
". He
suggested that Bauer visit the Fedora security team room on Matrix,
which Bauer did. (Note that link and several others require logging
into a Matrix server. It is possible to log in without creating an
account, click "Continue", select the Element client, and then click
"Continue in your browser".)
He said
that "a question was asked whether this package requires approval
from Fedora Security
". He got a quick suggestion from Demi Marie
Obenour about compilation options that did not address the question of
approval directly. He thanked Obenour, and seems to have left the
channel before he received messages about Fedora's
crypto policies or suggesting that new crypto libraries would require
a change request. Bauer plunged ahead and updated
the review ticket to report he'd been given advice on
building WolfSSL, and was requesting a package repository now.
The WolfSSL package entered the testing repositories for
EPEL 9, Fedora 39, 40, 41 (which is still in development),
and rawhide on August 20.
Valentini replied to Bauer's email on fedora-devel saying that the question of whether WolfSSL complies with system crypto libraries had not been answered. He was unhappy that it had already been imported into Fedora. Daniel P. Berrangé wrote that it seemed non-compliant and that a Fedora packaging committee (FPC) exception is required.
More than two weeks later, Valentini followed
up in the review ticket and said it looked like
Bauer had ignored his earlier comment about crypto compliance
completely. The two went back and forth a bit. Bauer again
complained the documentation was unclear and said: "Once we
identify the source of authority, looks like that's Felix, I'll be
glad to [work] through this.
"
Retirement
Instead of debating further with Bauer, Valentini opened a ticket with FESCo about
the matter. He noted that "the reviewer of the package (not the
submitter)
" had filed a request with the packaging committee to
approve the package, but it had already made its way into Fedora's
stable repositories.
Stephen Gallagher said that he'd spoken with the crypto-policies maintainer and members of Red Hat's security team. Their opinion was that WolfSSL should not be permitted in Fedora without honoring crypto-policies. Gallagher proposed that FESCo vote on the following:
WolfSSL is immediately retired from Fedora. The maintainers may file a new package review request when WolfSSL respects the crypto system policy. This review request must be presented to the FPC, who must approve it before it is added back to the repositories.
On September 10, Valentini wrote that Gallagher's proposal had been discussed, voted on, and approved by FESCo. Five votes were in favor of the proposal, none against or abstaining.
Typically, packages are only retired from rawhide or branched versions of rawhide that are in the process of becoming the next stable Fedora release but have not entered final freeze. Retiring packages from stable versions of Fedora is a rare step, and usually only undertaken when a package has licensing problems. Once the package is retired, it is removed from subsequent composes and will be removed from the archives. It will not be removed from systems it was installed on unless it is also added to the fedora-obsolete-packages package. That has not been done in this case. As of this writing, the package is still installable, but it should be disappearing soon.
Aftermath
There are several points when Bauer clearly should have slowed down and gotten the message that he needed to consult Fedora's packaging committee. If nothing else, Valentini was waving a caution flag that Bauer should have heeded.
Still, he is not wrong that the documentation leaves much to be desired in terms of
clarity and completeness. And, as Matthew Miller noted in
the FESCo ticket, the security team is in a "disordered
state
" and it may be time to "reorganize and re-formalize the
security team
". Michel Lind commented
that he had been sounding out frequent posters in the Matrix security
room about improving things, so the project may be gathering steam to
put a real team in place, once again, to avoid a recurrence of non-packagers giving
random advice that is taken as official. It might also behoove Fedora
to consider whether a chat room is a real substitute for an email list
that does not emphasize real-time communication or require a special
client to use.
Fedora's policies may also need some updating to take into account use cases that require special consideration for compatibility with obsolete hardware and software. It is a net good that Fedora users can use software like Netatalk to connect with classic computers, and it would be a shame if Fedora's policies blocked such software permanently. In the end, no real harm has been done, and Bauer's over-eagerness has helped the Fedora project identify a few real problems to address.