New features in APT-RPM

APT-RPM is a port of Debian's APT tool to RPM based distributions (Conectiva Linux, Fedora Linux, SuSE Linux, ALT-Linux, etc), written and maintained by Conectiva. APT is an advanced package management utility front-end which allows one to easily perform package installation, upgrading and removal. Dependencies are automatically handled, so if one tries to install a package that requires others to be installed, it will download all needed packages and install them.

Recently, an intensive amount of development has been happening under the hood in the APT-RPM world, but unfortunately, most of the features are only perceived by a small number of people that follow the development closely. This article is an attempt to introduce the reader to some of the latest features available today in the RPM port of APT.

 2. Dealing with local packages

One of the recently introduced features, which was on the top of the TODO list for a long time, is the capability of dealing with random local packages using APT's ability to handle dependencies. With this feature, installing a local package with dependency handling is as easy as installing a file in any remote repository. Here is an example, assuming that the file is in the current working path:

% apt-get install rpmver-2.0-13498cl.i386.rpm

Reading Package Lists... Done
Building Dependency Tree... Done
Selecting rpmver to represent rpmver-2.0-13498cl.i386.rpm
The following NEW packages will be installed:  rpmver
0 upgraded, 1 newly installed, 0 removed and 8 not upgraded.
Need to get 0B/6404B of archives.
After unpacking 5552B of additional disk space will be used.
Committing changes...
Preparing...                ########################################### [100%]
   1:rpmver                 ########################################### [100%]
Done.

This is valid for operations dealing with source packages as well. The following example shows an operation that checks every build-time dependency of the given source package, asks for confirmation, fetches, and installs them locally.

% apt-get build-dep apt-listchanges-1.49-11104cl.src.rpm

Reading Package Lists... Done
Building Dependency Tree... Done
The following NEW packages will be installed:
  rpmver
0 upgraded, 1 newly installed, 0 removed and 8 not upgraded.
Need to get 6404B of archives.
Need to get 6404B of archives.
After unpacking 5552B of additional disk space will be used.
Do you want to continue? [Y/n]

Get:1 ftp://mapi8.distro.conectiva latest/conectiva/all rpmver 2.0-13498cl [6404B]
Fetched 6404B in 8s (798B/s)
Committing changes...
Preparing...                ########################################### [100%]
   1:rpmver                 ########################################### [100%]
Done.

 3. Local repositories

Dealing with package files directly is just one way to use APT-RPM. Another easy way is to set up a local repository. Traditionally, local repositories are always available, but they require maintenance of meta-information, which APT-RPM expects to find in the base/ subdirectory. Now APT-RPM has learned to deal with a special kind of local repository, which does not require the maintenance of meta-information. Instead, information is fetched directly from the packages, and new packages dropped into these directories are automatically recognized.

The configuration of this new kind of local repository is straightforward. One only needs to replace the rpm source type with the rpm-dir source type in the sources.list repository configuration file. For example, to maintain a repository in /repos/conectiva/RPMS.local and to have every package dropped in this directory automatically recognized by APT-RPM, one should include the following line in the sources.list file:

rpm-dir file:///repos conectiva local

No additional configuration is needed.

The same rules apply to source repositories as well, using rpm-src-dir instead of rpm-src in the sources.list repository configuration file. Extending the example above, the following line would allow one to maintain SRPM packages in /repos/conectiva/SRPMS.local without any further work:

rpm-src-dir file:///repos conectiva local

 4. Installing packages by filenames

One feature that seems logical for most package tool users is the ability to install packages by providing file names instead of package names. This feature was only available in APT-RPM through an external Lua extension, until recently. Now this is available internally in APT-RPM, which is able to translate any filename included in the meta information of the remote repository.

The following example shows the feature working. The filename is translated to the package name, and with user confirmation, the package is downloaded and installed.

% apt-get install /usr/bin/rpmver

Reading Package Lists... Done
Building Dependency Tree... Done
The following extra packages will be installed:
  rpmver
The following NEW packages will be installed:
  rpmver
0 upgraded, 1 newly installed, 0 removed and 8 not upgraded.
Need to get 6404B of archives.
After unpacking 5552B of additional disk space will be used.
Do you want to continue? [Y/n]

Get:1 ftp://mapi8.distro.conectiva latest/conectiva/all rpmver 2.0-13498cl [6404B]
Fetched 6404B in 17s (376B/s)
Committing changes...
Preparing...                ########################################### [100%]
   1:rpmver                 ########################################### [100%]
Done.

Notice that it's possible to exclude some files from the meta-information, for space saving purposes.

 5. APT Shell

Another interesting feature developed recently is the apt-shell. This is a tool which offers a shell to make the user experience more pleasant while navigating through the APT cache and installing, erasing, and upgrading packages.

Here is a very short list of available features to leave the reader excited enough to look further:

• Stateful cache. Mark packages as you want and when satisfied with the current selection, commit to the system.
• Gradual selection. Each time you select a package that will include more changes in the cache than was requested, you'll be notified about what changes will be made, and given the option to cancel.
• Smart completion. Command line completion for commands, package names, and versions. When removing, only installed packages will complete. When keeping, only packages selected for changing will complete.
• New list/ls command, allowing one to list packages in a comfortable way, including shell wildcards. Also, options for listing only installed packages, only upgradeable packages, listing installed and candidate versions, and listing summaries.
• Wide set of commands, including most of the apt-get and apt-cache functionality.
• Inline help.

Here is an interactive sample section, hacked for objectiveness.

Reading Package Lists... Done
Building Dependency Tree... Done
Welcome to the APT shell. Type "help" for more information.

apt> install dum [TAB pressed]
dummy  dump  

apt> install dummy= [TAB pressed]
0.1-1cl    1.0-1cl    1:1.0-1cl  1:1.5-1cl 

apt> install dummy=1:1.0-1cl

apt> install alsa-lib-devel
Unrequested changes are needed to execute this operation.
The following packages will be upgraded
  alsa-lib-devel libalsa2
(...)
Do you want to continue? [Y/n] n
Abort.

apt> commit
The following NEW packages will be installed:
  dummy
(...)
Executing RPM (-Uvh)...
Preparing...                ########################################### [100%]
   1:dummy                  ########################################### [100%]
(...)

apt> ls dum*
dummy  dump 

apt> ls -v dum*
Name   Installed           Candidate
----   ---------           ---------
dummy  1:1.0-1cl           1:1.5-1cl
dump   -                   0.4b28-11400cl

apt> ls -s dum*
dummy - Dummy package doing weird stuff.
dump - Programs for backing up and restoring filesystems.

 6. Meta-components

Unlike the Debian strategy of maintaining main, contrib, and a few other major components, some users of APT-RPM, like Conectiva Linux, have chosen to split components in a more finely grained fashion with, for example, devel, doc, audio, and many others. This introduces some interesting possibilities, as users may "subscribe" to just those components they are really interested in. On the other hand, splitting components like this also introduces some annoyance to those who want to simply subscribe to every available component. It also creates problems when a new component is created, since nobody is currently subscribed to it, and would take some time to discover that the new component was available.

The meta-component was created to solve these issues. Using meta-components one is able to create components that contain the information of other real components. For APT-RPM users, there's no difference between a meta-component and a real component. For repository maintainers who want to create a meta-component, it's just a matter of using the appropriate genbasedir options.

For example, suppose that the components main, extra, kde, and gnome are available, one may create an additional all meta-component which includes information from all of them with the following command:

genbasedir --meta=all /repos/path main extra kde gnome

Doing this, users may either include any combination of individual components in their sources.list repository configuration file, or use the meta-component.

 7. Lua interface

One exciting new capability recently introduced into APT-RPM is the embedding of the Lua interpreter. This allows customization and extension of APT-RPM in any direction, being limited mostly by the imagination and scripting ability of anyone who needs extensive packaging features.

The scripting interface is done using a slot concept. Slots are places in APT-RPM execution where any number of scripts may be plugged in to introduce special behavior. Some of these slots allow scripts to introduce new commands in the APT-RPM tools apt-get, apt-cache and apt-shell. Additionally, scripts may also be called with the script subcommand.

Currently, a good part of the APT API is already available to Lua scripts, and complete documentation is available at AptRpm/Scripting.

To give you an idea of how it works, the following script may be executed with apt-get script install-devel.lua (assuming it has this name), to install development packages with suffixes -devel and -devel-static, as long as the main package is already installed.

function installdevel(name)
        pkg = pkgfind(name.."-devel")
        if pkg and not pkgvercur(pkg) then
                markinstall(pkg)
        end
        pkg = pkgfind(name.."-devel-static")
        if pkg and not pkgvercur(pkg) then
                markinstall(pkg)
        end
end

for i, pkg in pairs(pkglist()) do
        if pkgvercur(pkg) then
                installdevel(pkgname(pkg))
        end
end

 8. Upgrading algorithm

Some changes in APT-RPM go mostly unnoticed by the everyday user. One of these changes happened during the release period of Conectiva Linux 9.0. Several days were spent to improve the behavior of APT-RPM in complex situations, like when whole distributions are upgraded. Today, APT-RPM is the only tool used for upgrading Conectiva Linux, and heavy tests were done, even upgrading two distribution versions in a single step. Recently, users of other distributions have reported that the upgrading algorithm is indeed working more reliably, as they are also able to upgrade their own distributions, and problems during upgrades are mostly related to packaging bugs.

 9. Internal committing of changes

One behavior that many users disliked, was the way it committed the programmed changes to the system. Until recently, APT-RPM did this by executing the rpm binary itself.

This behavior indeed had a few disadvantages, like splitting the transition into more than one unit, since the rpm binary doesn't currently support installation and erasure of packages in a single step.

Fortunately, since version 0.5.15cnc3, APT-RPM supports internal committing of changes, through the use of the rpmlib API, which leaves the mentioned problems behind. Using the old method is still supported, and is runtime configurable.

 10. Synaptic

Synaptic is very good graphic interface for package installation which supports both APT-RPM and APT. Lately this software is receiving special attention, and is evolving at large steps. If one would like to have access to the features of APT-RPM or APT with a graphical interface, looking at the Synaptic project is highly advisable.

 11. Changes in the original APT

Changes being done in the original version of APT, maintained by Debian developers, are constantly being integrated in APT-RPM as well. A small example of this is the recently introduced argument of the install subcommand of apt-get which shows which versions are going to be installed in the system, if the transaction is committed, as shows the following example:

% apt-get install -V rpmver
Reading Package Lists... Done
Building Dependency Tree... Done
The following NEW packages will be installed:
   rpmver (2.0-13498cl)
0 upgraded, 1 newly installed, 0 removed and 8 not upgraded.
Need to get 6404B of archives.
(...)

 12. Credits

The core maintenance of APT-RPM is done by the Conectiva developer Gustavo Niemeyer, but it would be unfair to take the credits and not mention other people (without any special order), like Panu Matilainen and Richard Bos, which have always been persistent APT-RPM contributors; Michael Vogt, a Debian developer that has been doing a wonderful job maintaining Synaptic with Gustavo, Jeff Johnson, the RPM software maintainer; ALT-Linux, which constantly send patches "upstream"; Vine Linux, which pushes APT-RPM in the oriental side of the world; the Debian developers which contribute to the original APT project, and many others which contribute to the continuous progress of APT-RPM.

 13. Links

• https://moin.conectiva.com.br/AptRpm
• https://moin.conectiva.com.br/AptRpm/Scripting
• http://www.nongnu.org/synaptic/

    for i, pkg in pairs(pkglist()) do
        ver = pkgvercur(pkg)
        verlist = pkgverlist(pkg)
        if ver and not verisonline(ver)
           and table.getn(verlist) == 1 then
            print(pkgname(pkg) .. "-" .. verstr(ver))
        end
    end

    apt-cache script ./list-extras.lua

    apt-cache list-extras

    if command_args[1] ~= "list-extras" then
        return
    end
    command_consume = 1

    Scripts::AptCache::Command:: "/path/to/list-extras.lua";

def devel_packages_to_install():
  def dev_pkgs(pkgs):
    for p in pkgs:
      try:
        yield package(p.name+'-deve')
        yield package(p.name+'-deve-static')
      except:
        pass
  return [ p for p in dev_pkgs(uptodate_packages())
    if not p.is_uptodate() ]
map(lambda p: p.install(),devel_packages_to_install())

% apt-get install ftp://mapi8.distro.conectiva/pub/(...)/rpmver-2.0-13498cl.i386.rpm 

Get:1 ftp://mapi8.distro.conectiva/(...)/rpmver-2.0-13498cl.i386.rpm [6404B]
Fetched 6404B in 1s (5844B/s)   
Reading Package Lists... Done
Building Dependency Tree... Done
Selecting rpmver for 'rpmver-2.0-13498cl.i386.rpm'
The following NEW packages will be installed:
  rpmver
0 upgraded, 1 newly installed, 0 removed and 5 not upgraded.
Need to get 0B/6404B of archives.
After unpacking 5552B of additional disk space will be used.
Committing changes...
Preparing...                ########################################### [100%]
   1:rpmver                 ########################################### [100%]
Done.