LXR Cross Referencer Git repo

48 lines (41 with data), 2.5 kB

Hacking LXR
-----------

Things have started to stabilize a bit, so I will attempt to explain
the internals of the new LXR. 

The goal of this new version is to support version control systems.
The old LXR did version control by maintaining separate source trees
and separate identifier databases. While this seemed to work fine it
really was a kludge. 

We realized early that db-file databases were too weak for our
purpose, a real relational database would enable us to do much more
without the speed penalties of the db-files. We have therefore
abstracted the database that contains information about the
identifiers into the module LXR::Index. The LXR::Index::DBI module
uses the perl DBI interface to a relational database (we used postgres
for most of our development), while LXR::Index::DB tries to accomplish
the same by using db-files. This makes it possible for those with
large source-trees to use DBI and those who do not want to install a
RDBMS can stay with the db-files.

The old lxr had to store a complete source-tree for each version you
wanted to index. When the number of versions were relatively low this
worked just fine, but if you wanted to index the whole linux kernel
history the space requirement would be very large. It would therefore
be nice to be able to do the indexing on a CVS tree. The LXR::Files
backend abstacts the underlying file storage mechanism from the rest
of the system. (If you think we are thinking too much object
orientation and abstracting too much remember that we are both brought
up at the university where object orientation was invented :-). The
LXR::Files::Plain uses the old method were each version is stored in a
separate repository, while LXR::Files::CVS fetches the files from a
CVS repository. It should be possible to add support for other forms
of version control in the future.

Another design-goal for this new version was to support multiple
languages. Our old index generator was very c-specific and its
internals were really messy. We decided to get rid of that code and
are now using external index generators. Exuberant c-tags is very
good, and finds as many indentifiers as our old genxref did. The new
genxref uses the LXR::Tagger module. This module dispatches each
source file to the correct index generator. The markup of the code
displayed to the user is handled by the LXR::Lang module. This means
that all that have to be done to add support for a new language is to
write a LXR::Lang::Foo and a LXR::Tagger::Foo module.