All entries for Tuesday 16 October 2007
October 16, 2007
I spent some time rewriting the backend for my package search over the weekend to
a) Improve indexing efficiency
b) Improve search result accuracy
Many of the repositories the search indexes change frequently, especially factory & packman. Updates to the index were starting to take unacceptably long, when indexing now about 208,000 packages, their descriptions & filelists.
With a few improvements such as skipping package details & files from packages which are unchanged after a repository update, redesigning the table/index structure. The indexer is now far faster and less i/o intensive when performing incremental updates. This should mean I can keep the search index more up to date, and be less annoying to other users of the server.
I also spent some time analysing the search queries, and used what I learnt to enable full search of the files within a package. Previously only a rough comparison to the filenames was made. Now you can even search for a full filename, like /opt/kde3/bin/kopete . Normal queries such as konversation or kde irc client will still work.
I made a few other small improvements too, e.g. filenames with binary matches will be ranked higher, so searches for something like kopete are a bit more useful, finding /opt/kde3/bin/kopete
The next thing on webpin TODO, after fixing all the problems that arise from these changes, is to make some frequently requested improvements to the web frontend, such as adding Install Now links to each search result. And perhaps alter the theme to fit with the new openSUSE-community theme .
If you prefer the command line to a web browser, check out yaloki’s command line client