All entries for Tuesday 16 October 2007

October 16, 2007

A few webpin improvements

I spent some time rewriting the backend for my package search over the weekend to

a) Improve indexing efficiency
b) Improve search result accuracy

Many of the repositories the search indexes change frequently, especially factory & packman. Updates to the index were starting to take unacceptably long, when indexing now about 208,000 packages, their descriptions & filelists.

With a few improvements such as skipping package details & files from packages which are unchanged after a repository update, redesigning the table/index structure. The indexer is now far faster and less i/o intensive when performing incremental updates. This should mean I can keep the search index more up to date, and be less annoying to other users of the server.

I also spent some time analysing the search queries, and used what I learnt to enable full search of the files within a package. Previously only a rough comparison to the filenames was made. Now you can even search for a full filename, like /opt/kde3/bin/kopete . Normal queries such as konversation or kde irc client will still work.

I made a few other small improvements too, e.g. filenames with binary matches will be ranked higher, so searches for something like kopete are a bit more useful, finding /opt/kde3/bin/kopete

The next thing on webpin TODO, after fixing all the problems that arise from these changes, is to make some frequently requested improvements to the web frontend, such as adding Install Now links to each search result. And perhaps alter the theme to fit with the new openSUSE-community theme .

If you prefer the command line to a web browser, check out yaloki’s command line client

October 2007

Mo Tu We Th Fr Sa Su
Sep |  Today  | Nov
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30 31            

Search this blog



Most recent comments

  • Hey skx, how do you do that ? I run into the fu….. problem, that curl terminate with unspec. error… by David on this entry
  • With some minor changes this will also work for updating from 11.0 to 11.1. by skx on this entry
  • You are the man!!! Great work and good documentation! It worked without any problem for me. Thanks a… by Vany on this entry
  • didn't work for me. In fact killed the system. Have to download 11.0 and burn to dvd to fix it. by maybe windows on this entry
  • Will this method work for 11 => 11.1 ? by Erik Jakobsen on this entry

Blog archive

Not signed in
Sign in

Powered by BlogBuilder