Next generation package management.
Link to screenshots for the impatient.
It seems to me there are several generations of package management:
- 0th Generation – No package management, user must place individual files in the correct place on his/her system.
- 1st Generation – Software is packaged for easy installation on the target system, user can just extract/run the package and files are placed in the correct place, but there is no dependency checking. If something the programme requires to run is not available it will simply fail to work when run. Examples: old style windows installers, zip files, slackware packages.
- 2nd Generation – Builds on 1st Generation by adding dependency/prerequisite support, so user can know what prerequisites he/she must install in order to use the programme. Examples: MSI, dpkg, rpm.
- 3rd Generation – Builds on 2nd generation by adding automatic dependency resolution from software catalogues/sources/repositories. This means that if all the software & dependencies that the user might wish to install are available in the pre-configured repositories he/she can easily install anything. Examples: apt/smart/urpmi/yast sw management .
We seem to have remained at the same basic concept of 3rd generation package management for quite a while. However, there are a remaining issues that stop it from being as useful as it could be. Problems I see with 3rd generation package management:
- User has to learn how to use a package management frontend. Be it apt-get, synaptic, yast, smart, or whatever. The design of these interfaces vary in ease of use, but even the best designed may still seem foreign to windows users conditioned into double clicking an install file and clicking next repeatedly until done.
- The packages may not be in the standard preconfigured repositories. This can happen both for legal and practical reasons. Some software is not legal to distribute freely everywhere in the world, and no repository can contain all software in any case. In the situation where the software the user wishes to install is not available in the already configured repositories, he or she will have to locate the repository where the software is located, and then learn how to add it as a source to whichever package manager he/she is using. All of which is a lot of effort for someone who just wanted to install something.
So, how can we solve these issues?
Well one solution is to make use of some form of meta-package file which simply contains information about software, and the repositories where it can be located. On top of a 3rd generation package management system this could enable:
- Single click adding of software catalogues, without the user having to worry about where they are located, or how to add them.
- Single click installation of software, whether the repository the software is located in is currently configured in the user’s package manager.
Today I had some spare time so I knocked up a proof of concept for how this could work:
Using an xml file in a format like
With a YaST module to interpret the above, and a default association for the file type in my file manager/web browser I have a working proof of concept for easy package installation.
Here is an example file to install tuxsaver from packman
So the user sees a link to “install tuxsaver” on a website or otherwise obtains the above installation file. And decides he/she wishes to install it. User clicks the link, and the appropriate repository can be added and the appropriate packages installed.
Alternatively if no packages are specified for installation, the same file handler could act as an easy way to add additional package repositories to the package manager.
If the module were opened without a file as a parameter it could grab a regularly updated list of available software catalogues from a central location. Obviously repositories hosting packages with dubious legality could not be listed here. However it would enable a project such as packman which is in this situation to host a “Click here to add our project catalogue” link on their homepage, and a “Click here to add mp3 support to your SUSE Linux” etc.
Link to the ycp for this proof of concept.
I think a proper implementation of this concept, included in the distro install by default would have potential to greatly simplify the package management experience for users.
A few issues were raised while I was looking at this which will need addressing:
- de-duping repositories.
There is no sense in a user having the same package repository added to their package management system twice. My demo will only re-add a repository if the same URL hasn’t already been added, but the same repository may have different URLs on different mirrors. There seems to be no unique repository identifier in rpm-md repositories. One possibility is using the md5sum of one of the metadata files, but this would require making several checks, and ensuring the repository metadata is synced before the process can start.
- Security issues.
This demo gets for free the security warning from YaST when adding an untrusted repository. It is likely a more clear warning will be required of both the security and technological issues that can be caused by installing random software.
Clearly the format of the meta-package file would need to be thought about carefully. What information does it need to contain, can it be bundled with packages to allow third parties to distribute RPMs which depend on packages in other people’s repositories.
Hopefully this demo might explain the concept better than my previous attempts, and might inspire some real improvements in user experience in future distributions.
04 Dec 2006 19:05
| 
Tags: Management Opensuse Package Suse
| 
Comments (7)
| 
Report a problem
Loading…
7 comments by 1 or more people
[Skip to the latest comment]asker
.
Have you ever looked at klik ?
(http://klik.atekon.de/)
06 Dec 2006, 13:13
@asker
Yes I have, There are also things like autopackage I didn’t mention. klik is useful for trying out software.
In my above categories I would define klik as 1st generation package management. On the server side there is more complex logic to generate the package the user sees, but from a user point of view it is 1st generation.
As the klik FAQ states it is not intended to replace traditional package management. Using it for more than intended will negate the benefits of package management, which I have not really covered above. It is also only really suited to self contained applications, such applications could be equally easily installed by a archive with a wizard installer.
06 Dec 2006, 18:10
asker
.
autopackage and klik do complement each other nicely.
autopackage developers have created some excellent tools to build (=compile) software in a way that circumvents some nasty C++ binary incompatibilities.
klik developers have created a way to make it really easy to users to access new packages (precompiled by .rpm or .deb or .autopackage packagers). No, it’s not just for “trying out”. (Though you can start using klik for trying out, and it’s great for that as well.)
Do you see the similarities of klik and the Mac OS X (or even the NeXT) way of handling software packages? “AppDirs”? Do you think Apple uses their packaging technology only for “trying out” software??
I myself would not subsum klik amongst the 1st generation of packaging software. It rather is on its way to become the 4th generation (and as such, embodies parts of the previous generaations as well). klik already utilizes (on the server side) a very advanced and automatic dependency resolution (they call it “server-side apt”). If LSB 3.x or 4.x sometime soon creates a good and complete definition of a standard system, klik will work on each distro that complies with that standard. And it will not matter if the $INPUT of a klik recipe are RPM, .deb, .tar.gz or .tbz2 archives—the output will be a .cmg bundle that can be copied to any system and started there out-of-the-box.
The basic idea and the principles of klik are very sound. What’s missing is just implementation details, and broad adoption. However, that may never happen. Linux and Unix folks sometimes are the most conservative creatures on this planet…. Sigh.
06 Dec 2006, 21:18
Pascal Bleser
@asker: klik is flawed by design for anything else than openoffice or gimp. Forget it, you’re overlooking all the nitty gritty behind packaging, dependencies and such.
06 Dec 2006, 21:29
asker
@Pascal:
Are you aware that klik packages are not meant to work only for OOo and Gimp (actually—klik-ed Gimp does not currently work, and never did, due to Gimp’s inherent “non-relocateability”!)? But that it actually works for Acroread, RealPlayer, Picasa, GoogleEarth, Skype, WengoPhone, Nero, Opera, Xara LX, NVU, yakuake, KOffice_complete (or Krita_single, KWord_single, Karbon_single,....), Gambas, Eric3, Amarok, Apollon, Gaim, GCompris, PuTTY, TaskJuggler, Stellarium, Twinkle, Wesnoth….. I won’t mention that I even have built a (private) klik package of “Wine+InternetExplorer6+someGoodies” that I can take and run on ever Linux system of mine (but for obvious licensing reasons I can’t distribute it)?
Uhmmm… above is the complete list of the applications I currently use week by week, running from an USB stick, packaged in the klik format.
klik works and is feasible for virtually every single end-user application with a GUI. (I know how to build and use .cmgs also for console programs such as ‘scli’, but these are not so easy for end-users to use.). klik does (of course) not work (or work well) for server programs (needing access to ports <= 1024) or for libraries. That’s why klik wants a good standard and definition for a “Linux Base System” in LSB 3.x or 4.x.
In short: you got it wrong, Pascal. Have you ever run klik, and looked at its merits from the point of view of an ordinary user?
Or is your conclusion based on what you read about klik (plus your undoubtedly rich and enormous experience and knowledge as an RPM packager)?
Could you (at least) outline which design flaws you found in klik?
07 Dec 2006, 09:24
Paulo Trezentos
The new generation of meta-installers will be probably the 3rd generation with steroids:
1 – Dependency solving in a really effective way. Just for the case you didn’t notice yast, portage, yum and apt just don’t find a solution where sometimes it is possible to do so.
2 – P2P method for downloading files
3 – Rollback in a effective way
For further information check http://gsyc.es/~herraiz/fosdem2007/papers/NewGenMI_tools.pdf and EDOS Project Workpackage 2.
23 May 2007, 22:47
@Paulo
Interesting paper, have you researched openSUSE’s approach to driver packages? http://www.suse.de/~agruen/KMPM/KernelModulePackagesManual-CODE10.pdf . These depend on both a specific kernel version and can also depend on the hardware being present. http://en.opensuse.org/Software_Management/Dependencies/Hardware. libzypp allows packages to depend on any system property as well as other packages. KMPs have proven effective at allowing people to use nvidia, ati, and madwifi drivers from third parties without breaking on kernel update.
By the way you need to replace “week” with “weak” at the end of page 4 of your paper.
24 May 2007, 07:00
Add a comment
You are not allowed to comment on this entry as it has restricted commenting permissions.