[Exherbo-dev] That categories thing again...

[Bernd Steinhauser]

So this topic has been around for several years without much progress... still I 
want to bring it up for discussion again.

If I remember correctly, one of the proposals was to use the homepage (or rather 
the second+top level domain) of the project as a way to handle packages on the 
filesystem layout.
In my proposal, i want to build upon that idea, but extent it a little bit 
further. First, I would like to avoid a strict use of domain names and instead 
talk of "providers". A provider could for example be a homepage (if nothing else 
makes sense), but it might be a simple organization name, like gnu, fdo, gnome 
or kde, too. As far as I can remember, the main issue with this approach was, 
that especially for large providers like fdo or kde, the number of packages 
associated to one provider (or homepage in the original terms) might be very 
large. I think directory listing was one of the performance issues Ciaran talked 
about (but as always I might be horribly wrong).
However, I think that there might be one solution, which introduces a quite 
natural solution for this problem. Large projects like kde tend to build up an 
organization structure to maintain packages in projects (i.e. like 
kdemultimedia), that most packages can be assigned to. Therefore I would like to 
introduce "subproviders" (or projects, if that might be a better name), creating 
one more level in the fs-structure. So a full package name would be 
${PROVIDER}/${SUBPROVIDER/PROJECT}/${PNVR}. This raises a few questions and/or 
problems and maybe even some opportunities, which should be discussed. I'm going 
to point the ones that came to my mind, but obviously, this list will be far 
from being complete...

1. Can ${SUBPROVIDER} be empty? The main question here is if we can mix 2-level 
and 3-level dep-specification. I don't know anything about the resolver 
internals, so I can't really discuss this one. This is very important for the 
feasibility of this approach, otherwise we would have to introduce bogus 
subproviders even were something like this does not make sense.

2. Can a 3-level dep-spec be used with the resolver at all? Or maybe a 
dep-specification like kde/kdemultimedia/libkcompactdisc feels to long for some 
people?

3. For an approach like this, we definitely need a proper way to handle package 
moves. There are two reasons for this:
     a) Providers can change, especially if the homepage has been set as a 
provider. But even if not, project renames happen quite often.
     b) Some providers like kde use special projects like for example 
"extragear" or "playground" for stuff that is somehow bound to the project but 
is in an experimental stage. When these reach a stable stage, they are moved to 
a different project (like for example kdemultimedia) at which point the package 
should be moved, too.
I'm not sure about the way package moves could be handled properly. The simplest 
solution that comes to my mind is using symlinks (or some other way to point to 
the new package name). So if a/b/c has been moved to a/d/c, a/b/c would contain 
a pointer which points to the new package. This could actually coexist with the 
old package name, if there are still old versions around.
So given that a/b/c contains
a/b/c -> a/d/c
a/b/c-1.8

and a/d/c
a/d/c-2.0

and a package depends on a/b/c[>1.9], the resolver might issue a warning or an 
error that no suitable match could be found for a/b/c[>1.9], but a/d/c-2.0 would 
fullfill the requirement and should be used instead. (Maybe this could even be 
taken automatically.)
This is just a rough idea and I guess it has horrible implications which I 
totally forgot about?

4. What happens for package splits? This is one topic which we have to deal for 
projects growing bigger and at some point deciding to split up. Actually, I 
think that this approach has advantages here. If something depends on 
kde/kdemultimedia[=scm] and this gets split up, the new package names are for 
example kde/kdemultimedia/libkcompactdisc. So if a package is converted to a 
subprovider, a dependency specification might be considered to be transitional, 
so that in this case the version spec would apply to all packages in the 
kdemultimedia-subprovider. The main question here is if it would be allowed that 
both the subprovider kde/kdemultimedia and the package (i.e. 
kde/kdemultimedia-4.9) may coexist. Actually, I guess they have to, but proper 
handling is necessary here.

5. I think using this approach, at least large providers can be split up into 
smaller portions and actually these portions should have a reasonable size, 
since their structure comes quite naturally from upstream (at least as long as 
upstream is sort of sane, which is not always the case).

6. One of the downsides might be that there could be *a lot* of small providers 
containing only 1 or 2 packages, which blows up the provider-tree. Here it would 
help, if actually don't rely on the homepage as a provider but use the source 
code provider or hoster instead. So quite a few projects might be categorized in 
github/, sourceforge/ or similar. The problem with these providers is that they 
usually do not provide a natural sub-structure, so here we have to ensure that 
we wouldn't blow them up with zillions of packages? I didn't check how big they 
would be, though.

7. From the user side, which package specifications might a user use (i.e. for 
cave resolve) for a desired package a/b/c?
Would all of these cases be ok?
cave resolve c
cave resolve a/c
cave resolve b/c

Maybe warning about ambiguous names, if necessary?


So now it's your turn to tell me about all the important issues I forgot and 
yell at me why this approach totally sucks. :-P
Anyway, comments welcome, especially about the feasibility of such an approach 
within the resolver. ;-)