[Exherbo-dev] [RFC] Rethinking fetchers

Alex Elsayed eternaleye at gmail.com
Wed Mar 14 18:13:19 UTC 2012


Mike Kazantsev <mk.fraggod at ...> writes:

> And fetching from stuff like bittorrent in general will greatly benefit
> from being parallel for all distfiles needed, because time to bootstrap
> download from a magnet link should be longer than to find a mirror and
> download the whole file from there.
> 
> Performance aside, having bittorrent fetcher as exlib can have a
> benefit that sha1 hash, currently passed only to for checksum.exlib,
> can be used to produce a magnet link as well, so all exherbo users can
> be used as an "exherbo mirror network", provided the link will be passed
> to some long-running daemon (like transmission), not oneshot leecher.
> 

You missed the part about Metalink files in my suggestion. They basically name a
list of mirrors (of arbitrary protocols, though HTTP(S) and FTP are the most
commonly used), designate chunks the file can be split into, give checksums
(which may be stronger than SHA1, such as SHA512) for both the individual chunks
and the whole file, allow downloading those chunks in parallel from the
individual mirrors, and it can even include (as part of the Metalink file) a
PGP/GPG signature for the file. Check out RFC 5854 for the spec, it's pretty
damn sweet.

http://tools.ietf.org/html/rfc5854




More information about the Exherbo-dev mailing list