Re: dchanges and changes file formate

[Erick Branderhorst <branderhorst@fgg.eur.nl>]

Before we decide on some naming structure I would like bring in some
arguments and a new proposal as well.

From: Andy Guy <awpguy@acs.ucalgary.ca>

>A couple of points have come up in my disscussions with Bill:
>
>1. The need for parsing file names.  In this I propose a new standard
>format for filenames that are going to be uploaded so they can be
>easily parsed. [Similar to the message Bill sent last week].

Parsing file names is a much discussed item, I'm going to add an other
item, perhaps we can create an intermediar file name
convention. Packages hold there regular name at the developer site,
dchanges is run on them and renames/copies them into a parseble
format. Upload to some incoming, parsing and reshuffling into the
regular names and into specific subdirectories.

>2. Source package names - maintainers should be associated with them
>as well as the binary packages.  These mostly overlap binary package
>names but some don't (ncurses).  I am not sure if the bug system
>already manages this?

IMO the source packages shouldn't be changed (no debian. files added),
this isn't possible right now, but I think the future should. The
debian changes should be provided in a seperate diff file the same as
it is right now. Anyway source packages should remain their original
name and if this isn't the same in Debian, a symlink being the Debian
source name should link to original source. A source field should be
added to binary packages if it is another name.

>3. Problems of registering maintainers for source/binarys over
>multiple architectures.

I don't expect very much architecture specific maintaining required or
am I wrong on this one?

>4. Proper use of the Source: field in packages. ncurses binary
>packages fail to include a proper source field (and probably so do
>others, I was using ncurses to test the program); pine source produces
>a pico binary package with a different version - how do we deal with
>this?  I suggest that the Source: field be composed of two parts,
>first the source package name and second the source package version.
>If the Source field is absent, then the source name and version are
>the same as the binary package.  If the Source: field contains a
>single word, it is the source name, the source version is the same as
>the binary version, if the Source: field contains two words (ie space
>seperated) the second is the source package version.  This allows one
>to automatically find the source package associated with any binary
>package.  Dpkg will already acccept this (and nothing really uses the
>Source: field at the momemt).  So for pico - "Source: pine 3.91"

It should be able to specify more names in the source field. How the
field and it contents should look like can be discussed later. More
source packages for one binary are sometimes required or easier. The
tex system like it is right now might benifit from this
approach. Better packages might be created by merging a few or parts
from a few sources together. However this shouldn't be done too much
becuase it will clutter up the system if implemented wrong. If it is
needed the upstream packages should be revised probably. 

>5. Various other gotchas: Ian J.s need to upload the nondebin version
>of dpkg, kermit package.  It has to be flexible enough to allow for
>various exceptional conditions and future changes without breaking
>everything.

Providing URL's for all exceptionals might be a solution, the URL
might go to the file right away or to a informative document like
kermit.txt. I don't know how this should be implemented right now, but
it isn't the most important issue.

>The aim is produce a changes file that can be used by scripts at the
>ftp site to check and place files in the correct locations
>automatically.  The other half is the dchanges program, this does as
>many consistency checks as possible so that problems can be caught
>early and dealt with easily.  I hope that once it is completed every
>developer will use it to upload files.  

The ftp script which is on the upload site should be included in the
dchanges package to allow developers testing their created dchanges
file and according packages. This will prevent errors at the
maintainer site. The upload site will only receive correctly formatted
changes files and packages. I think that you shouldn't hope that every
developer will use it, we should require it (if it is completed).

>Filename format
>===============

As intermediate format ok, in case of final filename use the currently
in use scheme of subdirs for contrib, non-free, stable, unstable,
(local), binary-architectures, sources etc. and the currently in use
names with normal "not overloaded with hashes" filenames. Keep it
short. Dchanges (in combination with dpkg-deb) will name the files
before uploading and "Undchanges" (or whatever) will rename them on
the receiving site and move them to the proper place in the archive if
the upload was correct (md5sum, ls check &c). The files in the archive
will be informative like they are now, the files in the upload stage
will be parseble and everyone can see right away that those files (if
people want to do manual downloads from incoming or from a maintainer
straight away) aren't "undchanged" and not verified. Additionally with
this namingscheme all files can go in one upload directory.

>The new filename format would be:
><pkgname>--<version>[--<arch>]--<ext>
>where <version> includes the debian revision.
Good thing.
>All of <pkgname> <version> <arch> <ext> shouldn't include '--'. 

Making filenames too long is a bad thing, typo's will occur, they are
hard to remember, directory listing will be "hardly readable" by
humans. Managing it in subdir is a better approach.

>This allows all the information to be easily extracted from the
intermediate
>filename.

>There are 4 types of files that can be uploaded:
>Binary packages:
>These should include the <arch> component (lack implies it is an i386
>package for backwards compatability).

Backward compatibility will be gone anyway with this approach so we
might require i386 for consistency.

>Source files and source diff files:
>These should not include the <arch> component.
>eg: ncurses--1.9.8a-4--tar.gz
>    ncurses--1.9.8a-4--diff.gz

ncurses-1.9.8a.tar.gz (original source)
ncurses-1.9.8a-debian-diff-4.gz (debian specific changes)
ncurses-1.9.8a-diff-architecture-4.gz (architecture specific changes)
ncurses-1.9.8a-debian-diff-architecture-4.gz (debian & architecture
specific changes, this will probably never be required)

>Other files:
ok but a few of these are covered above
>    spiffy--8.9-2--txt (info about getting the sources because of some
>       copyright)
This will probably required.

>Also the <ext> component shouldn't lead to confusion with the other
>standard <ext>s (ie not be deb, tar.gz, or diff.gz).  But nothing
>stops it being x.diff.gz)

See above, intermediate format will be capable to handle almost if not
all possible situations.


>Format: format version number of the changes file (Bill's version was
>1.3, I have made a few changes so will call it 1.4)
Ok, if we agree on the above specified it should probably go to 2.0 :)

>Distribution: which distribution the files should be placed in.  I
>think this should be one of "developement", "stable", "non-free",
>"contrib" - this would be defined by the distribution maintainer.
Or local (no upload but for internal orginization), or unstable.

>Source: name of the source package
Source: names of the original source file(s)

If the sources are not uploaded the script should check if the md5sum
supplied in the changes file is the same as the sources files present
in the current archive. This will save unrequired reloads of the same
sources every time. Actually all not supplied files but listed in
changes file should be checked for if they are all ready present in
the archive.

Diff: names of the applied diffs

>Binary: space seperated list of binary package names uploaded

>Version: version of the source package (this should normally be the
>same as the version in all the binary files uploaded - note pico
>exception above)

Version of the debian specific stuff containing the source-version and
the debian-revision with a hash between them. Perhaps a Source-version
field can be included containing the version of the original source(s).

>Architecture: space seperated list of architectures of the uploaded
>files also implies that the maintainer for these architectures should
>be set.  It would be useful to add another dummy architecture
>"source", this would make it easier (or even possible) to identify the
>source maintainer.
Perhaps, but is this really necessary?

>Description: summary of the descriptions in the binary files uploaded
>or infomation supplied by the user
>
>Changes: the changes that have been made to the package since the last upload
>
>Files: one line per file uploaded:

Or the not uploaded unchanged files, if so a check should be made on
the file in the archive.

>Format: 1.3
>Source: spiffy
>Binary: spiffy spiffy-extra
>Architecture: alpha all
>Maintainer: Joe Bloggs <joe@bloggs.org>
>Changes: Joe is taking over
>
>This says that Joe Bloggs is taking over spiffy for archiectures alpha
>and all.  This suggests that one of binary packages spiffy or
>spiffy-extra is architecture independent.  It is also odd that Joe is
>taking over the "all" architecture but not the "source".
>
>Uploading a changes file with a blank maintainer orphans the package.
Seems ok to me.

>#define ARCHIVE_FILENAME_PATTERN "*.deb"
>to
>#define ARCHIVE_FILENAME_PATTERN "*deb"

A bit philosophic perhaps, but extension shouldn't go into the
filename part. Think about .tar.gz it tells you that the file in
question is tarred and then gziped. You might extract this information
from targz too, but you might interprete this as a file being "targ"ed
and "z"ed. However not very common it is confusing.

Erick
--
Erick Branderhorst@heel.fgg.eur.nl +31-10-4635142
Department of General Surgery (Intensive Care) University Hospital Rotterdam NL