[fhs-discuss] FHS/LSB Updates

Hi Bruce,

Post by Bruce Dubbs
The last I looked, LSB Core required several programs that seem to
me to be obsolete: 'cpio', 'ed', 'at', and 'batch' come to mind.
I have not seen those in active use in the last 10 years or so.

What, obsolete?

- at(1) and batch(1) are POSIX and not marked obsolete.
I use them daily at work, and we ship them, enabled,
to tens of thousands of customer machines on SLES 11.

- While i'm not sure whether there is a standard for cpio(1)
except the LSB, i consider a tool people have get used to.
I use at about monthly, most often in conjunction with
rpm2cpio(1).

- While ed(1) is arguably used rarely, it's POSIX as well,
it is not marked obsolete either, and there are still systems
around where it's the only editor you have, e.g. the OpenBSD
installation image.

Post by Bruce Dubbs
Occasionally I have seen ed or cpio them used when trying to install
a very dated package (last century), but I do not recall any use of
'at' or 'batch'.
I'm not suggesting that distros drop them, but I wouldn't think that
people who rely on the FHS/LSB would really need them.

Well, as far as i understand, the LSB builds on POSIX, so to have
ed, at, and batch around is not an additional requirement imposed
by the FHS.

Yours,
Ingo

Mats Wichmann

2011-11-16 19:30:09 UTC

Post by Ingo Schwarze
Hi Bruce,

What, obsolete?
- at(1) and batch(1) are POSIX and not marked obsolete.
I use them daily at work, and we ship them, enabled,
to tens of thousands of customer machines on SLES 11.

indeed, if you install the package for Chrome, it fires off an at job so
even "modern" applications may use it :)

Post by Ingo Schwarze
- While i'm not sure whether there is a standard for cpio(1)
except the LSB, i consider a tool people have get used to.
I use at about monthly, most often in conjunction with
rpm2cpio(1).
- While ed(1) is arguably used rarely, it's POSIX as well,
it is not marked obsolete either, and there are still systems
around where it's the only editor you have, e.g. the OpenBSD
installation image.

Post by Bruce Dubbs
Occasionally I have seen ed or cpio them used when trying to install
a very dated package (last century), but I do not recall any use of
'at' or 'batch'.

I'm quite willing to admit to being a dinosaur but I do use cpio from
time to time to make sure a transfer of a file tree gets done as I want.
Although I believe rsync (which is not in LSB) does just as well.

There was some effort to try to converge tar and cpio into pax, such
that the requirements would be "one required tool, correctly handles
both file formats". I don't think that really got as much traction as
the people who named it thought it would.

Post by Ingo Schwarze

Post by Bruce Dubbs
I'm not suggesting that distros drop them, but I wouldn't think that
people who rely on the FHS/LSB would really need them.

Well, as far as i understand, the LSB builds on POSIX, so to have
ed, at, and batch around is not an additional requirement imposed
by the FHS.

That's correct for LSB; continuing to track POSIX and it does
occasionally obsolete commands. FHS doesn't require LSB though, it's
more the other way around.

Ingo Schwarze

2011-11-16 19:42:21 UTC

Hi,

Post by Mats Wichmann
There was some effort to try to converge tar and cpio into pax, such
that the requirements would be "one required tool, correctly handles
both file formats". I don't think that really got as much traction
as the people who named it thought it would.

For what it's worth, /bin/pax, /bin/tar and /bin/cpio have been
hard links to the same statically linked executable (such that
they work even when /usr is not mounted and ld.so is broken)
since 1997 on OpenBSD.

So anybody who wants to use that code is free to grab it here,
of course with a GPL-compatible license (3-clause BSD):

http://www.openbsd.org/cgi-bin/cvsweb/src/bin/pax/

Yours,
Ingo

Bruno Cornec

2011-11-16 19:58:58 UTC

And initrd/initramfs is based on cpio content at least on my Mandriva
distro.
$ sudo gzip -cd /boot/initrd-2.6.33.7-desktop-2mnb.img | file -
/dev/stdin: ASCII cpio archive (SVR4 with no CRC)

Bruno.

--
Open Source & Linux Profession Lead EMEA http://opensource.hp.com
Open Source Solutions Initiative Chief Technologist http://hpintelco.net
FLOSS projects: http://mondorescue.org http://project-builder.org
Musique ancienne? http://www.musique-ancienne.org http://www.medieval.org

Steve Langasek

2011-11-16 22:19:00 UTC

Post by Bruno Cornec

And initrd/initramfs is based on cpio content at least on my Mandriva
distro.

Yes, because this is the format supported by the kernel.

Changing the LSB to tell vendors they can't rely on the availability of cpio
therefore seems pointless to me.

Bruce, is there a particular reason you expect a system that would otherwise
be LSB-compliant to not have these tools available?
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slangasek at ubuntu.com vorlon at debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: Digital signature
URL: <http://lists.linuxfoundation.org/pipermail/fhs-discuss/attachments/20111116/1abd454f/attachment-0001.sig>

Jan Engelhardt

2011-11-16 21:13:14 UTC

Post by Ingo Schwarze

What, obsolete?
- at(1) and batch(1) are POSIX and not marked obsolete.
I use them daily at work, and we ship them, enabled,
to tens of thousands of customer machines on SLES 11.

at is /the/ counterpart to cron, especially for non-recurring jobs,
so indeed, it should not go.

Since LSB wants rpm, and rpm uses the cpio format, it sure would
be considered well-thought to keep cpio as well.

Post by Ingo Schwarze
- While ed(1) is arguably used rarely, it's POSIX as well,
it is not marked obsolete either, and there are still systems
around where it's the only editor you have, e.g. the OpenBSD
installation image.

And at times, one should consider that POSIX... does not always
do things right (e.g. the 512-byte block size, which humans are
not excelling in doing calculations with - 1024 or so would have
made more sense in that regard).

If ed is wanted, perhaps LSB should only include it by reference
to POSIX, and not explicitly include it in LSB.

Julian Fagir

2011-11-16 23:26:09 UTC

Hi,

Post by Jan Engelhardt

And at times, one should consider that POSIX... does not always
do things right (e.g. the 512-byte block size, which humans are
not excelling in doing calculations with - 1024 or so would have
made more sense in that regard).
If ed is wanted, perhaps LSB should only include it by reference
to POSIX, and not explicitly include it in LSB.

vi is often (or always?) placed in /usr/bin, and /usr is on a separate
partition. At least this is useful for all the source distributions which
place source files in /usr. Or another setup, a small root on the disk
with /usr being mounted over nfs...
There are many scenarios, and they all exclude vi from being in /. Thus,
having at least some editor in / without mounting another (perhaps not even
working) partition is useful, and exactly.

Regards, Julian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <http://lists.linuxfoundation.org/pipermail/fhs-discuss/attachments/20111117/4728379e/attachment.sig>

Jan Engelhardt

2011-11-17 00:58:36 UTC

Post by Julian Fagir

Post by Jan Engelhardt
If ed is wanted, perhaps LSB should only include it by reference
to POSIX, and not explicitly include it in LSB.

vi is often (or always?) placed in /usr/bin, and /usr is on a separate
partition.

So.. I don't quite see the connection of ed to vi being in /usr.

Post by Julian Fagir
There are many scenarios, and they all exclude vi from being in /.

Counter-example: openSUSE which ships vim in /bin.

Bruce Dubbs

2011-11-17 02:33:36 UTC

Post by Jan Engelhardt

Post by Julian Fagir

Post by Jan Engelhardt
If ed is wanted, perhaps LSB should only include it by reference
to POSIX, and not explicitly include it in LSB.

vi is often (or always?) placed in /usr/bin, and /usr is on a separate
partition.

So.. I don't quite see the connection of ed to vi being in /usr.

Post by Julian Fagir
There are many scenarios, and they all exclude vi from being in /.

Counter-example: openSUSE which ships vim in /bin.

Many of the legacy decisions have been made when disk drives were
expensive and small. I remember being excited when I bought my first
hard drive: 80MB for the new low price of $600. That was somewhere
around 1985.

Today, I don't think you can get a new rotational disk drive that is
less than 250GB (for about $40 to $60). Even thumb drives are 16GB for
less than $20. The economics of HW today does have an impact on the
disk layout. What was appropriate in 1990 is not appropriate today. I
think the FHS should reflect that.

I have seen discussion about removing /usr/bin completely and putting
everything on /usr. There are multiple distributions that today say
that they don't support a separate (or at least a remote) /usr partition.

What the FHS/LSB should be about is to not only set a standard about
what facilities are available for a product today, but to provide a
roadmap to what will be available in the future. This means that
programs that are of marginal use should be deprecated.

In the days when memory is measured in GB and disk in TB, things like vi
in /bin and vim in /usr/bin is nonsensical. The same can be said for
the differences between dash and bash.

I don't know of any *programs* that rely on ed/at/batch. Sure users can
and do use them, but do they need to be part of a standard?

-- Bruce

Dr. Werner Fink

2011-11-17 12:17:36 UTC

Post by Bruce Dubbs
I don't know of any *programs* that rely on ed/at/batch. Sure users
can and do use them, but do they need to be part of a standard?

Not a program but the official configuration script of TeXLive
uses beside sed the ed text editor for modifying configuration
files.

Werner

--
"Having a smoking section in a restaurant is like having
a peeing section in a swimming pool." -- Edward Burr

Mats Wichmann

2011-11-17 14:33:29 UTC

I don't know of any *programs* that rely on ed/at/batch. Sure users can
and do use them, but do they need to be part of a standard?

yes... the point is what install and startup scripts can rely on, as
mentioned the other day the "postinstall" script of Google Chrome uses
at to schedule an upgrade check (maybe I wouldn't have chosen to do it
this way), but can only do so because it has a high degree of confidence
it will be found. Similarly for the example cited of ed being used from
a script to "localize" a configuration.

It's not that your points in the rest of the message are invalid, it's
clear a lot has changed and some choices don't seem useful any longer.
But it's always hard to shed legacy stuff, as you can see from the
various arguments that have arisen over efforts to do so - "system V"
style initscripts vs. systemd is a current one. Maybe this is a useful
transition, but what happens to apps packaged with initscripts? (this
is an LSB concern). Will they continue to work properly through
transition mechanisms, or will they just break?

The difficulty for LSB and FHS is trying to describe what systems
actually do, and to not unreasonably constrain progress forward, but at
the same time provide a little consistency for people (app developers,
users, etc.). Not saying we get that balancing act right all the time,
but discussions like this one are healthy in keeping it in the foreground.

Julian Fagir

2011-11-17 11:27:07 UTC

Hi,

Post by Bruce Dubbs

Post by Jan Engelhardt

Post by Julian Fagir

Post by Jan Engelhardt
If ed is wanted, perhaps LSB should only include it by reference
to POSIX, and not explicitly include it in LSB.

vi is often (or always?) placed in /usr/bin, and /usr is on a separate
partition.

So.. I don't quite see the connection of ed to vi being in /usr.

Post by Julian Fagir
There are many scenarios, and they all exclude vi from being in /.

Counter-example: openSUSE which ships vim in /bin.

This is no counter-example for that scenario. These scenarios do exist.
What I was talking about: The system should have a minimal set of tools to be
usable when everything else fails.

Post by Bruce Dubbs
Many of the legacy decisions have been made when disk drives were
expensive and small. I remember being excited when I bought my first
hard drive: 80MB for the new low price of $600. That was somewhere
around 1985.
Today, I don't think you can get a new rotational disk drive that is
less than 250GB (for about $40 to $60). Even thumb drives are 16GB for
less than $20. The economics of HW today does have an impact on the
disk layout. What was appropriate in 1990 is not appropriate today. I
think the FHS should reflect that.

this is not about disk space, this is about usage scenarios.
Having an external (via nfs) /usr does not need to be for disk space reasons,
but for management purposes.
Having an extra filesystem for /usr could be for many reasons, having many
source files there is one of them. The source-building systems like portage,
ports and pkgsrc (though ports is a BSD thing) have many small files, thus
it is better having another inode/block ratio than usual.

Or encrypting /usr, or whatever you want to do with it.
There are many reasons to make /usr an extra partition.

Post by Bruce Dubbs
I have seen discussion about removing /usr/bin completely and putting
everything on /usr. There are multiple distributions that today say
that they don't support a separate (or at least a remote) /usr partition.
What the FHS/LSB should be about is to not only set a standard about
what facilities are available for a product today, but to provide a
roadmap to what will be available in the future. This means that
programs that are of marginal use should be deprecated.
In the days when memory is measured in GB and disk in TB, things like vi
in /bin and vim in /usr/bin is nonsensical. The same can be said for
the differences between dash and bash.

That's a separate discussion going on, removing /usr completely or requiring
it to be in /bin would change this situation.

Post by Bruce Dubbs
I don't know of any *programs* that rely on ed/at/batch. Sure users can
and do use them, but do they need to be part of a standard?

With vi and possibly all other editors being on separate partition, / would
be completely without an interactive editor. And there are situations in
which you only have that choice, even on the most modern hardware.

Regards, Julian
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://lists.linuxfoundation.org/pipermail/fhs-discuss/attachments/20111117/83678c6a/attachment.sig>

Steve Langasek

2011-11-17 21:25:47 UTC

Yes, there are. These distributions have effectively decided to not support
the FHS.

That's their right, but we shouldn't change the FHS every time a
distribution decides it's too much work to comply with a standard.

Post by Bruce Dubbs
What the FHS/LSB should be about is to not only set a standard about
what facilities are available for a product today, but to provide a
roadmap to what will be available in the future. This means that
programs that are of marginal use should be deprecated.

The FHS and LSB are not the same thing. This is, in fact, the wrong list on
which to discuss the LSB.

The FHS has always been about defining a standard, cross-platform filesystem
layout based on a handful of simple principles such that distributions,
admins, and providers of third-party software all know what to expect from a
compliant system. As such, the cost of changing the spec to reverse
existing rules is high for everyone involved, and should only be entertained
when those rules give an unambiguously *wrong* result.

The LSB, on the other hand, is entirely about providing a platform to ISVs,
and does need to be responsive to their real needs.

Post by Bruce Dubbs
In the days when memory is measured in GB and disk in TB, things
like vi in /bin and vim in /usr/bin is nonsensical.

What's your point? Are you arguing that the LSB should include vim? If you
think vim is reasonable to rely on, there's no reason not to also include
ed/at/batch, which are all much smaller than vim.

Post by Bruce Dubbs
The same can be said for the differences between dash and bash.

Only by someone who doesn't know what they're talking about.

Post by Bruce Dubbs
I don't know of any *programs* that rely on ed/at/batch. Sure users
can and do use them, but do they need to be part of a standard?

You've already been given an example of a third-party package that uses at.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slangasek at ubuntu.com vorlon at debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: Digital signature
URL: <http://lists.linuxfoundation.org/pipermail/fhs-discuss/attachments/20111117/f3896088/attachment.sig>

Jon Dowland

2011-11-18 14:10:44 UTC

Post by Steve Langasek
The FHS and LSB are not the same thing. This is, in fact, the wrong list on
which to discuss the LSB.

This is a key point.

Post by Bruce Dubbs
I don't know of any *programs* that rely on ed/at/batch. Sure users
can and do use them, but do they need to be part of a standard?

You've already been given an example of a third-party package that uses at.

Just about every program named in the FHS as it stands has been defended by
someone.

What I don't understand is, why are there *any* programs guaranteed to be
present in the *Filesystem Heirarchy Standard*? We don't make any guarantees
about what the programs actually do, beyond a very brief english description,
("mounts filesystems"), not whether/how they work. One could provide a
mount(8) which was totally incompatible with the conventional mount(8) and be
FHS-compliant, and utterly useless. (perhaps you would be POSIX-incompliant,
in which case, POSIX is already covering the "mount is available and works"
requirement, so why does the FHS mention it?)

I can understand a vendor writing software for systems which are LSB-compliant
and therefore the LSB specifiying the availability of certain standard bits of
software (possibly via inheriting POSIX, in which case it does not need
repeating in a downstream standard). But what has that got to do with the
filesystem layout? Is this not entirely the wrong standard to stick this stuff
in?

--
Jon Dowland

Bill Nottingham

2011-11-21 21:36:25 UTC

Yes, there are. These distributions have effectively decided to not support
the FHS.
That's their right, but we shouldn't change the FHS every time a
distribution decides it's too much work to comply with a standard.

I'll bite. How is /bin as a symlink invalid in the FHS? The spec
specifically allows it.

Bill

Steve Langasek

2011-11-23 06:21:36 UTC

Yes, there are. These distributions have effectively decided to not support
the FHS.
That's their right, but we shouldn't change the FHS every time a
distribution decides it's too much work to comply with a standard.

I'll bite. How is /bin as a symlink invalid in the FHS? The spec
specifically allows it.

FHS chapter 3:

* To boot a system, enough must be present on the root partition to mount
other filesystems. This includes utilities, configuration, boot loader
information, and other essential start-up data. /usr, /opt, and /var are
designed such that they may be located on other partitions or
filesystems.
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slangasek at ubuntu.com vorlon at debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: Digital signature
URL: <http://lists.linuxfoundation.org/pipermail/fhs-discuss/attachments/20111123/3bd84e49/attachment.sig>

Bill Nottingham

2011-11-23 19:12:23 UTC

Yes, there are. These distributions have effectively decided to not support
the FHS.
That's their right, but we shouldn't change the FHS every time a
distribution decides it's too much work to comply with a standard.

I'll bite. How is /bin as a symlink invalid in the FHS? The spec
specifically allows it.

* To boot a system, enough must be present on the root partition to mount
other filesystems. This includes utilities, configuration, boot loader
information, and other essential start-up data. /usr, /opt, and /var are
designed such that they may be located on other partitions or
filesystems.

Well, we're already violating that in most distributions where /boot is
allowed to be separate. Aside from that point, is there anything that
prevents updating this section? For example, the references to restoring from
potential backups to floppies...

We've already designed a requirement where, in cases where /usr may be
separate, or not local, the initramfs must contain all the tools to properly
mount such filesystems. At that point, the distinction of the 'root'
filesystem being the initramfs could be a point to discuss.

Bill

Jan Engelhardt

2011-11-23 21:16:01 UTC

Post by Steve Langasek
* To boot a system, enough must be present on the root partition to
mount other filesystems. This includes utilities, configuration,
boot loader information, and other essential start-up data. /usr,
/opt, and /var are designed such that they may be located on
other partitions or filesystems.

Well, we're already violating that in most distributions where /boot is
allowed to be separate.

In what way? /boot usually contains no files required after kernel
boot (only short of System.map for use with ksyslogd), since the
initramfs image is in memory already.

What was probably intended to be said what: requiring the root
partition to carry tools to mount /usr is optional if this mountpoint
was already dealt with in initramfs. But then again, one has to
consider that one may want to umount or remount /usr, or any
other volume for that matter, and therefore, the mount utility
is needed in the root fs anyway, since the initramfs is gone by that
time.

Post by Bill Nottingham
We've already designed a requirement where, in cases where /usr may be
separate, or not local, the initramfs must contain all the tools to properly
mount such filesystems. At that point, the distinction of the 'root'
filesystem being the initramfs could be a point to discuss.

I don't find FHS's wording wrong; initramfs is not something that FHS
should need to care about, ergo, "root filesystem" would also imply
mounts available prior to /.

Steve Langasek

2011-11-23 23:17:16 UTC

Post by Steve Langasek
Yes, there are. These distributions have effectively decided to not
support the FHS.
That's their right, but we shouldn't change the FHS every time a
distribution decides it's too much work to comply with a standard.

I'll bite. How is /bin as a symlink invalid in the FHS? The spec
specifically allows it.

* To boot a system, enough must be present on the root partition to
mount other filesystems. This includes utilities, configuration,
boot loader information, and other essential start-up data. /usr,
/opt, and /var are designed such that they may be located on other
partitions or filesystems.

Well, we're already violating that in most distributions where /boot is
allowed to be separate.

Yep, I noticed that as I was quoting it. That's clearly a bug in the FHS;
even at the time the FHS was being drafted, it was common practice to have
bootloader configuration on a separate boot partition in some cases (or to
have it in the MBR which is not a partition at all) due to various
limitations.

So this is a case where I think the FHS clearly needs to be fixed because
following it as written gives the wrong result.

However, the same can not be said for the handling of /usr. The proposal
currently on the table for Fedora to make /bin a symlink to /usr/bin is
altogether contrary to the FHS design and discards several invariants that
Kay as udev upstream has decided are not interesting to him, but that are
nevertheless interesting to others who have committed to this standard.

Post by Bill Nottingham
Aside from that point, is there anything that prevents updating this
section? For example, the references to restoring from potential backups
to floppies...
We've already designed a requirement where, in cases where /usr may be
separate, or not local, the initramfs must contain all the tools to properly
mount such filesystems.

In this context, is "we" Fedora, or someone else? I'm aware of
<https://fedoraproject.org/wiki/Features/UsrMove>, but don't know of its
current status in Fedora or elsewhere. Is this now considered a
requirement, rather than a proposal?
--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slangasek at ubuntu.com vorlon at debian.org
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 828 bytes
Desc: Digital signature
URL: <http://lists.linuxfoundation.org/pipermail/fhs-discuss/attachments/20111123/7aa646dd/attachment.sig>

Mats Wichmann

2011-11-24 13:49:37 UTC