Bug#996019: debootstrap: bootstrap of ubuntu impish is failing

Discussion:

(too old to reply)

Sylvestre Ledru

2021-10-10 10:50:01 UTC

Package: debootstrap
Version: 1.0.124
Severity: normal

Dear Maintainer,

$ sudo debootstrap impish impish https://www-ftp.lip6.fr/pub/linux/distributions//Ubuntu/archive/
I: impish uses zstd compression, setting --extractor=ar option
I: Retrieving InRelease
I: Retrieving Packages
I: Validating Packages
[...]
I: Validating zlib1g 1:1.2.11.dfsg-2ubuntu7
I: Chosen extractor for .deb packages: ar
I: Extracting base-files...
I: Extracting base-passwd...
E: Extracting .//var/cache/apt/archives/base-passwd_3.5.51_amd64.deb requires the zstdcat command, which is not available

adding --include=zstd doesn't fix the issue
[...]
I: Retrieving zstd 1.4.8+dfsg-2.1
I: Validating zstd 1.4.8+dfsg-2.1
I: Chosen extractor for .deb packages: ar
I: Extracting base-files...
I: Extracting base-passwd...
E: Extracting .//var/cache/apt/archives/base-passwd_3.5.51_amd64.deb requires the zstdcat command, which is not available

Thanks,
Sylvestre

-- System Information:
Debian Release: bookworm/sid
APT prefers unstable
APT policy: (600, 'unstable'), (500, 'oldstable-updates'), (500, 'oldoldstable'), (500, 'stable'), (500, 'oldstable'), (300, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 5.10.0-8-amd64 (SMP w/8 CPU threads)
Kernel taint flags: TAINT_WARN, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages debootstrap depends on:
ii wget 1.21-1+b1

Versions of packages debootstrap recommends:
pn arch-test <none>
ii debian-archive-keyring 2021.1.1
ii gnupg 2.2.27-2

Versions of packages debootstrap suggests:
ii binutils 2.37-7
pn squid-deb-proxy-client <none>
pn ubuntu-archive-keyring <none>
ii xz-utils 5.2.5-2
pn zstd <none>

-- no debconf information

Dimitri John Ledkov

2021-10-10 14:30:01 UTC

Permalink

Post by Sylvestre Ledru
Package: debootstrap
Version: 1.0.124
Severity: normal
Dear Maintainer,
$ sudo debootstrap impish impish

https://www-ftp.lip6.fr/pub/linux/distributions//Ubuntu/archive/

Post by Sylvestre Ledru
I: impish uses zstd compression, setting --extractor=ar option
I: Retrieving InRelease
I: Retrieving Packages
I: Validating Packages
[...]
I: Validating zlib1g 1:1.2.11.dfsg-2ubuntu7
I: Chosen extractor for .deb packages: ar
I: Extracting base-files...
I: Extracting base-passwd...
E: Extracting .//var/cache/apt/archives/base-passwd_3.5.51_amd64.deb

requires the zstdcat command, which is not available

Post by Sylvestre Ledru
adding --include=zstd doesn't fix the issue

One needs to install zstd on the host, including zstd in the debootstrap
set will not help.

Does $ sudo apt install zstd => fix the issue for you?

Post by Sylvestre Ledru
[...]
I: Retrieving zstd 1.4.8+dfsg-2.1
I: Validating zstd 1.4.8+dfsg-2.1
I: Chosen extractor for .deb packages: ar
I: Extracting base-files...
I: Extracting base-passwd...
E: Extracting .//var/cache/apt/archives/base-passwd_3.5.51_amd64.deb

requires the zstdcat command, which is not available

Post by Sylvestre Ledru
Thanks,
Sylvestre
Debian Release: bookworm/sid
APT prefers unstable
APT policy: (600, 'unstable'), (500, 'oldstable-updates'), (500,

'oldoldstable'), (500, 'stable'), (500, 'oldstable'), (300, 'experimental')

Post by Sylvestre Ledru
Architecture: amd64 (x86_64)
Kernel: Linux 5.10.0-8-amd64 (SMP w/8 CPU threads)
Kernel taint flags: TAINT_WARN, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=fr_FR.UTF-8, LC_CTYPE=fr_FR.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
ii wget 1.21-1+b1
pn arch-test <none>
ii debian-archive-keyring 2021.1.1
ii gnupg 2.2.27-2
ii binutils 2.37-7
pn squid-deb-proxy-client <none>
pn ubuntu-archive-keyring <none>
ii xz-utils 5.2.5-2
pn zstd <none>
-- no debconf information

Sylvestre Ledru

2021-10-11 16:20:01 UTC

Permalink

Hello

Post by Sylvestre Ledru
Package: debootstrap
Version: 1.0.124
Severity: normal
Dear Maintainer,
$ sudo debootstrap impish impish https://www-ftp.lip6.fr/pub/linux/distributions//Ubuntu/archive/
I: impish uses zstd compression, setting --extractor=ar option
I: Retrieving InRelease
I: Retrieving Packages
I: Validating Packages
[...]
I: Validating zlib1g 1:1.2.11.dfsg-2ubuntu7
I: Chosen extractor for .deb packages: ar
I: Extracting base-files...
I: Extracting base-passwd...
E: Extracting .//var/cache/apt/archives/base-passwd_3.5.51_amd64.deb requires the zstdcat command, which is not available
One needs to install zstd on the host, including zstd in the

debootstrap,set will not help.,,Does $ sudo apt install zstd => fix the
issue for you?

Yeah, it fixed my test case but my actual problem was:

$ export d=impish
$ export a=amd64
$ sudo DIST=$d ARCH=$a cowbuilder --create --basepath
/var/cache/pbuilder/base-$d-$a.cow --distribution $d --debootstrap
debootstrap --architecture $a --debootstrapopts --arch --debootstrapopts
amd64 --debootstrapopts --variant=buildd --configfile=/tmp/configfile
--hookdir /usr/share/jenkins-debian-glue/pbuilder-hookdir/
I: Invoking pbuilder
I: forking: pbuilder create --debootstrap debootstrap --debootstrapopts
--arch --debootstrapopts amd64 --debootstrapopts --variant=buildd
--configfile /tmp/configfile --hookdir
/usr/share/jenkins-debian-glue/pbuilder-hookdir/ --buildplace
/var/cache/pbuilder/base-impish-amd64.cow --mirror
http://www-ftp.lip6.fr/pub/linux/distributions//Ubuntu/archive/
--architecture amd64 --distribution impish --no-targz --extrapackages '
ccache zstd cowdancer'
I: Running in no-targz mode
I: Distribution is impish.
I: Current time: Mon Oct 11 16:01:34 UTC 2021
I: pbuilder-time-stamp: 1633968094
I: Building the build environment
I: running debootstrap
/usr/sbin/debootstrap
I: impish uses zstd compression, setting --extractor=ar option
I: Retrieving InRelease
I: Retrieving Packages
I: Validating Packages
I: Resolving dependencies of required packages...
I: Resolving dependencies of base packages...
I: Checking component main on
http://www-ftp.lip6.fr/pub/linux/distributions//Ubuntu/archive...
I: Validating adduser 3.118ubuntu5
[...]
I: Validating util-linux 2.36.1-8ubuntu1
I: Validating xz-utils 5.2.5-2
I: Validating zlib1g 1:1.2.11.dfsg-2ubuntu7
I: Chosen extractor for .deb packages: ar
I: Extracting base-files...
E: Unknown compression type for data.tar.zst in
.//var/cache/apt/archives/base-files_11.1ubuntu5_amd64.deb
E: debootstrap failed
E: Tail of debootstrap.log:
2021-10-11 16:01:35
URL:http://www-ftp.lip6.fr/pub/linux/distributions//Ubuntu/archive/dists/impish/InRelease
[270090/270090] ->
"/var/cache/pbuilder/base-impish-amd64.cow/var/lib/apt/lists/partial/www-ftp.lip6.fr_pub_linux_distributions__Ubuntu_archive_dists_impish_InRelease"
[1]
2021-10-11 16:01:36
URL:http://www-ftp.lip6.fr/pub/linux/distributions//Ubuntu/archive/dists/impish/main/binary-amd64/by-hash/SHA256/5f55112843e1755f687849f2f61645c3428d00cc103b646b696cd5ed4ae7d3cc
[1417328/1417328] ->
"/var/cache/pbuilder/base-impish-amd64.cow/var/lib/apt/lists/partial/www-ftp.lip6.fr_pub_linux_distributions__Ubuntu_archive_dists_impish_main_binary-amd64_Packages.xz"
[1]
E: End of debootstrap.log
W: Aborting with an error
E: pbuilder create failed
I: forking: rm -rf /var/cache/pbuilder/base-impish-amd64.cow

And the system has zstd:

$ dpkg -l|grep zstd
ii libzstd1:amd64                        1.4.8+dfsg-2.1
amd64        fast lossless compression algorithm
ii zstd                                  1.4.8+dfsg-2.1
amd64        fast lossless compression algorithm -- CLI tool

Sylvestre Ledru

2021-10-16 09:10:02 UTC

Permalink

Hello Dimitri,

Any idea how to fix that? I cannot generate packages for
https://apt.llvm.org/ for impish from a Debian.

thanks
Sylvestre

Post by Sylvestre Ledru
Hello

debootstrap,set will not help.,,Does $ sudo apt install zstd => fix the
issue for you?
$ export d=impish
$ export a=amd64
$ sudo DIST=$d ARCH=$a cowbuilder --create --basepath
/var/cache/pbuilder/base-$d-$a.cow --distribution $d --debootstrap
debootstrap --architecture $a --debootstrapopts --arch --debootstrapopts
amd64 --debootstrapopts --variant=buildd --configfile=/tmp/configfile
--hookdir /usr/share/jenkins-debian-glue/pbuilder-hookdir/
I: Invoking pbuilder
I: forking: pbuilder create --debootstrap debootstrap --debootstrapopts
--arch --debootstrapopts amd64 --debootstrapopts --variant=buildd
--configfile /tmp/configfile --hookdir
/usr/share/jenkins-debian-glue/pbuilder-hookdir/ --buildplace
/var/cache/pbuilder/base-impish-amd64.cow --mirror
http://www-ftp.lip6.fr/pub/linux/distributions//Ubuntu/archive/
--architecture amd64 --distribution impish --no-targz --extrapackages '
ccache zstd cowdancer'
I: Running in no-targz mode
I: Distribution is impish.
I: Current time: Mon Oct 11 16:01:34 UTC 2021
I: pbuilder-time-stamp: 1633968094
I: Building the build environment
I: running debootstrap
/usr/sbin/debootstrap
I: impish uses zstd compression, setting --extractor=ar option
I: Retrieving InRelease
I: Retrieving Packages
I: Validating Packages
I: Resolving dependencies of required packages...
I: Resolving dependencies of base packages...
I: Checking component main on
http://www-ftp.lip6.fr/pub/linux/distributions//Ubuntu/archive...
I: Validating adduser 3.118ubuntu5
[...]
I: Validating util-linux 2.36.1-8ubuntu1
I: Validating xz-utils 5.2.5-2
I: Validating zlib1g 1:1.2.11.dfsg-2ubuntu7
I: Chosen extractor for .deb packages: ar
I: Extracting base-files...
E: Unknown compression type for data.tar.zst in
.//var/cache/apt/archives/base-files_11.1ubuntu5_amd64.deb
E: debootstrap failed
2021-10-11 16:01:35
URL:http://www-ftp.lip6.fr/pub/linux/distributions//Ubuntu/archive/dists/impish/InRelease
[270090/270090] ->
"/var/cache/pbuilder/base-impish-amd64.cow/var/lib/apt/lists/partial/www-ftp.lip6.fr_pub_linux_distributions__Ubuntu_archive_dists_impish_InRelease"
[1]
2021-10-11 16:01:36
URL:http://www-ftp.lip6.fr/pub/linux/distributions//Ubuntu/archive/dists/impish/main/binary-amd64/by-hash/SHA256/5f55112843e1755f687849f2f61645c3428d00cc103b646b696cd5ed4ae7d3cc
[1417328/1417328] ->
"/var/cache/pbuilder/base-impish-amd64.cow/var/lib/apt/lists/partial/www-ftp.lip6.fr_pub_linux_distributions__Ubuntu_archive_dists_impish_main_binary-amd64_Packages.xz"
[1]
E: End of debootstrap.log
W: Aborting with an error
E: pbuilder create failed
I: forking: rm -rf /var/cache/pbuilder/base-impish-amd64.cow
$ dpkg -l|grep zstd
ii libzstd1:amd64                        1.4.8+dfsg-2.1
amd64        fast lossless compression algorithm
ii zstd                                  1.4.8+dfsg-2.1
amd64        fast lossless compression algorithm -- CLI tool

Dimitri John Ledkov

2021-10-20 16:20:01 UTC

Permalink

Hi,

I've tried to reproduce this more and I have a few questions:

1) what is the version of debootstrap ?
2) do any config files divert as to which debootstrap's `functions`
file is used ?
3) what settings do you have in /tmp/configfile ?

I'm trying to eliminate a case where new scripts/impish is used that
sets AR extractor, and yet old functions file is used which doesn't
know how to zstdcat in the AR extractor.

ie.
```
$ grep zstdcat /usr/share/debootstrap/functions
control.tar.zst) cat_cmd=zstdcat ;;
data.tar.zst) cat_cmd=zstdcat ;;
```
I'm not sure what in your case is listed in /tmp/configfile. I assume
that one thing is pointer to the custom mirror.

I launched Debian sid, enabled experimental, installed debootstrap
from experimental, install cowbuilder and all the jenkins-debian-glue
stuff. I did echo custom mirror of
http://www-ftp.lip6.fr/pub/linux/distributions//Ubuntu/archive into
/tmp/configfile. Cause that looked like the only custom thing. Not
sure if there are anything else set there. and Things mostly
progressed fine for me:

# DIST=$d ARCH=$a cowbuilder --create --basepath
/var/cache/pbuilder/base-$d-$a.cow --distribution $d --debootstrap
debootstrap --architecture $a --debootstrapopts --arch
--debootstrapopts amd64 --debootstrapopts --variant=buildd
--configfile=/tmp/configfile --hookdir
/usr/share/jenkins-debian-glue/pbuilder-hookdir/
I: Invoking pbuilder
I: forking: pbuilder create --debootstrap debootstrap
--debootstrapopts --arch --debootstrapopts amd64 --debootstrapopts
--variant=buildd --configfile /tmp/configfile --hookdir
/usr/share/jenkins-debian-glue/pbuilder-hookdir/ --buildplace
/var/cache/pbuilder/base-impish-amd64.cow --mirror
http://www-ftp.lip6.fr/pub/linux/distributions//Ubuntu/archive/
--architecture amd64 --distribution impish --no-targz --extrapackages
cowdancer
W: /root/.pbuilderrc does not exist
I: Running in no-targz mode
I: Distribution is impish.
I: Current time: Wed Oct 20 15:07:00 UTC 2021
I: pbuilder-time-stamp: 1634742420
I: Building the build environment
I: running debootstrap
/usr/bin/which: this version of `which' is deprecated; use `command
-v' in scripts instead.
/usr/sbin/debootstrap
I: Target architecture can be executed
I: impish uses zstd compression, setting --extractor=ar option
I: Retrieving InRelease
I: Checking Release signature
I: Valid Release signature (key id F6ECB3762474EDA9D21B7022871920D1991BC93C)
I: Retrieving Packages
I: Validating Packages
I: Resolving dependencies of required packages...
I: Resolving dependencies of base packages...
I: Checking component main on
http://www-ftp.lip6.fr/pub/linux/distributions//Ubuntu/archive...

...

I: Retrieving zlib1g 1:1.2.11.dfsg-2ubuntu7
I: Validating zlib1g 1:1.2.11.dfsg-2ubuntu7
I: Chosen extractor for .deb packages: ar
I: Extracting base-files...
I: Extracting base-passwd...

....

I: Installing core packages...
I: Unpacking required packages...
I: Unpacking base-files...
I: Unpacking base-passwd...

...

I: Configuring required packages...
I: Configuring gcc-11-base:amd64...
I: Configuring lsb-base...
I: Configuring libtirpc-common..
...

Eventually it failed for me with:

Package cowdancer is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source

E: Package 'cowdancer' has no installation candidate
E: Package 'aptitude' has no installation candidate
I: unmounting dev/pts filesystem
I: unmounting dev/shm filesystem
I: unmounting proc filesystem
I: unmounting sys filesystem
E: pbuilder create failed
I: forking: rm -rf /var/cache/pbuilder/base-impish-amd64.cow

But that seems like further along than what you have. It is possible
that I didn't enable universe, cause aptitude is available in Ubuntu.

Post by Sylvestre Ledru
Hello Dimitri,
Any idea how to fix that? I cannot generate packages for
https://apt.llvm.org/ for impish from a Debian.
thanks
Sylvestre

Post by Sylvestre Ledru
Hello

debootstrap,set will not help.,,Does $ sudo apt install zstd => fix the
issue for you?
$ export d=impish
$ export a=amd64
$ sudo DIST=$d ARCH=$a cowbuilder --create --basepath
/var/cache/pbuilder/base-$d-$a.cow --distribution $d --debootstrap
debootstrap --architecture $a --debootstrapopts --arch --debootstrapopts
amd64 --debootstrapopts --variant=buildd --configfile=/tmp/configfile
--hookdir /usr/share/jenkins-debian-glue/pbuilder-hookdir/
I: Invoking pbuilder
I: forking: pbuilder create --debootstrap debootstrap --debootstrapopts
--arch --debootstrapopts amd64 --debootstrapopts --variant=buildd
--configfile /tmp/configfile --hookdir
/usr/share/jenkins-debian-glue/pbuilder-hookdir/ --buildplace
/var/cache/pbuilder/base-impish-amd64.cow --mirror
http://www-ftp.lip6.fr/pub/linux/distributions//Ubuntu/archive/
--architecture amd64 --distribution impish --no-targz --extrapackages '
ccache zstd cowdancer'
I: Running in no-targz mode
I: Distribution is impish.
I: Current time: Mon Oct 11 16:01:34 UTC 2021
I: pbuilder-time-stamp: 1633968094
I: Building the build environment
I: running debootstrap
/usr/sbin/debootstrap
I: impish uses zstd compression, setting --extractor=ar option
I: Retrieving InRelease
I: Retrieving Packages
I: Validating Packages
I: Resolving dependencies of required packages...
I: Resolving dependencies of base packages...
I: Checking component main on
http://www-ftp.lip6.fr/pub/linux/distributions//Ubuntu/archive...
I: Validating adduser 3.118ubuntu5
[...]
I: Validating util-linux 2.36.1-8ubuntu1
I: Validating xz-utils 5.2.5-2
I: Validating zlib1g 1:1.2.11.dfsg-2ubuntu7
I: Chosen extractor for .deb packages: ar
I: Extracting base-files...
E: Unknown compression type for data.tar.zst in
.//var/cache/apt/archives/base-files_11.1ubuntu5_amd64.deb
E: debootstrap failed
2021-10-11 16:01:35
URL:http://www-ftp.lip6.fr/pub/linux/distributions//Ubuntu/archive/dists/impish/InRelease
[270090/270090] ->
"/var/cache/pbuilder/base-impish-amd64.cow/var/lib/apt/lists/partial/www-ftp.lip6.fr_pub_linux_distributions__Ubuntu_archive_dists_impish_InRelease"
[1]
2021-10-11 16:01:36
URL:http://www-ftp.lip6.fr/pub/linux/distributions//Ubuntu/archive/dists/impish/main/binary-amd64/by-hash/SHA256/5f55112843e1755f687849f2f61645c3428d00cc103b646b696cd5ed4ae7d3cc
[1417328/1417328] ->
"/var/cache/pbuilder/base-impish-amd64.cow/var/lib/apt/lists/partial/www-ftp.lip6.fr_pub_linux_distributions__Ubuntu_archive_dists_impish_main_binary-amd64_Packages.xz"
[1]
E: End of debootstrap.log
W: Aborting with an error
E: pbuilder create failed
I: forking: rm -rf /var/cache/pbuilder/base-impish-amd64.cow
$ dpkg -l|grep zstd
ii libzstd1:amd64 1.4.8+dfsg-2.1
amd64 fast lossless compression algorithm
ii zstd 1.4.8+dfsg-2.1
amd64 fast lossless compression algorithm -- CLI tool

--
Regards,

Dimitri.