Discussion:
Bug#655301: xcp-xapi fails to start
(too old to reply)
Ritesh Raj Sarraf
2012-01-10 07:40:01 UTC
Permalink
Package: xcp-xapi
Version: 1.3-15
Severity: normal

My packages were initially installed from the deb repository at download.xensource.org.
Eventually, I upgraded them to the debian repository ones.


Right now, xcp-xapi fails to start with the following messages.


lnx200-39:~# /etc/init.d/xcp-xapi start
Error: Connection refused (calling connect )
lnx200-39:~#


-- System Information:
Debian Release: wheezy/sid
APT prefers unstable
APT policy: (500, 'unstable'), (500, 'testing'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 3.1.0-1-amd64 (SMP w/24 CPU cores)
Locale: LANG=en_IN, LC_CTYPE=en_IN (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages xcp-xapi depends on:
ii hwdata 0.233-1
ii libc6 2.13-24
ii libpam0g 1.1.3-6
ii libuuid1 2.20.1-1.1
ii libvhd0 2.0.90-1
ii libxen-4.1 4.1.2-2
ii libxenstore3.0 4.1.2-2
ii python 2.7.2-9
ii python-xenapi 1.3-15
ii stunnel4 [stunnel] 3:4.50-1
ii xcp-eliloader 0.1-4
ii xcp-fe 0.5.2-3
ii xcp-squeezed 1.3-15
ii xcp-storage-managers 0.1.1-2
ii xcp-v6d 1.3-15
ii xcp-xe 1.3-15
ii xen-hypervisor-4.1-amd64 [xen-hypervisor-4.1] 4.1.2-2
ii xen-utils-4.1 4.1.2-2
ii zlib1g 1:1.2.3.4.dfsg-3

Versions of packages xcp-xapi recommends:
ii smbfs 2:5.2-1
ii xcp-guest-templates 0.1-3
ii xcp-vncterm 0.1-2

xcp-xapi suggests no packages.

-- Configuration Files:
/etc/xapi.conf changed:
base_path = /usr/lib/xen-common/xapi
schema_filename = /etc/xensource/db_schema.sql
license_filename = /etc/xensource/license
http-host = 10.72.200.39
http-port = 80
use-ssl = true
ssl-cert = /etc/ssl/certs/xensource-rio.pem
ssl-port = 443
gc-debug = true


-- no debconf information
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Thomas Goirand
2012-01-10 11:20:02 UTC
Permalink
Post by Ritesh Raj Sarraf
Package: xcp-xapi
Version: 1.3-15
Severity: normal
My packages were initially installed from the deb repository at download.xensource.org.
Eventually, I upgraded them to the debian repository ones.
Right now, xcp-xapi fails to start with the following messages.
lnx200-39:~# /etc/init.d/xcp-xapi start
Error: Connection refused (calling connect )
lnx200-39:~#
Hi Ritesh,

Thanks for your bug reports. It's really cool to see that there's
already some people using the XCP packages we worked on since last
summer, and even more to see that some of them are DDs! :)

I wonder, what is trying to connect to what here? Is it xe trying to
connect to xapi?

How can we make sure that the issue isn't specific to upgrades from
download.xensource.org, which frankly, Debian should nothing to do with?
I'm not saying that we shouldn't try to fix (it'd be nice if we could),
but that I have very little time to investigate the issue.

If we were to fix issues when upgrade from packages downloaded from
download.xensource.org (which really, I have no time to investigate),
then don't you think this is a bit out of scope and would deserve a
Severity: wishlist, and a title like "xcp-xapi fails to start if
upgrading from packages at download.xensource.org"?

Cheers,

Thomas
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Ritesh Raj Sarraf
2012-01-10 12:50:02 UTC
Permalink
Post by Thomas Goirand
Hi Ritesh,
Thanks for your bug reports. It's really cool to see that there's
already some people using the XCP packages we worked on since last
summer, and even more to see that some of them are DDs! :)
I wonder, what is trying to connect to what here? Is it xe trying to
connect to xapi?
How can we make sure that the issue isn't specific to upgrades from
download.xensource.org, which frankly, Debian should nothing to do with?
I'm not saying that we shouldn't try to fix (it'd be nice if we could),
but that I have very little time to investigate the issue.
If we were to fix issues when upgrade from packages downloaded from
download.xensource.org (which really, I have no time to investigate),
then don't you think this is a bit out of scope and would deserve a
Severity: wishlist, and a title like "xcp-xapi fails to start if
upgrading from packages at download.xensource.org"?
This seems to be happening because of this:

# Wait for xapi to write its "init complete" cookie: after here it's
safe to modify templates.
wait_for_xapi() {
MAX_RETRIES=50
RETRY=0
while [ ${RETRY} -lt ${MAX_RETRIES} ]; do
if [ -e ${XAPI_STARTUP_COOKIE} ]; then
return 0
fi
sleep 1
RETRY=$(( ${RETRY} + 1 ))
done
return 1
}



The daemon is started with daemon args as:
DAEMON_ARGS="-daemon -writereadyfile $XAPI_STARTUP_COOKIE
-writeinitcomplete $XAPI_INIT_COMPLETE_COOKIE"



I admit this bug report was done in a hurry without trying all options.

My wild guess is that the xcp cookies is dependent on the domain uuid.
In my case, the uuid was carried froward from the other repository. I
will try this out and update the bug report.


Thank you for getting the packages done. We'll get it in shape for
wheezy. :-)
--
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
Mike McClurg
2012-01-11 23:40:01 UTC
Permalink
Post by Ritesh Raj Sarraf
# Wait for xapi to write its "init complete" cookie: after here it's
safe to modify templates.
wait_for_xapi() {
   MAX_RETRIES=50
   RETRY=0
   while [ ${RETRY} -lt ${MAX_RETRIES} ]; do
       if [ -e ${XAPI_STARTUP_COOKIE} ]; then
           return 0
       fi
       sleep 1
       RETRY=$(( ${RETRY} + 1 ))
   done
   return 1
}
DAEMON_ARGS="-daemon -writereadyfile $XAPI_STARTUP_COOKIE
-writeinitcomplete $XAPI_INIT_COMPLETE_COOKIE"
I admit this bug report was done in a hurry without trying all options.
My wild guess is that the xcp cookies is dependent on the domain uuid.
In my case, the uuid was carried froward from the other repository. I
will try this out and update the bug report.
Thank you for getting the packages done. We'll get it in shape for
wheezy. :-)
Hi Ritesh,

It is likely that the uuid's changed in the upgrade because we didn't
handle the file renaming well. Do you have both
/etc/xensource-inventory and /etc/xcp/inventory? The second file is
the correct location. You may be able to copy the uuid from the first
inventory file into the second.

If this doesn't work, could you please attach both inventory files,
and the log file /var/log/xcp-xapi.log to the bug report? Thanks for
testing our software!

Mike
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Ritesh Raj Sarraf
2012-01-12 06:50:02 UTC
Permalink
Post by Mike McClurg
It is likely that the uuid's changed in the upgrade because we didn't
handle the file renaming well. Do you have both
/etc/xensource-inventory and /etc/xcp/inventory? The second file is
the correct location. You may be able to copy the uuid from the first
inventory file into the second.
Yes, both the files were there.
Post by Mike McClurg
If this doesn't work, could you please attach both inventory files,
and the log file /var/log/xcp-xapi.log to the bug report? Thanks for
testing our software!
Sorry, I wiped it out. We should document about uuid and other stuff.
If there's a web page that talks about them (UUID, Management Interface,
Current Interface etc), we should mention its link in the README.Debian
file.
--
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
Ritesh Raj Sarraf
2012-01-12 07:30:01 UTC
Permalink
Post by Ritesh Raj Sarraf
Sorry, I wiped it out. We should document about uuid and other stuff.
If there's a web page that talks about them (UUID, Management
Interface, Current Interface etc), we should mention its link in the
README.Debian file.
So I had purged the xapi packages and rebooted the host. Then I did a
reinstall.


Setting up xcp-fe (0.5.2-3) ...
Setting up xcp-squeezed (1.3-15) ...
xc: error: Could not obtain handle on privileged command interface (2 =
No such file or directory): Internal error
Fatal error: exception Xenctrl.Error("Unable to open XC interface")
Setting up xcp-v6d (1.3-15) ...
Setting up xcp-storage-managers (0.1.1-2) ...
Setting up xcp-xe (1.3-15) ...
Setting up xcp-xapi (1.3-15) ...
Error: Connection refused (calling connect )
md5sum: /var/lib/xcp/templates.md5: No such file or directory
Waiting for xapi to signal init complete

failed to detect xapi init complete after 300s
Setting up xcp-guest-templates (0.1-3) ...
Setting up xcp-vncterm (0.1-2) ...


The connection error is still there. And because of that there's a 300s
timeout.

This is my cleaned up environement.
lnx200-39:~# ls /etc/x
xcp/ xdg/ xen/ xen-tools/ xml/



I'll look more into it.
--
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
Mike McClurg
2012-01-12 08:10:01 UTC
Permalink
Post by Ritesh Raj Sarraf
Setting up xcp-fe (0.5.2-3) ...
Setting up xcp-squeezed (1.3-15) ...
xc: error: Could not obtain handle on privileged command interface (2 =
No such file or directory): Internal error
Fatal error: exception Xenctrl.Error("Unable to open XC interface")
Setting up xcp-v6d (1.3-15) ...
Setting up xcp-storage-managers (0.1.1-2) ...
Setting up xcp-xe (1.3-15) ...
Setting up xcp-xapi (1.3-15) ...
Error: Connection refused (calling connect )
md5sum: /var/lib/xcp/templates.md5: No such file or directory
Waiting for xapi to signal init complete
failed to detect xapi init complete after 300s
Setting up xcp-guest-templates (0.1-3) ...
Setting up xcp-vncterm (0.1-2) ...
The connection error is still there. And because of that there's a 300s
timeout.
This is my cleaned up environement.
lnx200-39:~# ls /etc/x
xcp/       xdg/       xen/       xen-tools/ xml/
I'll look more into it.
Could you attach the fresh logs from this environment? There should be
a few hundred lines of logs in /var/log/xcp-xapi.log coming from
thread_zero, which is xapi's initialisation bit. Could you attach
that?

Mike
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Ritesh Raj Sarraf
2012-01-12 12:00:01 UTC
Permalink
Post by Mike McClurg
Could you attach the fresh logs from this environment? There should be
a few hundred lines of logs in /var/log/xcp-xapi.log coming from
thread_zero, which is xapi's initialisation bit. Could you attach
that?
Attached is the log with this email.
--
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
Jon Ludlam
2012-01-13 02:00:02 UTC
Permalink
That log looks to me like you weren't running on xen - the 'failed to
open xenctrl interface' almost always means that - could you please
double check?

Thanks,

Jon
Post by Ritesh Raj Sarraf
Post by Mike McClurg
Could you attach the fresh logs from this environment? There should be
a few hundred lines of logs in /var/log/xcp-xapi.log coming from
thread_zero, which is xapi's initialisation bit. Could you attach
that?
Attached is the log with this email.
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Ritesh Raj Sarraf
2012-01-16 17:00:02 UTC
Permalink
dmesg does report successful xen msgs.
Is there anything you want me to verify? The machine is up and hasn't
been rebooted yet.
Attached is the dmesg text. Help that helps.
--
Ritesh Raj Sarraf
RESEARCHUT - http://www.researchut.com
"Necessity is the mother of invention."
Ritesh Raj Sarraf
2012-01-16 17:00:02 UTC
Permalink
Post by Jon Ludlam
That log looks to me like you weren't running on xen - the 'failed to
open xenctrl interface' almost always means that - could you please
double check?
This is the kernel running.

lnx200-39:~# uname -a
Linux lnx200-39 3.1.0-1-amd64 #1 SMP Tue Jan 10 05:01:58 UTC 2012 x86_64
GNU/Linux


Interestingly there's nothing under /proc/xen.


The xm interface is disabled because xcp conflicts xend.


dmesg does report successful xen msgs.


Is there anything you want me to verify? The machine is up and hasn't
been rebooted yet.

lnx200-39:~# uptime
06:13:30 up 4 days, 9:15, 1 user, load average: 0.00, 0.01, 0.05
--
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
Thomas Goirand
2012-01-16 21:10:02 UTC
Permalink
Post by Ritesh Raj Sarraf
Post by Jon Ludlam
That log looks to me like you weren't running on xen - the 'failed to
open xenctrl interface' almost always means that - could you please
double check?
This is the kernel running.
lnx200-39:~# uname -a
Linux lnx200-39 3.1.0-1-amd64 #1 SMP Tue Jan 10 05:01:58 UTC 2012 x86_64
GNU/Linux
Interestingly there's nothing under /proc/xen.
The /etc/init.d/xend init script should have mount xenfs for you. Try to
restart it by hand, and see if it does it. xenfs and xen-evtchn modules
are also loaded by this init script. Can you also check if they are
loaded? If they aren't loaded after doing a /etc/init.d/xend start, then
you got an issue here, which really, has nothing to do with XCP.

Cheers,

Thomas
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Ritesh Raj Sarraf
2012-01-17 07:00:02 UTC
Permalink
Post by Thomas Goirand
Post by Ritesh Raj Sarraf
Interestingly there's nothing under /proc/xen.
The /etc/init.d/xend init script should have mount xenfs for you. Try to
restart it by hand, and see if it does it. xenfs and xen-evtchn modules
are also loaded by this init script. Can you also check if they are
loaded? If they aren't loaded after doing a /etc/init.d/xend start, then
you got an issue here, which really, has nothing to do with XCP.
Yes, running xend mounts /proc/xen.

But the problem is, xcp-xapi wants to have xend disabled.

# Exit with failure if xend is running
if [ -f /var/run/xend.pid ]; then
log_failure_msg "/var/run/xend.pid exists; ${NAME} conflicts
with xend"
exit 1
fi


So I disabled the xend init file and then ran into this problem. If I
run xapi with xend enabled, I get the expected err msg.

lnx200-39:~# ls /proc/xen/
capabilities privcmd xenbus xsd_kva xsd_port
lnx200-39:~# /etc/init.d/xcp-xapi start
/var/run/xend.pid exists; xapi conflicts with xend ... failed!
--
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
Mike McClurg
2012-01-17 09:30:02 UTC
Permalink
Post by Ritesh Raj Sarraf
Post by Thomas Goirand
Post by Ritesh Raj Sarraf
Interestingly there's nothing under /proc/xen.
The /etc/init.d/xend init script should have mount xenfs for you. Try to
restart it by hand, and see if it does it. xenfs and xen-evtchn modules
are also loaded by this init script. Can you also check if they are
loaded? If they aren't loaded after doing a /etc/init.d/xend start, then
you got an issue here, which really, has nothing to do with XCP.
Yes, running xend mounts /proc/xen.
But the problem is, xcp-xapi wants to have xend disabled.
# Exit with failure if xend is running
if [ -f /var/run/xend.pid ]; then
       log_failure_msg "/var/run/xend.pid exists; ${NAME} conflicts
with xend"
       exit 1
fi
So I disabled the xend init file and then ran into this problem. If I
run xapi with xend enabled, I get the expected err msg.
lnx200-39:~# ls /proc/xen/
capabilities  privcmd  xenbus  xsd_kva  xsd_port
lnx200-39:~# /etc/init.d/xcp-xapi start
/var/run/xend.pid exists; xapi conflicts with xend ... failed!
Hi Ritesh,

We're in a bit of a bind with the xend init script. Xapi depends on
most of the init script being run (as you've seen), but xend itself
conflicts with xapi. We need to come up with a good solution to this
conflict.

I think it would be best if we could create a file /etc/default/xen
that would let us switch between the xm, xl and xapi toolstacks. Until
then, do you think it would be best to copy the required parts of
xend.init into xcp-xapi.init? Are there any other solutions that we're
overlooking?

Mike
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Thomas Goirand
2012-01-17 21:40:02 UTC
Permalink
Post by Ritesh Raj Sarraf
But the problem is, xcp-xapi wants to have xend disabled.
[ ... ]
So I disabled the xend init file and then ran into this problem.
No wonder why then! Nobody ever wrote/said that you should do that. The
xend init.d script does a lot more than just starting xend. It also:
- modprobe the xenfs and xen-evtchn kernel modules
- mounts /proc/xen as xenfs
- Starts xenconsoled
- Starts xenstored

If the above isn't done, then there's no way that XCP, or even Xen, will
ever work! The only thing that should be disabled is starting xend, all
the rest should stay.
Post by Ritesh Raj Sarraf
I think it would be best if we could create a file /etc/default/xen
that would let us switch between the xm, xl and xapi toolstacks.
This file already exists!!!

By default, it has nothing in the TOOLSTACK= variable, which is
understood as being the same as TOOLSTACK=xm. If you put TOOLSTACK=xl in
it, then it's xl that is used, and "xm list" (for example) will return:

ERROR: A different toolstack (xl) have been selected!

The same will happen the other way (eg: xl list will fail if TOOLSTACK
contains "xm" or nothing in it).
Post by Ritesh Raj Sarraf
Until
then, do you think it would be best to copy the required parts of
xend.init into xcp-xapi.init? Are there any other solutions that we're
overlooking?
No, that's *not* the solution. The solution is to patch xen-commons,
which I just did (see attached patch for
xen-common-4.1.2/debian/xen-utils-common.xend.init).

By the way, I have proposed to have TOOLSTACK to be configured by
debconf, but Bastian rejected it because he thinks that I shouldn't be
rewriting the /etc/default/xen file. I'm ok with that, but I think that
a sed would be a weaker way, so I asked what Bastian though we should
do, and had no reply to this question. I don't mind doing:

TMPFILE=`mktemp`
grep -v "^TOOLSTACK=" /etc/default/xen >${TMPFILE}
echo "TOOLSTACK=${TOOLSTACK}" >>${TMPFILE}
cat <${TMPFILE} >/etc/default/xen
rm ${TMPFILE}

if you think it's better than just:

echo "# Configuration for Xen system
# ----------------------------

# There exists several tool stacks to configure a Xen system.
#
# Attention: You need to reboot after changing this!
TOOLSTACK=${TOOLSTACK}
" >/etc/default/xen

Please Bastian, let me know what you think, so that this can get
included in the next upload of xen-utils-common.

In the mean while, please accept the attached patch for not starting
xend if TOOLSTACK is "xm" or empty.

I'm doing a retitle of this bug, and I'm assigning it to
xen-utils-common instead of xcp-xapi, which is doing nothing wrong.

Cheers,

Thomas Goirand (zigo)
Ian Campbell
2012-01-17 22:50:02 UTC
Permalink
Post by Thomas Goirand
By the way, I have proposed to have TOOLSTACK to be configured by
debconf, but Bastian rejected it because he thinks that I shouldn't be
rewriting the /etc/default/xen file.
It's not (just) Bastian but rather Debian policy that says you must not
do this, 10.7.3 in particular.

Ian.
--
Ian Campbell


Yow! Am I in Milwaukee?
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Thomas Goirand
2012-01-18 05:10:02 UTC
Permalink
Post by Ian Campbell
Post by Thomas Goirand
By the way, I have proposed to have TOOLSTACK to be configured by
debconf, but Bastian rejected it because he thinks that I shouldn't be
rewriting the /etc/default/xen file.
It's not (just) Bastian but rather Debian policy that says you must not
do this, 10.7.3 in particular.
Ian.
Overwriting the /etc/default/xen file did *NOT* overwrite a local change
in the *value* stored in the file. It would have only overwrite things
like comments, but keep the value written in the "TOOLSTACK" variable
(well, only if it has one of the 3 valid values, but do we care keeping
a broken value?). I believe doing this way is also Debian policy
compliant, there's nothing in the policy that forces you to keep
administrator's comments in a configuration file. If you think I'm
wrong, let me know.

Anyway, I am still waiting for comments on how you or Bastian want this
to be done. I have nothing against grep and friends...

Thomas
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Ian Campbell
2012-01-18 07:30:01 UTC
Permalink
Post by Thomas Goirand
Overwriting the /etc/default/xen file did *NOT* overwrite a local change
in the *value* stored in the file. It would have only overwrite things
like comments, but keep the value written in the "TOOLSTACK" variable
(well, only if it has one of the 3 valid values, but do we care keeping
a broken value?). I believe doing this way is also Debian policy
compliant, there's nothing in the policy that forces you to keep
administrator's comments in a configuration file.
Rubbish, policy quite clearly says, WRT configuration files:
local changes must be preserved during a package upgrade

It makes no distinction about "values" or "comments" or any other sort
of change.

Maybe the comments are actually important to the administrator? Why else
would he have added them.

Anyway, you don't know what else an administrator has added to that
file. Perhaps they have added their own variables to support their
modifications to the initscripts? Perhaps they have locally added a
fourth option for $TOOLSTACK to support their own use cases?
Post by Thomas Goirand
If you think I'm wrong, let me know.
You are wrong.
Post by Thomas Goirand
Anyway, I am still waiting for comments on how you or Bastian want this
to be done. I have nothing against grep and friends...
Can't ucf be used for this? (e.g. generate a $TMP, or /var?, version of
the file and use ucf to handle the update).

Ian.
--
Ian Campbell


Satire is what closes in New Haven.
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Thomas Goirand
2012-01-19 20:00:01 UTC
Permalink
Post by Ian Campbell
Can't ucf be used for this? (e.g. generate a $TMP, or /var?, version of
the file and use ucf to handle the update).
As much as I know, ucf will only ask if the configuration file should be
overwritten or not, and that's not what we want to do here, we want
debconf as an interface to switch the TOOLSTACK value. Am I wrong here
again? :)

Can you comment on my grep -v solution?

Thomas
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Ian Campbell
2012-01-19 21:10:01 UTC
Permalink
Post by Thomas Goirand
Post by Ian Campbell
Can't ucf be used for this? (e.g. generate a $TMP, or /var?, version of
the file and use ucf to handle the update).
As much as I know, ucf will only ask if the configuration file should be
overwritten or not, and that's not what we want to do here, we want
debconf as an interface to switch the TOOLSTACK value.
You ask the user if they want to overwrite their local configuration
with the one you have just generated from debconf. Maybe this isn't a
valid use of ucf, I don't know -- it was just an idea.
Post by Thomas Goirand
Am I wrong here
again? :)
Can you comment on my grep -v solution?
Why not have a look at one of the undoubtedly many packages which
already does this?

Ian.
--
Ian Campbell


Having the fewest wants, I am nearest to the gods.
-- Socrates
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Thomas Goirand
2012-01-28 22:30:02 UTC
Permalink
Post by Ian Campbell
Post by Thomas Goirand
Am I wrong here
again? :)
Can you comment on my grep -v solution?
Why not have a look at one of the undoubtedly many packages which
already does this?
Attached to this mail is a version that keeps eventual comments an admin
has put in the /etc/default/xen file. See choose_xl_with_debconf_2.patch.

Together with the xen-utils-common.xend.init.patch, this solves #655301.
Bastian, can you apply these patches?

Cheers,

Thomas
Ian Campbell
2012-01-31 07:20:02 UTC
Permalink
+ Note that if you wish to use Xen with XCP (Xen Cloud Platform), then using
+ xl is mandatory.
As I've said before I don't think this is correct. The correct value if
you are using XCP is "xapi" or "xe" or however you decide to word it.

Ian.
--
Ian Campbell


Do not do unto others as you would they should do unto you. Their tastes
may not be the same.
-- George Bernard Shaw
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Thomas Goirand
2012-01-31 21:50:02 UTC
Permalink
Post by Ian Campbell
+ Note that if you wish to use Xen with XCP (Xen Cloud Platform), then using
+ xl is mandatory.
As I've said before I don't think this is correct. The correct value if
you are using XCP is "xapi" or "xe" or however you decide to word it.
Ian.
When you choose "xm", then "xl" doesn't work. When you choose "xl", then
"xm" doesn't work.

Now, whatever the value of TOOLSTACK, we have "xe" that always work. So
there's no consistency, maybe we should make xe to fail if TOOLSTACK is
set to something else than xe?

I don't mind implementing a wrapper for xe though, which would do the
same thing as the wrappers we have for both xl and xe, and then package
the "real" xe binary in somewhere else. But then I see an issue. It
might well be possible to use xe only to remote-control an XCP server,
in which case Xen might not even be installed on the local computer. In
such case, xe shouldn't be set to fail, because of a wrong value of
TOOLSTACK.

What's your take on this? What should I do then? Thoughts?

Thomas
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Mike McClurg
2012-02-01 08:40:02 UTC
Permalink
Post by Thomas Goirand
Post by Ian Campbell
+ Note that if you wish to use Xen with XCP (Xen Cloud Platform), then using
+ xl is mandatory.
As I've said before I don't think this is correct. The correct value if
you are using XCP is "xapi" or "xe" or however you decide to word it.
Ian.
When you choose "xm", then "xl" doesn't work. When you choose "xl", then
"xm" doesn't work.
Now, whatever the value of TOOLSTACK, we have "xe" that always work. So
there's no consistency, maybe we should make xe to fail if TOOLSTACK is
set to something else than xe?
I don't mind implementing a wrapper for xe though, which would do the
same thing as the wrappers we have for both xl and xe, and then package
the "real" xe binary in somewhere else. But then I see an issue. It
might well be possible to use xe only to remote-control an XCP server,
in which case Xen might not even be installed on the local computer. In
such case, xe shouldn't be set to fail, because of a wrong value of
TOOLSTACK.
What's your take on this? What should I do then? Thoughts?
I prefer for xapi to only start when TOOLSTACK=xapi. xe isn't the
toolstack, it's just a CLI to xapi, and the TOOLSTACK switch should
have nothing to do with whether the user can run xe on that host. This
is analogous to how xend would work: setting toolstack to xl doesn't
prevent you from running xm, it just prevents xend from running, which
makes running xm fairly useless. xe is useful even if xapi isn't
running because it can talk to other hosts.

In my development branch of xapi, I have set the init script to only
start xapi if TOOLSTACK=xapi. I think that this is the way that it
should work. (But honestly, I don't care whether it's TOOLSTACK=xapi
or TOOSTACK=xe, as long as xapi won't start unless the correct value
is set.)

Mike
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Ian Campbell
2012-02-01 09:00:02 UTC
Permalink
Post by Mike McClurg
Post by Thomas Goirand
Post by Ian Campbell
+ Note that if you wish to use Xen with XCP (Xen Cloud Platform), then using
+ xl is mandatory.
As I've said before I don't think this is correct. The correct value if
you are using XCP is "xapi" or "xe" or however you decide to word it.
Ian.
When you choose "xm", then "xl" doesn't work. When you choose "xl", then
"xm" doesn't work.
Now, whatever the value of TOOLSTACK, we have "xe" that always work. So
there's no consistency, maybe we should make xe to fail if TOOLSTACK is
set to something else than xe?
I don't mind implementing a wrapper for xe though, which would do the
same thing as the wrappers we have for both xl and xe, and then package
the "real" xe binary in somewhere else. But then I see an issue. It
might well be possible to use xe only to remote-control an XCP server,
in which case Xen might not even be installed on the local computer. In
such case, xe shouldn't be set to fail, because of a wrong value of
TOOLSTACK.
What's your take on this? What should I do then? Thoughts?
I prefer for xapi to only start when TOOLSTACK=xapi. xe isn't the
toolstack, it's just a CLI to xapi, and the TOOLSTACK switch should
have nothing to do with whether the user can run xe on that host. This
is analogous to how xend would work: setting toolstack to xl doesn't
prevent you from running xm, it just prevents xend from running, which
makes running xm fairly useless. xe is useful even if xapi isn't
running because it can talk to other hosts.
In theory I think xm can be used against a remote xend as well. However
I don't know how widespread that usage model actually is (nor even if it
actually works or is just a rumour). I suggest we leave that to one side
for now, given that xend is going to be deprecated in the 4.2 release.
Post by Mike McClurg
In my development branch of xapi, I have set the init script to only
start xapi if TOOLSTACK=xapi. I think that this is the way that it
should work. (But honestly, I don't care whether it's TOOLSTACK=xapi
or TOOSTACK=xe, as long as xapi won't start unless the correct value
is set.)
Agree wrt TOOLSTACK=xapi, but I also don't really care so long as it
does the right thing for whichever value.

It occurs to me that you might want to arrange
for /etc/init.d/xendomains to not do anything unless toolstack == xl or
xm too. Presumably xapi has it's own method for starting domains at
start of day but I'm pretty sure it isn't to call xe with xl/xm style
parameters.

Ian.
--
Ian Campbell


"Do you think there's a God?"
"Well, ____SOMEbody's out to get me!"
-- Calvin and Hobbs
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Ritesh Raj Sarraf
2012-01-19 08:00:03 UTC
Permalink
Post by Thomas Goirand
Post by Ritesh Raj Sarraf
But the problem is, xcp-xapi wants to have xend disabled.
[ ... ]
So I disabled the xend init file and then ran into this problem.
No wonder why then! Nobody ever wrote/said that you should do that. The
- modprobe the xenfs and xen-evtchn kernel modules
- mounts /proc/xen as xenfs
- Starts xenconsoled
- Starts xenstored
If the above isn't done, then there's no way that XCP, or even Xen, will
ever work! The only thing that should be disabled is starting xend, all
the rest should stay.
What do you mean by "disable starting xend" ?

For now, just calling the xend stop in the xapi init script should fix
the problem.
--
Ritesh Raj Sarraf
RESEARCHUT - http://www.researchut.com
"Necessity is the mother of invention."
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Mike McClurg
2012-01-19 09:10:02 UTC
Permalink
Post by Ritesh Raj Sarraf
Post by Thomas Goirand
Post by Ritesh Raj Sarraf
But the problem is, xcp-xapi wants to have xend disabled.
[ ... ]
So I disabled the xend init file and then ran into this problem.
No wonder why then! Nobody ever wrote/said that you should do that. The
- modprobe the xenfs and xen-evtchn kernel modules
- mounts /proc/xen as xenfs
- Starts xenconsoled
- Starts xenstored
If the above isn't done, then there's no way that XCP, or even Xen, will
ever work! The only thing that should be disabled is starting xend, all
the rest should stay.
What do you mean by "disable starting xend" ?
We'd like that the xend init script continue to start the other
services that xpi-xapi relies on, but only actually start the xend
daemon iff TOOLSTACK=xm. This way we don't have to worry about
renaming or splitting init scripts.
Post by Ritesh Raj Sarraf
For now, just calling the xend stop in the xapi init script should fix
the problem.
Agreed, but as was pointed out to me by Ian Campbell, that's a bit of
a hack, since one service shouldn't be allowed to just shut another
one down. I was in favour of making this change to xcp-xapi.init
temporarily, until the xend init script is modified to only
selectively start xend, but I think now that we should instead ask
users to work around this issue until we sort out the xend script. The
simplest work-around is to ask the user to add the line 'service xend
stop' to the user's xcp-xapi.init script.

Thomas, I believe that you submitted a patch to xend.init to check
TOOLSTACK=xm. Do you know the status of this patch?

Mike
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Ritesh Raj Sarraf
2012-01-19 09:50:02 UTC
Permalink
Thank you. Documenting it would be good for now.

sent fr0m a $martphone, excuse typ0s
Post by Mike McClurg
Post by Ritesh Raj Sarraf
Post by Thomas Goirand
Post by Ritesh Raj Sarraf
But the problem is, xcp-xapi wants to have xend disabled.
[ ... ]
So I disabled the xend init file and then ran into this problem.
No wonder why then! Nobody ever wrote/said that you should do that. The
- modprobe the xenfs and xen-evtchn kernel modules
- mounts /proc/xen as xenfs
- Starts xenconsoled
- Starts xenstored
If the above isn't done, then there's no way that XCP, or even Xen, will
ever work! The only thing that should be disabled is starting xend, all
the rest should stay.
What do you mean by "disable starting xend" ?
We'd like that the xend init script continue to start the other
services that xpi-xapi relies on, but only actually start the xend
daemon iff TOOLSTACK=xm. This way we don't have to worry about
renaming or splitting init scripts.
Post by Ritesh Raj Sarraf
For now, just calling the xend stop in the xapi init script should fix
the problem.
Agreed, but as was pointed out to me by Ian Campbell, that's a bit of
a hack, since one service shouldn't be allowed to just shut another
one down. I was in favour of making this change to xcp-xapi.init
temporarily, until the xend init script is modified to only
selectively start xend, but I think now that we should instead ask
users to work around this issue until we sort out the xend script. The
simplest work-around is to ask the user to add the line 'service xend
stop' to the user's xcp-xapi.init script.
Thomas, I believe that you submitted a patch to xend.init to check
TOOLSTACK=xm. Do you know the status of this patch?
Mike
Mike McClurg
2012-01-19 10:10:01 UTC
Permalink
Post by Ritesh Raj Sarraf
Thank you. Documenting it would be good for now.
Agreed. We can mention this on the How-to wiki page. Should this also
go in the README.Debian?

Mike
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Thomas Goirand
2012-01-19 10:50:01 UTC
Permalink
----- Original message -----
Post by Mike McClurg
Post by Ritesh Raj Sarraf
Thank you. Documenting it would be good for now.
Agreed. We can mention this on the How-to wiki page. Should this also
go in the README.Debian?
Mike
Nop, we shall wait until Bastian uploads a new
version of xen-utils-common, that's it!

I don't know the status since Bastian didn't reply.
He tends to communicate less that he does, which
I have been annoying him about, so now I just shut
up and wait silently until an upload... :)

By the way, it'd be great if Xen was soon moving to
Git for it's packaging: I hate SVN. I guess it's on
Bastian's todo, but we could help for this move if
he just accepted.

Thomas
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Anders Kaseorg
2012-01-19 20:40:02 UTC
Permalink
If I might make a proposal, I think things would be simpler for everyone
if we put the separate toolstacks into separate conflicting packages.
Then we don’t have to deal with /etc/defaults or debconf or user
configuration at all, and each toolstack would know to shut itself down
when it got uninstalled, so that toolstacks don’t need to worry at
runtime about other toolstacks being enabled.

Anders
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Mike McClurg
2012-01-20 08:40:01 UTC
Permalink
If I might make a proposal, I think things would be simpler for everyone if
we put the separate toolstacks into separate conflicting packages. Then we
don’t have to deal with /etc/defaults or debconf or user configuration at
all, and each toolstack would know to shut itself down when it got
uninstalled, so that toolstacks don’t need to worry at runtime about other
toolstacks being enabled.
This might be the way to go when Xen 4.2 is released, when xend will
be formally deprecated. It might then be a good time to split xend
into a separate binary package from the rest of Xen, which could then
use xl by default.

I can also see that being difficult to implement in a way that won't
break existing installations on upgrade. Also, it would be nice to
allow the various toolstacks to coexist on the filesystem, and be able
to switch in between them without having to uninstall one in place of
the other.

Mike
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Thomas Goirand
2012-01-29 17:10:03 UTC
Permalink
Post by Mike McClurg
I can also see that being difficult to implement in a way that won't
break existing installations on upgrade. Also, it would be nice to
allow the various toolstacks to coexist on the filesystem, and be able
to switch in between them without having to uninstall one in place of
the other.
That's a request for using Debconf!

And I've just sent a patch for that.

Thomas
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Thomas Goirand
2012-01-19 10:40:02 UTC
Permalink
----- Original message -----
Post by Ritesh Raj Sarraf
Post by Thomas Goirand
If the above isn't done, then there's no way that XCP, or even Xen,
will ever work! The only thing that should be disabled is starting
xend, all the rest should stay.
What do you mean by "disable starting xend" ?
I mean that /etc/init.d/xend shouldn't start
/usr/sbin/xend.
Post by Ritesh Raj Sarraf
For now, just calling the xend stop in the xapi init script should fix
the problem.
Well, it shouldn't start in the first place! And
it's not up to xcp-xapi to mess arround with
daemon that doesn't bellong to it.

Thomas (from my phone)
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Jonathan Ludlam
2012-01-17 00:10:02 UTC
Permalink
I actually ran into a problem very similar to this myself this morning - it came from the fact that the init scripts were reordered at some point, and I still had the old init script ordering.

I sorted it temporarily by starting the scripts by hand - the normal order is:

20: xcp-fe, xend, xcp-v6d
22: xcp-squeezed
23: xcp-xapi

If you manually start the scripts in this order does it work?

It's interesting that your /proc/xen isn't mounted though - that should happen in the xend init script, and that should have worked.

Jon
Post by Ritesh Raj Sarraf
Post by Jon Ludlam
That log looks to me like you weren't running on xen - the 'failed to
open xenctrl interface' almost always means that - could you please
double check?
This is the kernel running.
lnx200-39:~# uname -a
Linux lnx200-39 3.1.0-1-amd64 #1 SMP Tue Jan 10 05:01:58 UTC 2012 x86_64
GNU/Linux
Interestingly there's nothing under /proc/xen.
The xm interface is disabled because xcp conflicts xend.
dmesg does report successful xen msgs.
Is there anything you want me to verify? The machine is up and hasn't
been rebooted yet.
lnx200-39:~# uptime
06:13:30 up 4 days, 9:15, 1 user, load average: 0.00, 0.01, 0.05
--
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Mike McClurg
2012-01-17 13:10:02 UTC
Permalink
On Mon, Jan 16, 2012 at 11:49 PM, Jonathan Ludlam
Post by Jonathan Ludlam
I actually ran into a problem very similar to this myself this morning - it came from the fact that the init scripts were reordered at some point, and I still had the old init script ordering.
20: xcp-fe, xend, xcp-v6d
22: xcp-squeezed
23: xcp-xapi
If you manually start the scripts in this order does it work?
It's interesting that your /proc/xen isn't mounted though - that should happen in the xend init script, and that should have worked.
I think that we put a line in the xcp-xapi init script that quit
without error if it found a xend pid file. This means that to start
xapi you need to either 1) edit /etc/init.d/xend so that xend is never
actually run, or 2) first let xend start, and then do 'service xend
stop' to kill xend, and then do a 'service xcp-xapi start'.

Neither of those is very attractive. Perhaps we could instead have the
user create a /etc/default/xen file, which will contain a
"TOOLSTACK=foo" line. If that file exists, and only if TOOLSTACK=xapi,
we could then, within the xcp-xapi init script, check to see if xend
is running, and if so, shut it down. This allows us to use xend as a
common xen init script, but forces the user to explicitly say that
they want to use xapi over xend.

I would feel better about killing xend from within the xcp-xapi init
file if we were to it this way. Thoughts?

Mike
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Ritesh Raj Sarraf
2012-01-17 16:50:02 UTC
Permalink
Post by Mike McClurg
I think that we put a line in the xcp-xapi init script that quit
without error if it found a xend pid file. This means that to start
xapi you need to either 1) edit /etc/init.d/xend so that xend is never
actually run, or 2) first let xend start, and then do 'service xend
stop' to kill xend, and then do a 'service xcp-xapi start'.
Neither of those is very attractive. Perhaps we could instead have the
user create a /etc/default/xen file, which will contain a
"TOOLSTACK=foo" line. If that file exists, and only if TOOLSTACK=xapi,
we could then, within the xcp-xapi init script, check to see if xend
is running, and if so, shut it down. This allows us to use xend as a
common xen init script, but forces the user to explicitly say that
they want to use xapi over xend.
I would feel better about killing xend from within the xcp-xapi init
file if we were to it this way. Thoughts?
You should just do all the stuff, that you expect from xend, inside
xcp-xapi. xend is a separate stack which will interest some users.

Similarly, if xend is not really needed by xcp, I'd rather want to purge
it and only have xcp.
--
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
Ian Campbell
2012-01-17 17:30:02 UTC
Permalink
(hrm, I seem not to have received this)
Post by Ritesh Raj Sarraf
Post by Mike McClurg
I think that we put a line in the xcp-xapi init script that quit
without error if it found a xend pid file. This means that to start
xapi you need to either 1) edit /etc/init.d/xend so that xend is never
actually run, or 2) first let xend start, and then do 'service xend
stop' to kill xend, and then do a 'service xcp-xapi start'.
Neither of those is very attractive. Perhaps we could instead have the
user create a /etc/default/xen file, which will contain a
"TOOLSTACK=foo" line. If that file exists, and only if TOOLSTACK=xapi,
we could then, within the xcp-xapi init script, check to see if xend
is running, and if so, shut it down. This allows us to use xend as a
common xen init script, but forces the user to explicitly say that
they want to use xapi over xend.
I would feel better about killing xend from within the xcp-xapi init
file if we were to it this way. Thoughts?
It is never appropriate for one initscript to kill a daemon controlled
by another one. (I hope it is obvious why).

We've been through this conversation before. The xend initscript should
not start xend unless TOOLSTACK=xend. If xapi has needs things like this
then it needs to *co-operate* with the other toolstacks and their
initscripts, not start unilaterally killing stuff. Please send patches
to implement what you need in the xend initscript (and elsewhere).
Post by Ritesh Raj Sarraf
You should just do all the stuff, that you expect from xend, inside
xcp-xapi. xend is a separate stack which will interest some users.
Ideally the common stuff which the xend initscript does would be
factored out but Bastian seems to think there is some issue with this.
That doesn't conflict with the above though.

Ian.
Post by Ritesh Raj Sarraf
Similarly, if xend is not really needed by xcp, I'd rather want to purge
it and only have xcp.
_______________________________________________
Pkg-xen-devel mailing list
http://lists.alioth.debian.org/cgi-bin/mailman/listinfo/pkg-xen-devel
--
Ian Campbell
Current Noise: Red Hot Chili Peppers - If You Have To Ask

pediddel:
A car with only one working headlight.
-- "Sniglets", Rich Hall & Friends
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Jonathan Ludlam
2012-01-17 21:40:02 UTC
Permalink
Bastian had some arguments against that approach:

http://lists.alioth.debian.org/pipermail/pkg-xen-devel/2011-December/003858.html

Jon

Sent from my iPad
Post by Ritesh Raj Sarraf
Post by Mike McClurg
I think that we put a line in the xcp-xapi init script that quit
without error if it found a xend pid file. This means that to start
xapi you need to either 1) edit /etc/init.d/xend so that xend is never
actually run, or 2) first let xend start, and then do 'service xend
stop' to kill xend, and then do a 'service xcp-xapi start'.
Neither of those is very attractive. Perhaps we could instead have the
user create a /etc/default/xen file, which will contain a
"TOOLSTACK=foo" line. If that file exists, and only if TOOLSTACK=xapi,
we could then, within the xcp-xapi init script, check to see if xend
is running, and if so, shut it down. This allows us to use xend as a
common xen init script, but forces the user to explicitly say that
they want to use xapi over xend.
I would feel better about killing xend from within the xcp-xapi init
file if we were to it this way. Thoughts?
You should just do all the stuff, that you expect from xend, inside
xcp-xapi. xend is a separate stack which will interest some users.
Similarly, if xend is not really needed by xcp, I'd rather want to purge
it and only have xcp.
--
Ritesh Raj Sarraf | http://people.debian.org/~rrs
Debian - The Universal Operating System
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Thomas Goirand
2012-01-17 21:50:02 UTC
Permalink
Post by Jonathan Ludlam
I actually ran into a problem very similar to this myself this morning - it came from the fact that the init scripts were reordered at some point, and I still had the old init script ordering.
20: xcp-fe, xend, xcp-v6d
22: xcp-squeezed
23: xcp-xapi
Did you have this issue in Debian SID? It'd be very surprising, because
for init.d scripts, we have:

# Provides: xcp-xapi
# Required-Start: $remote_fs $syslog xcp-squeezed xcp-v6d xcp-fe
# Required-Stop: $remote_fs $syslog

# Provides: xcp-squeezed
# Required-Start: $remote_fs $syslog xend
# Required-Stop: $remote_fs $syslog

So, to me, your issue maybe have been that the xcp-squeezed init.d
script should also depend on xcp-fe and xcp-v6d. Should I patch the
xen-api package in this way? Or is it not important that xcp-fe and
xcp-v6d starts before xcp-squeezed?

Also note that if I do the above changes to the init.d script of
xcp-squeezed, we will have to add dependencies at the package level, so
that xcp-squeezed depends on xcp-fe and xcp-v6d. Otherwise, we'll have
piuparts failure like when xcp-xapi didn't depend on xen-utils-common.

Please let me know,
Cheers,

Thomas Goirand (zigo)
--
To UNSUBSCRIBE, email to debian-bugs-dist-***@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org
Loading...