Discussion:
Bug#907853: liblwp-protocol-https-perl: turning off hostname verification does not work
(too old to reply)
Slaven Rezic
2018-09-03 06:30:01 UTC
Permalink
Package: liblwp-protocol-https-perl
Version: 6.06-2
Severity: normal

Dear Maintainer,

to disable hostname verification in https requests one would set ssl_opts'
verify_hostname to a false value. However, this does not work:

$ perl -MLWP::UserAgent -e '$ua=LWP::UserAgent->new; $ua->ssl_opts(verify_hostname=>0); $res = $ua->get("https://www.dwd.de"); warn $res->as_string'
500 Can't connect to www.dwd.de:443 (certificate verify failed)
Content-Type: text/plain
Client-Date: Mon, 03 Sep 2018 05:58:34 GMT
Client-Warning: Internal response

Can't connect to www.dwd.de:443 (certificate verify failed)

SSL connect attempt failed error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed at /usr/share/perl5/LWP/Protocol/http.pm line 47.

With a self-compiled perl and modules installed from CPAN this works as expected
(in this case there's no artificial 500 response, but a 403 Forbidden response).

I found out that it's possible to workaround the issue with
Debian's perl by setting SSL_verify_mode:

$ perl -MIO::Socket::SSL=SSL_VERIFY_NONE -MLWP::UserAgent -e '$ua=LWP::UserAgent->new; $ua->ssl_opts(SSL_verify_mode => SSL_VERIFY_NONE, verify_hostname => 0); $res = $ua->get("https://www.dwd.de"); warn $res->as_string'

The issue is still present on Ubuntu 18.04 which has a newer
version of liblwp-protocol-https-perl. I also don't know if the
problem lies in LWP, LWP::Protocol::https, IO::Socket::SSL,
Net::SSLeay, or any other module.

-- System Information:
Debian Release: 9.5
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.9.0-3-amd64 (SMP w/4 CPU cores)
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968), LANGUAGE=C (charmap=ANSI_X3.4-1968)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages liblwp-protocol-https-perl depends on:
ii ca-certificates 20161130+nmu1+deb9u1
ii libio-socket-ssl-perl 2.044-1
ii libnet-http-perl 6.12-1
ii libwww-perl 6.15-1
ii perl 5.24.1-3+deb9u4

liblwp-protocol-https-perl recommends no packages.

Versions of packages liblwp-protocol-https-perl suggests:
pn libcrypt-ssleay-perl <none>

-- no debconf information
Slaven Rezic
2021-01-02 09:40:01 UTC
Permalink
Post by Slaven Rezic
Package: liblwp-protocol-https-perl
Version: 6.06-2
Severity: normal
Dear Maintainer,
to disable hostname verification in https requests one would set ssl_opts'
$ perl -MLWP::UserAgent -e '$ua=LWP::UserAgent->new;
$ua->ssl_opts(verify_hostname=>0); $res =
$ua->get("https://www.dwd.de"); warn $res->as_string'
Post by Slaven Rezic
500 Can't connect to www.dwd.de:443 (certificate verify failed)
Content-Type: text/plain
Client-Date: Mon, 03 Sep 2018 05:58:34 GMT
Client-Warning: Internal response
Can't connect to www.dwd.de:443 (certificate verify failed)
SSL connect attempt failed error:1416F086:SSL
routines:tls_process_server_certificate:certificate verify failed at
/usr/share/perl5/LWP/Protocol/http.pm line 47.
Post by Slaven Rezic
With a self-compiled perl and modules installed from CPAN this works as expected
(in this case there's no artificial 500 response, but a 403 Forbidden response).
I found out that it's possible to workaround the issue with
$ perl -MIO::Socket::SSL=SSL_VERIFY_NONE -MLWP::UserAgent -e
'$ua=LWP::UserAgent->new; $ua->ssl_opts(SSL_verify_mode =>
SSL_VERIFY_NONE, verify_hostname => 0); $res =
$ua->get("https://www.dwd.de"); warn $res->as_string'
Post by Slaven Rezic
The issue is still present on Ubuntu 18.04 which has a newer
version of liblwp-protocol-https-perl. I also don't know if the
problem lies in LWP, LWP::Protocol::https, IO::Socket::SSL,
Net::SSLeay, or any other module.
Debian Release: 9.5
APT prefers stable-updates
APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)
Kernel: Linux 4.9.0-3-amd64 (SMP w/4 CPU cores)
Locale: LANG=C, LC_CTYPE=C (charmap=ANSI_X3.4-1968), LANGUAGE=C (charmap=ANSI_X3.4-1968)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
ii ca-certificates 20161130+nmu1+deb9u1
ii libio-socket-ssl-perl 2.044-1
ii libnet-http-perl 6.12-1
ii libwww-perl 6.15-1
ii perl 5.24.1-3+deb9u4
liblwp-protocol-https-perl recommends no packages.
pn libcrypt-ssleay-perl <none>
-- no debconf information
The problem still exists in debian/testing (libwww-perl 6.50 +
liblwp-protocol-https-perl 6.09-1 installed here):

perl -MLWP::UserAgent -e '$ua=LWP::UserAgent->new; $ua->ssl_opts(verify_hostname=>0); $res = $ua->get("https://quartier-heidestrasse.contempo-webcam.de/"); warn $res->as_string'
500 Can't connect to quartier-heidestrasse.contempo-webcam.de:443 (certificate verify failed)
Content-Type: text/plain
Client-Date: Sat, 02 Jan 2021 09:23:22 GMT
Client-Warning: Internal response

Can't connect to quartier-heidestrasse.contempo-webcam.de:443 (certificate verify failed)

SSL connect attempt failed error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed at /usr/share/perl5/LWP/Protocol/http.pm line 50.
gregor herrmann
2023-07-09 18:50:01 UTC
Permalink
Post by Slaven Rezic
The problem still exists in debian/testing (libwww-perl 6.50 +
perl -MLWP::UserAgent -e '$ua=LWP::UserAgent->new; $ua->ssl_opts(verify_hostname=>0); $res = $ua->get("https://quartier-heidestrasse.contempo-webcam.de/"); warn $res->as_string'
500 Can't connect to quartier-heidestrasse.contempo-webcam.de:443 (certificate verify failed)
Content-Type: text/plain
Client-Date: Sat, 02 Jan 2021 09:23:22 GMT
Client-Warning: Internal response
Can't connect to quartier-heidestrasse.contempo-webcam.de:443 (certificate verify failed)
SSL connect attempt failed error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed at /usr/share/perl5/LWP/Protocol/http.pm line 50.
I just tried your example and I don't get any errors.

This is in today's unstable with libwww-perl/6.71-2 and
liblwp-protocol-https-perl 6.10-1.

Could you please try as well?

(Please note that I'm about to upload
liblwp-protocol-https-perl/6.11-1 to unstable).


Cheers,
gregor
--
.''`. https://info.comodo.priv.at -- Debian Developer https://www.debian.org
: :' : OpenPGP fingerprint D1E1 316E 93A7 60A8 104D 85FA BB3A 6801 8649 AA06
`. `' Member VIBE!AT & SPI Inc. -- Supporter Free Software Foundation Europe
`-
Slaven Rezic
2023-07-11 06:20:01 UTC
Permalink
Post by gregor herrmann
Post by Slaven Rezic
The problem still exists in debian/testing (libwww-perl 6.50 +
perl -MLWP::UserAgent -e '$ua=LWP::UserAgent->new; $ua->ssl_opts(verify_hostname=>0); $res = $ua->get("https://quartier-heidestrasse.contempo-webcam.de/"); warn $res->as_string'
500 Can't connect to quartier-heidestrasse.contempo-webcam.de:443 (certificate verify failed)
Content-Type: text/plain
Client-Date: Sat, 02 Jan 2021 09:23:22 GMT
Client-Warning: Internal response
Can't connect to quartier-heidestrasse.contempo-webcam.de:443 (certificate verify failed)
SSL connect attempt failed error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed at /usr/share/perl5/LWP/Protocol/http.pm line 50.
I just tried your example and I don't get any errors.
This is in today's unstable with libwww-perl/6.71-2 and
liblwp-protocol-https-perl 6.10-1.
Could you please try as well?
(Please note that I'm about to upload
liblwp-protocol-https-perl/6.11-1 to unstable).
Confirmed. The former examples cannot be used anymore to prove the
problem, as the used websites fixed their certificates in the meantime.
But you can use something like "https://bla.bla.bing.com" which now
works if verify_hostname=>0 is set. Tried on debian:stretch and
debian:bookworm.

Regards, Slaven

Loading...