Discussion:
Bug#948471: libconfig-model-dpkg-perl: `value 'BSD~unspecified and/or CC0' does not match grammar from model` on scikit-learn/0.22.1:examples/cluster/plot_agglomerative_clustering_metrics.py
Add Reply
Sandro Tosi
2020-01-09 01:10:01 UTC
Reply
Permalink
Package: libconfig-model-dpkg-perl
Version: 2.122
Severity: normal

Hello,
while packaging scikit-learn/0.22.1 and running

`cme update dpkg-copyright`

it generated an error, requesting to submit a bugreport, so here it is:

```
Offending value: 'Please fill license BSD~unspecified and/or CC0 from header of examples/cluster/plot_agglomerative_clustering_metrics.py'
Error: Data extracted from source file is corrupted:
Configuration item 'Files:"examples/cluster/plot_agglomerative_clustering_metrics.py" License short_name' has a wrong value:
value 'BSD~unspecified and/or CC0' does not match grammar from model
This usually mean that cme or licensecheck (or both) have a bug. You may work-around this issue by adding an override entry in fill.copyright.blanks file. See https://github.com/dod38fr/config-model/wiki/Updating-debian-copyright-file-with-cme for instructions. Last but not least, please file a bug against libconfig-model-dpkg-perl.
```

File is available at: https://github.com/scikit-learn/scikit-learn/blob/0.22.1/examples/cluster/plot_agglomerative_clustering_metrics.py

Thanks,
Sandro


-- System Information:
Debian Release: 10.0
APT prefers unstable-debug
APT policy: (500, 'unstable-debug'), (500, 'unstable'), (1, 'experimental-debug'), (1, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.19.0-5-amd64 (SMP w/8 CPU cores)
Kernel taint flags: TAINT_PROPRIETARY_MODULE, TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE= (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages libconfig-model-dpkg-perl depends on:
ii libapt-pkg-perl 0.1.36+b2
ii libarray-intspan-perl 2.003-1
ii libconfig-model-backend-yaml-perl 2.133-2
ii libconfig-model-perl 2.133-1
ii libexporter-lite-perl 0.08-1
ii liblog-log4perl-perl 1.49-1
ii libmouse-perl 2.5.6-1+b2
ii libparse-recdescent-perl 1.967015+dfsg-2
ii libsoftware-licensemoreutils-perl 1.004-1
ii libtext-autoformat-perl 1.740000-2
ii libtext-levenshtein-damerau-perl 0.41-1
ii liburi-perl 1.76-1
ii libwww-perl 6.36-2
ii libyaml-perl 1.27-1
ii licensecheck 3.0.31-3
ii lintian 2.41.0
ii perl [libmodule-corelist-perl] 5.30.0-8
ii perl-modules-5.24 [libmodule-corelist-perl] 5.24.1-3+deb9u2
ii perl-modules-5.26 [libmodule-corelist-perl] 5.26.2-2

Versions of packages libconfig-model-dpkg-perl recommends:
ii libconfig-model-tkui-perl 1.369-2

libconfig-model-dpkg-perl suggests no packages.

-- no debconf information
Dominique Dumont
2020-01-12 18:30:01 UTC
Reply
Permalink
Post by Sandro Tosi
value 'BSD~unspecified and/or CC0' does not match grammar from model
This usually mean that cme or licensecheck (or both) have a bug. You may work-around this issue by adding an override entry in fill.copyright.blanks file. See https://github.com/dod38fr/config-model/wiki/Updating-debian-copyright-file-with-cme for instructions. Last but not least, please file a bug against libconfig-model-dpkg-perl.
```
This error is triggered by the "and/or" string in the license short_name. This "and/or" is produced by licensecheck (used by cme when updating copyright data).

Since libconfig-model-dpkg-perl 2.124 (released in July 2019), cme contains code to replace 'and/or' with plain 'or' so the resulting license short name complies with debian copyright format.

If possible, please upgrade libconfig-model-dpkg-perl.

I've tried 'cme update dpkg-copyright' on scikit-learn commit debian~2 and the update finishes, but the result is not correct.


First, licensecheck does not parse correctly the files containing this style of license specification:
# License: BSD 3 clause

licensecheck should produce 'BSD (3 clause)' instead of 'BSD (unspecified)'

Second, "licensecheck --encoding utf8" exits on error when parsing binary files (png files in this case):

$ licensecheck --encoding utf8 --copyright --machine --deb-fmt --recursive .
[snip]
./whats_new/v0.21.rst UNKNOWN *No copyright*
./whats_new/v0.22.rst UNKNOWN *No copyright*
utf8 "\xFF" does not map to Unicode at /usr/share/licensecheck/App/Licensecheck.pm line 358.
$ echo $?
130

Unfortunately, cme does not detect this error so the entries not parsed by licensecheck are missing. I'm going to fix this bug so cme also exit on error when licensecheck fails.

Jonas, can you check what's going on with licensecheck ?

All the best

Dod
Jonas Smedegaard
2020-01-13 12:10:01 UTC
Reply
Permalink
Quoting Dominique Dumont (2020-01-12 19:23:54)
Post by Dominique Dumont
This error is triggered by the "and/or" string in the license
short_name. This "and/or" is produced by licensecheck (used by cme
when updating copyright data).
Yes, licensecheck deliberately uses "and/or" when it is uncertain if the
user must comply with _both_ licenses or they are permitted to pick
_either_ of them.
Post by Dominique Dumont
Since libconfig-model-dpkg-perl 2.124 (released in July 2019), cme
contains code to replace 'and/or' with plain 'or' so the resulting
license short name complies with debian copyright format.
It worries me that cme is making such blind assumptions about licensing
relationships!
Post by Dominique Dumont
# License: BSD 3 clause
licensecheck should produce 'BSD (3 clause)' instead of 'BSD (unspecified)'
[...]
Post by Dominique Dumont
Jonas, can you check what's going on with licensecheck ?
Please file bugs against licensecheck, and reference them when
mentioning them elsewhere - e.g. in convoluted bugreports like this.

I will happily examine closer, but not easy to handle from within this
other bugreport.


- Jonas
--
* Jonas Smedegaard - idealist & Internet-arkitekt
* Tlf.: +45 40843136 Website: http://dr.jones.dk/

[x] quote me freely [ ] ask before reusing [ ] keep private
Dominique Dumont
2020-01-14 14:00:02 UTC
Reply
Permalink
Post by Jonas Smedegaard
Post by Dominique Dumont
Since libconfig-model-dpkg-perl 2.124 (released in July 2019), cme
contains code to replace 'and/or' with plain 'or' so the resulting
license short name complies with debian copyright format.
It worries me that cme is making such blind assumptions about licensing
relationships!
Grumble... You're right.

I'll update cme to warn instead of forbid when a "and/or" is found in debian/
copyright.

The user will be instructed to check the file content and update debian/
fill.copyright.blanks to suppress the warning.

Dear Debian-perl colleagues, do you have a better idea ?
Post by Jonas Smedegaard
Please file bugs against licensecheck, and reference them when
mentioning them elsewhere - e.g. in convoluted bugreports like this.
Done:
* https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=948890
* https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=948891

All the best

Loading...