Bug#1085728: python-multipart: Make room for https://pypi.org/project/multipart/ somehow?

[Colin Watson]

Source: python-multipart
Version: 0.0.9-1
Severity: wishlist

Please see https://github.com/pypa/packaging-problems/issues/818 for
background on this.

The multipart and python-multipart packages on PyPI have historically
had a namespace collision in terms of the files they install, which
makes it difficult to use both. multipart is recommended as a
replacement for parts of the old cgi module that was removed from the
standard library in Python 3.13, and so we're starting to have problems
with this in Debian. I've already needed to temporarily vendor a copy
of multipart into another source package, which is tolerable given that
it's a single-file module and in the case in question was only used in
tests, but it's still not ideal.

Fortunately, there has been progress on this upstream:
python-multipart's import name is being changed from "multipart" to
"python_multipart", but in a backward-compatible way. This means that
more upstreams are likely to add dependencies on multipart without fear
of causing incompatibilities with e.g. FastAPI.

So, how would we deal with this in Debian? It isn't currently possible
to package multipart in the usual way because the python-multipart
package has its natural binary package name (python3-multipart).

My preference would be to follow PyPI's naming and rename as follows:

* python-multipart source: python-python-multipart
* python3-multipart binary: python3-python-multipart

And although it's a bit odd, we could package the new python3-multipart
with a transitional dependency on python3-python-multipart until all its
real reverse-dependencies have been updated in a stable release.

But maybe you have another idea?

Thanks,

[Colin Watson]

This bug is starting to accumulate other Python 3.13-related bugs that
it blocks, so I'd like to agree a plan.

Post by Colin Watson
Please see https://github.com/pypa/packaging-problems/issues/818 for
background on this.

This has been mostly sorted out upstream. It should now be possible to
use both multipart and python-multipart in parallel, and
python-multipart has some transitional support for projects that do
`import multipart` as long as only python-multipart is installed. In a
Debian context, this is a stricter criterion, because it's more likely
to have both packages installed when they're system packages than when
they're only in a project-specific virtualenv; but it should still be
manageable enough.

I believe these are all the reverse-dependencies involved, and I've CCed
all the affected maintainers (assuming they read their
@packages.debian.org email, anyway):

$ reverse-depends src:python-multipart
Reverse-Recommends
==================
* python3-moto (for python3-multipart)
* python3-starlette (for python3-multipart)

Reverse-Depends
===============
* matrix-synapse [amd64 arm64 armel armhf i386 mips64el ppc64el riscv64 s390x]
* python3-asgi-csrf (for python3-multipart)
* python3-gavo (for python3-multipart)

Packages without architectures listed are reverse-dependencies in: all, amd64, arm64, armel, armhf, i386, mips64el, ppc64el, riscv64, s390x

$ reverse-depends -b src:python-multipart
Reverse-Testsuite-Triggers
==========================
* asgi-csrf (for python3-multipart)
* python-moto (for python3-multipart)

Reverse-Build-Depends
=====================
* fastapi (for python3-multipart)
* matrix-synapse (for python3-multipart)
* python-curies (for python3-multipart)
* starlette (for python3-multipart)

Reverse-Build-Depends-Indep
===========================
* asgi-csrf (for python3-multipart)
* python-moto (for python3-multipart)

So, my proposal is as follows:

* Rename python-multipart source package to python-python-multipart,
and its binary package to python3-python-multipart. Yes, it's a bit
ugly, but there are currently 13 other binary packages in the archive
whose names start with "python3-python-", so there's precedent, and
it matches the way we usually transform PyPI names into Debian
package names. python3-python-multipart should have Breaks/Replaces:
python3-multipart (<< 0.1) (the oldest release of
https://pypi.org/project/multipart/, which is also newer than any
release of https://pypi.org/project/python-multipart/).

* Since there are only seven other source packages involved (asgi-csrf,
fastapi, gavodachs, matrix-synapse, python-curies, python-moto, and
starlette), it should be fairly straightforward to update all their
Depends/Recommends/Build-Depends/Build-Depends-Indep/Testsuite-Triggers
to use python3-python-multipart.

* Once all this has reached testing, the way will be clear to upload a
new python-multipart source package based on
https://pypi.org/project/multipart/, building a new python3-multipart
binary package. Conveniently, the version numbers of multipart and
python-multipart upstream are such that this won't require an epoch.
The new python3-multipart binary package should declare Breaks on old
versions of all the packages where we had to update run-time
dependencies (matrix-synapse, python3-asgi-csrf, python3-gavo,
python3-moto, and python3-starlette).

How does this sound? Sandro, I'd especially like to get your feedback
on this, because doing a source+binary package rename in an NMU sounds
... painful, especially if it turns out later that there wasn't
consensus.

Thanks,

[Simon McVittie]

Post by Colin Watson
* Rename python-multipart source package to python-python-multipart,
and its binary package to python3-python-multipart. Yes, it's a bit
ugly, but there are currently 13 other binary packages in the archive
whose names start with "python3-python-", so there's precedent, and
it matches the way we usually transform PyPI names into Debian
package names.

The policy says we do not transform PyPI names into Debian package names
(or at least, not binary package names). Instead, we transform the name
that you can `import` into a Debian binary package name.

But if src:python-multipart is moving towards the name that you should
canonically import being `python_multipart`, then its correct binary
package name would be python3-python-multipart anyway.

smcv

[Colin Watson]

Post by Simon McVittie

Post by Colin Watson
* Rename python-multipart source package to python-python-multipart,
and its binary package to python3-python-multipart. Yes, it's a bit
ugly, but there are currently 13 other binary packages in the archive
whose names start with "python3-python-", so there's precedent, and
it matches the way we usually transform PyPI names into Debian
package names.

The policy says we do not transform PyPI names into Debian package names
(or at least, not binary package names). Instead, we transform the name
that you can `import` into a Debian binary package name.
But if src:python-multipart is moving towards the name that you should
canonically import being `python_multipart`, then its correct binary
package name would be python3-python-multipart anyway.

Thanks for the correction. I agree.

[Julian Gilbey]

Hi Colin,

Post by Colin Watson
This bug is starting to accumulate other Python 3.13-related bugs that
it blocks, so I'd like to agree a plan.
[...]
* Rename python-multipart source package to python-python-multipart,
and its binary package to python3-python-multipart. Yes, it's a bit
ugly, but there are currently 13 other binary packages in the archive
whose names start with "python3-python-", so there's precedent, and
it matches the way we usually transform PyPI names into Debian
python3-multipart (<< 0.1) (the oldest release of
https://pypi.org/project/multipart/, which is also newer than any
release of https://pypi.org/project/python-multipart/).
[...]
How does this sound? Sandro, I'd especially like to get your feedback
on this, because doing a source+binary package rename in an NMU sounds
... painful, especially if it turns out later that there wasn't
consensus.

My thoughts...

(1) I agree with this renaming and Colin's general proposals,
especially as upstream has now renamed their import.

(2) Please take a look at
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1073445#25 (though
I'm not proposing changing the package name in this NMU).

(3) I would suggest tweaking Colin's proposal: the other binary/source
packages currently (build-)depending on python3-multipart should all
be upgraded *first* to change their python3-multipart dependency to
read:

python3-multipart (<< 0.1) | python3-python-multipart

This way, when the python3-multipart package is renamed, these other
packages will continue to work without change (unless the newer
version of python3-multipart that I'm planning to NMU breaks them
anyway).

(4) With the maintainer's current non-responsiveness, it may make
sense to salvage these three interlinked packages (fastapi,
python-multipart, sqlmodel). That will take longer than an NMU, but
will help with the longer-term maintenance of these packages. (I
would propose transferring them to the Debian Python Team if they are
salvaged.)

Best wishes,

Julian

[Andreas Tille]

Post by Julian Gilbey
(4) With the maintainer's current non-responsiveness, it may make
sense to salvage these three interlinked packages (fastapi,
python-multipart, sqlmodel). That will take longer than an NMU, but
will help with the longer-term maintenance of these packages. (I
would propose transferring them to the Debian Python Team if they are
salvaged.)

+1 for transferring packages to DPT

Thank you all for working on this
Andreas.

[Colin Watson]

Post by Julian Gilbey
(2) Please take a look at
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1073445#25 (though
I'm not proposing changing the package name in this NMU).

Looks reasonable, though I haven't looked in much detail ...

Post by Julian Gilbey
(3) I would suggest tweaking Colin's proposal: the other binary/source
packages currently (build-)depending on python3-multipart should all
be upgraded *first* to change their python3-multipart dependency to
python3-multipart (<< 0.1) | python3-python-multipart
This way, when the python3-multipart package is renamed, these other
packages will continue to work without change (unless the newer
version of python3-multipart that I'm planning to NMU breaks them
anyway).

That might be a good idea, but I wonder if the order should be flipped:

python3-python-multipart | python3-multipart (<< 0.1)

My instinct is that apt will behave slightly better that way, but even
if I'm wrong about that, it seems better to put the eventually-preferred
name first.

Post by Julian Gilbey
(4) With the maintainer's current non-responsiveness, it may make
sense to salvage these three interlinked packages (fastapi,
python-multipart, sqlmodel). That will take longer than an NMU, but
will help with the longer-term maintenance of these packages. (I
would propose transferring them to the Debian Python Team if they are
salvaged.)

I'm not very familiar with the salvaging process, but I don't object.
Judging from
https://www.debian.org/doc/manuals/developers-reference/pkgs.en.html#package-salvaging
it looks like that takes a minimum of about a month, giving plenty of
time for the maintainer to react and stop the process? And would you
like to drive that (since I suppose I might be seen as having a slight
conflict of interest due to being an upstream committer to multipart)?

[Sandro Tosi]

Post by Julian Gilbey
it may make
sense to salvage these three interlinked packages (fastapi,
python-multipart, sqlmodel).

I dont think so.

[Julian Gilbey]

Post by Sandro Tosi

Post by Julian Gilbey
it may make
sense to salvage these three interlinked packages (fastapi,
python-multipart, sqlmodel).

I dont think so.

Evidence:

fastapi: last upload March 2024, RC bug since June 2024, no response
from maintainer

python-multipart: last upload March 2024, bug report about renaming on
22 October, no response from maintainer

sqlmodel: last upload June 2024, RC bug since August 2024, no response
from maintainer


From Developers Reference:

5.12. Package Salvaging
=======================

Package salvaging is the process by which one attempts to save a
package that, while not officially orphaned, appears poorly maintained
or completely unmaintained. ...


I think the above evidence makes these packages eminently eligible for
the salvaging process. If you want to start actively maintaining
these packages, please feel free to do so - I have no personal
interest or capacity to maintain them myself; I have only proposed
this set of NMUs because they are blocking another package from
building. But if you don't have the time or interest to do so, please
do hand them on to someone who does.

Best wishes,

Julian

[Sandro Tosi]

Post by Colin Watson
* python-multipart source: python-python-multipart
* python3-multipart binary: python3-python-multipart
And although it's a bit odd, we could package the new python3-multipart
with a transitional dependency on python3-python-multipart until all its
real reverse-dependencies have been updated in a stable release.
But maybe you have another idea?

yes i do. I just uploaded python-multipart_0.0.17-2 to experimental
with the *binary* package name change from python3-multipart to
python3-python-multipart (common python3- prefix plus the module
import name) and without the compatibility `multipart` module in it.
Once it clears NEW, please check it does what you need it to do.

I will not rename the source package. PyPI has it as python-multipart,
upstream gh has it as python-multipart, and debian already has it as
src:python-multipart so that's not gonna change.

the module you want to introduce is named `multipart` in PyPI, so you
can introduce a package in debian with src:multipart name (no, there's
no requirement whatsoever to have the `python` prefix on source
package names) which is currently available, and a
bin:python3-multipart, probably with a conflict with the old version
of that package.

Along with this, i will *not* handle the transition from the old
bin:python3-multipart for its reverse dependencies, so please have a
look if you wanna take care of that or ask the support of someone
else.

Once you confirm the 0.0.17-2 package works as for your needs, i'll
upload it to unstable and your work can continue,

Regards,

[Julian Gilbey]

[Cc: ftpmaster - please see the discussion below and, if necessary,
the earlier discussion in this bug report. We would really like your
input on this bug - how should we proceed?]

Post by Sandro Tosi

Post by Colin Watson
* python-multipart source: python-python-multipart
* python3-multipart binary: python3-python-multipart
And although it's a bit odd, we could package the new python3-multipart
with a transitional dependency on python3-python-multipart until all its
real reverse-dependencies have been updated in a stable release.
But maybe you have another idea?

yes i do. I just uploaded python-multipart_0.0.17-2 to experimental
with the *binary* package name change from python3-multipart to
python3-python-multipart (common python3- prefix plus the module
import name) and without the compatibility `multipart` module in it.
Once it clears NEW, please check it does what you need it to do.
I will not rename the source package. PyPI has it as python-multipart,
upstream gh has it as python-multipart, and debian already has it as
src:python-multipart so that's not gonna change.
the module you want to introduce is named `multipart` in PyPI, so you
can introduce a package in debian with src:multipart name (no, there's
no requirement whatsoever to have the `python` prefix on source
package names) which is currently available, and a
bin:python3-multipart, probably with a conflict with the old version
of that package.

Unfortunately, the ftpmasters have made the decision that they now
require new source package names that are specific to Python to have a
source package name of python-* (unless the package name is already
clearly Python specific). So we will have to discuss this with them -
copying in ftpmaster for their input on this question.

Post by Sandro Tosi
Along with this, i will *not* handle the transition from the old
bin:python3-multipart for its reverse dependencies, so please have a
look if you wanna take care of that or ask the support of someone
else.
Once you confirm the 0.0.17-2 package works as for your needs, i'll
upload it to unstable and your work can continue,

Best wishes,

Julian

[Thorsten Alteholz]

Hi Julian,

Post by Julian Gilbey
Unfortunately, the ftpmasters have made the decision that they now
require new source package names that are specific to Python to have a
source package name of python-* (unless the package name is already
clearly Python specific).

do you have any pointer to where this decision was published?

Thorste

[Julian Gilbey]

Hi Thorsten,

Post by Thorsten Alteholz
Hi Julian,

Post by Julian Gilbey
Unfortunately, the ftpmasters have made the decision that they now
require new source package names that are specific to Python to have a
source package name of python-* (unless the package name is already
clearly Python specific).

do you have any pointer to where this decision was published?
Thorste

I'm not aware of it being published, but I had lots of comments on
ITPs in recent times, for example:

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1073496

Now, though, I realise that Jonas was not on the ftpmaster team; I had
assumed that this was an ftpmaster team directive and complied without
complaint.

So happy for you to guide us on this one: the two proposed options are
currently

src: python-multipart binary: python3-python-multipart
src: multipart binary: python3-multipart

and

src: python-python-multipart binary: python3-python-multipart
src: python-multipart binary: python3-multipart

("multipart" is not currently in the archive.)

I certainly prefer the latter option, being less confusing, but now
that it's not an ftpmaster policy, you may prefer to not get
involved.

Best wishes,

Julian

[Thorsten Alteholz]

Hi Julian,

Post by Julian Gilbey
I certainly prefer the latter option, being less confusing, but now
that it's not an ftpmaster policy, you may prefer to not get
involved.

Jonas has a valid argument about not polluting the namespace and I tend to
agree with him. But as this has nothing to do with any ftpmaster policy
but is a decision of the Python team, I leave you free to decide what you
like best.

Thorsten

[Colin Watson]

Post by Thorsten Alteholz

Post by Julian Gilbey
I certainly prefer the latter option, being less confusing, but now
that it's not an ftpmaster policy, you may prefer to not get
involved.

Jonas has a valid argument about not polluting the namespace and I tend to
agree with him. But as this has nothing to do with any ftpmaster policy but
is a decision of the Python team, I leave you free to decide what you like
best.

While I would have preferred python-python-multipart (source) →
python3-python-multipart (binary) and python-multipart (source) →
python3-multipart (binary) - as Julian says, we usually prefix "python-"
to source package names nowadays and it avoids awkward conflicts with
packages from other language ecosystems - I think we can live with
python-multipart (source) → python3-python-multipart (binary) and
multipart (source) → python3-multipart (binary) as Sandro proposes and
the first part of which is currently in NEW.

(Failing that, there are more awkward possibilities such as
python-defnull-multipart, including the author's GitHub username. But
on balance that seems worse.)

[Julian Gilbey]

Post by Thorsten Alteholz
Hi Julian,

Post by Julian Gilbey
I certainly prefer the latter option, being less confusing, but now
that it's not an ftpmaster policy, you may prefer to not get
involved.

Jonas has a valid argument about not polluting the namespace and I tend to
agree with him. But as this has nothing to do with any ftpmaster policy but
is a decision of the Python team, I leave you free to decide what you like
best.
Thorsten

Thanks Thorsten!

Best wishes,

Julian

[Colin Watson]

python-multipart (0.0.17-2) experimental; urgency=medium
.
* debian/control
- rename the binary package to python3-python-multipart; Closes: #1085728

Much appreciated!

I've begun testing this with starlette. I tried both "Build-Depends:
python3-multipart (<< 0.1) | python3-python-multipart" (Julian's
suggestion) and "Build-Depends: python3-python-multipart |
python3-multipart (<< 0.1)" (my adjustment). Unfortunately both of
those behave a bit awkwardly right now.

Builds for experimental consider all "|"-separated alternatives, but
builds for unstable only consider the first alternative. Right now,
while python3-multipart (<< 0.1) is in unstable and
python3-python-multipart is in experimental, "Build-Depends:
python3-multipart (<< 0.1) | python3-python-multipart" works fine; but
it will break once python3-multipart (<< 0.1) is no longer in unstable,
requiring another coordinated round of uploads of the seven
reverse-dependencies. That's not ideal. Similarly, flipping the order
means that those seven packages can only be updated once your new
package reaches unstable; python3-python-multipart would have to be
blocked from testing until all the reverse-dependencies have been
updated so that it can add Breaks, and then the whole lot would have to
move to testing at once. Also not really ideal.

How about this patch, still intended for experimental for now? I don't
think it should be annoying to maintain, and I believe it will allow a
smoother transition. It's hard to test this completely without
something quite involved preparing a modified mirror of unstable, but I
did the best I could:

* modified starlette to use "Build-Depends: python3-multipart (<< 0.1)
| python3-python-multipart"

* ran "sbuild -Ad unstable
--extra-package=python3-multipart_1.1.0-1_all.deb
--extra-package=python3-python-multipart_0.0.17-2.1_all.deb
starlette_0.41.3-2.dsc" (using my in-progress multipart packaging and
this patch against python-multipart), and confirmed that it installs
python3-python-multipart and not python3-multipart

diff --git a/debian/changelog b/debian/changelog
index 92aff05..1a93b7d 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,10 @@
+python-multipart (0.0.17-2.1) UNRELEASED; urgency=medium
+
+ * Provide python3-multipart (= ${binary:Version) temporarily, to simplify
+ transitioning build-dependencies to the new package name.
+
+ -- Colin Watson <***@debian.org> Wed, 27 Nov 2024 00:25:23 +0000
+
python-multipart (0.0.17-2) experimental; urgency=medium

* debian/control
diff --git a/debian/control b/debian/control
index 3aa985c..7e8a8b7 100644
--- a/debian/control
+++ b/debian/control
@@ -30,5 +30,6 @@ Depends: ${misc:Depends},
${python3:Depends},
Recommends: ${python3:Recommends},
Suggests: ${python3:Suggests},
+Provides: python3-multipart (= ${binary:Version}),
Description: streaming multipart parser for Python
Python-Multipart is a streaming multipart parser for Python.

Thanks,

[Sandro Tosi]

Post by Colin Watson
--- a/debian/control
+++ b/debian/control
@@ -30,5 +30,6 @@ Depends: ${misc:Depends},
${python3:Depends},
Recommends: ${python3:Recommends},
Suggests: ${python3:Suggests},
+Provides: python3-multipart (= ${binary:Version}),
Description: streaming multipart parser for Python
Python-Multipart is a streaming multipart parser for Python.

done, uploaded to experimental as python-multipart_0.0.17-3


--
Sandro "morph" Tosi
My website: http://sandrotosi.me/
Me at Debian: http://wiki.debian.org/SandroTosi
Twitter: https://twitter.com/sandrotosi

[Colin Watson]

Control: block -1 by 1088569 1088642 1088646 1088647

Post by Sandro Tosi
done, uploaded to experimental as python-multipart_0.0.17-3

Thanks, this looks better.

The next stage is to adjust the reverse-dependencies and wait for all
that to reach testing (if they're currently in testing). Here's the
current status of that:

* asgi-csrf: uploaded 0.11-1 to unstable
* fastapi: https://bugs.debian.org/1088569 (but not currently in
testing, so it would probably be enough to get this fixed in
unstable)
* gavodachs: https://bugs.debian.org/1088642
* matrix-synapse: https://bugs.debian.org/1088647
* python-curies: https://bugs.debian.org/1088646 (but not currently in
testing, so it would probably be enough to get this fixed in
unstable)
* python-moto: this is a weird one; it seems that it actually wants
multipart rather than python-multipart, so I guess it's undertested
in Debian; I guess we can just leave it alone, and maybe bump its
Build-Depends-Indep/Recommends to python3-multipart (>= 0.1) once
that's in unstable
* starlette: uploaded 0.41.3-2 to unstable

[Colin Watson]

Post by Colin Watson

Post by Sandro Tosi
done, uploaded to experimental as python-multipart_0.0.17-3

Thanks, this looks better.
The next stage is to adjust the reverse-dependencies and wait for all
that to reach testing (if they're currently in testing).

This has all now happened. Could you please upload experimental's
python-multipart to unstable? Then, once that reaches testing, I'll
upload the new multipart.

Thanks,