Packaging Python modules
Packaging Python Modules
There are a large number of packages which make up "Python" on PCLinuxOS. Not only is there the main python3 package which contains the interpreter plus the standard library but also a large number of additional python modules which extend the capabilities of the language and provide ready-made python code which can be used by applications. These modules are contained in separate packages (one for each module).
Keeping a standard format for these module packages will greatly help us maintain Python on PCLinuxOS by allowing a degree of automation when a new release requires the rebuilding of hundreds of packages.
The vast majority of modules can be covered by the following boilerplate SPEC where you can just change the module name, version, description etc. as appropriate.
%global pypi_name iniconfig
Name: python-%{pypi_name}
Version: 2.1.0
Release: %mkrel 1
Summary: Brain-dead simple config-ini parsing
Group: Development/Python
License: MIT
URL: https://pypi.org/project/iniconfig
Source0: %{pypi_name}-%{version}.tar.xz
BuildArch: noarch
BuildRequires: python3-devel
%description
iniconfig is a small and simple INI-file parser module.
%package -n python3-%{pypi_name}
Summary: %{summary}
%{?python_provide:%python_provide python3-%{pypi_name}}
%description -n python3-%{pypi_name}
iniconfig is a small and simple INI-file parser module.
%prep
%autosetup -n %{pypi_name}-%{version}
%generate_buildrequires
%pyproject_buildrequires
%build
%pyproject_wheel
%install
%pyproject_install
%pyproject_save_files %{pypi_name}
%files -n python3-%{pypi_name} -f %{pyproject_files}
%doc LICENSE
%doc README.rst
%changelog
Naming conventions
Each SPEC file should start by defining a tag identifying the module e.g.
%global pypi_name iniconfig
This defines the name of the module and should be used throughout the SPEC wherever the name of the module is required. Here we have used the preferred pypi_name as the tag but you may see other things like module or srcname. It doesn't matter too much what you use as long as it is consistent and obvious
Our standard requires that the name of the SRPM conforms to the form python-module_name so the SPEC will have
Name: python-%{pypi_name}
This will generate an SRPM called python-initconfig. Similarly the SPEC file should be named pclos-python-iniconfig. Do NOT use versioned names like python3-requests for the source artifacts.
Since the same SRPM can be used to generate packages for different versions of python this should be accounted for by creating sub-packages for each version of python.
%package -n python3-%{pypi_name}
Important Macros
Please don't hardcode the pathnames to the Python site-packages directories or the Python version as this will require us to edit the SPEC rather than just rebuild for a new version of Python. There are a number of macros which should be used when a path or version are required:
%{python3_sitelib}
Python modules are stored in a standard directory on the system where the python interpreter will search for them. The macro %{python3_sitelib} expands to this base directory where the module is stored. As an example, in python 3.10 this directory will be /usr/lib/python3.10/site-packages.
%{python3_sitearch}
Some packages build extension module libraries which will be architecture specific. In this case use the macro %{python3_sitearch} to specify the module base directory instead of %{python3_sitelib}. The difference is that this is based under /usr/lib64 rather than /usr/lib
%{python3_version}
This expands to the version of python being packaged for (e.g. 3.10).
Here is an example of these macros in use:
%files -n python3-%{pypi_name}
%doc LICENSE README.md HISTORY.md
%{python3_sitelib}/%{pypi_name}-%{version}-py%{python3_version}.dist-info
%{python3_sitelib}/%{pypi_name}/
%{python_provide}
This ensures that proper versioned Provides are generated for compatibility with current and prior module naming schemes. This includes the pythonX.Y Provides which are used by the automatic dependency generation system. In addition it will generate a legacy (unversioned) Provide as well as Obsoletes for any prior versions. Typical usage is:
%package -n python3-%{pypi_name}
Summary: %{summary}
%{?python_provide:%python_provide python3-%{pypi_name}}
The result of this is generation of Provides for python3-iniconfig, python3.13-iniconfig and python-iniconfig.
Building
After Python 3.10 distutils is no longer available. It was distutils that provided the setup.py method of building Python modules. Nowadays we use the PEP 517 (pyproject) method for building modules. For reference the old setup.py method is documented later on this page for completeness.
The problem with the legacy setup.py approach is that it is not known what packages that file depends on. It is not possible to reliably introspect Python code without executing it which will trigger all global level imports. This presents a nasty chicken/egg problem for projects which want to use something other than setuptools (e.g. flit) to build the module. To solve this a new Python standard has been created (PEP 517 and PEP 518) which introduces a new file pyproject.toml which specifies the requirements for the module and will used to control the build/install.
Generating Dependencies
All SPEC files for building Python Modules should have:
BuildRequires: python3-devel
This will pull in the Python Development library and all the necessary macro packages. This includes python3-rpm-generators which will generate the necessary Provides: and Requires: for the Python dependencies of the package so it is no longer necessary to add those manually to the SPEC (other non-Python dependencies still need to be specified).
The main advantage of the modern building method is that the python build dependencies can be read from the source prior to building. The SPEC file should have a %generate_buildrequires section and in that section you can use the macro %pyproject_buildrequires to generate the build dependencies.
%generate_buildrequires %pyproject_buildrequires
When rpmbuild is run against the SPEC any missing build dependencies will be flagged:
error: Failed build dependencies:
python3dist(flit-core) >= 3.8 is needed by python-wheel-0.45.1-3pclos2025.noarch
Wrote: /home/terry/src/rpm/SRPMS/python-wheel-0.45.1-3pclos2025.buildreqs.nosrc.rpm
Note that an SRPM (.buildreqs.nosrc.rpm) has been created. This SRPM will have any missing build dependencies as it's requires
[terry@Builder SRPMS]$ rpm -qp --requires python-wheel-0.45.1-3pclos2025.buildreqs.nosrc.rpm ... python3dist(flit-core) >= 3.8 python3dist(packaging) python3dist(pip) >= 19
This means that you can run something like:
[terry@Builder SRPMS]$ dnf builddep ./python-wheel-0.45.1-3pclos2025.buildreqs.nosrc.rpm
to install the dependencies.
If using pkgutils the br_install command (or Right Click equivalent) will also still work on the SPEC file (but may need to be done twice, once to get the BuildRequires and then again for the generate_buildrequires)
Building the package
The following macros should be used to do the actual build and install phases:
%build %pyproject_wheel %install %pyproject_install
There is also a macro which can generate a %files list for the files which make up the module.
%pyproject_save_files %{pypi_name}
The file list can than be used in the %files section but note that you still have to manually add any doc and licence files plus any files which are outside the module tree (e.g. in %{_bindir}).
%files -n python3-%{pypi_name} -f %{pyproject_files}
%doc README LICENSE
%{_bindir}/keyring
If the source builds multiple modules they can be specified as additional arguments separated by spaces
HINT: if on building you see this error:
FileExistsError: %pyproject install has found more than one *.dist-info/RECORD file. Currently, %pyproject_save_files supports only one wheel → one file list mapping. Feel free to open a bugzilla for pyproject-rpm-macros and describe your usecase.
The delete the file pyproject-record under the BUILD directory and re-try the build.
Legacy setup.py building
This is a legacy interface which the Python developers have moved away from because it has no good mechanisms for declaring build dependencies. The majority of Python modules are distributed as a .tar.gz file (sometimes called an sdist or source distribution). Usually this will have been generated using distutils which has been the standard for packaging Python modules for a long time. Modules packaged with distutils come with a setup.py which will build and install the module.
Packaging such a module for PCLinuxOS should be as simple as:
%build %py3_build %install %py3_install
These macros will automatically run the setup.py with appropriate parameters to generate the module under the BUILDROOT.
The files produced then need to be packaged in the %files section. The general format will be:
%files -n python3-%{module}
%doc LICENSE README.md HISTORY.md
%{python3_sitelib}/%{module}-%{version}-py%{python3_version}.egg-info
%{python3_sitelib}/%{module}/
Note here important use of macro %{python3_sitelib}. Python modules are stored in a standard directory on the system where the python interpreter will search for them. The macro %{python3_sitelib} expands to this base directory where the module is stored. As an example, in python 3.10 this directory will be /usr/lib/python3.10/site-packages.
Some packages build extension module libraries which will be architecture specific. In this case use the macro %{python3_sitearch} to specify the module base directory instead of %{python3_sitelib}. The difference is that this is based under /usr/lib64 rather than /usr/lib
Another important macro is %{python3_version} which expands to the version of python being packaged for (e.g. 3.10). You can see it's use above in the definition of the path to the egg-info.
You should always use these macros. Do not hard-code paths or version numbers as this will mean we need to edit the SPEC file rather than just rebuild when we move to a new python version.