Recently while working on docker image build, running on RHEL7, we encountered an issue with pip install. The job was failing with error “RuntimeError: can't start new thread”
Here is how Jenkins error logs looked like –
Bash
14:09:59 Step 6/14 : RUN pip install -r 'requirements.txt'
14:09:59 ---> Running in 0ae2d5afacc0
14:10:06 Collecting pandas==1.5.2
14:10:06 Downloading pandas-1.5.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.2 MB)
14:10:06 [91mERROR: Exception:
14:10:06 Traceback (most recent call last):
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
14:10:06 status = run_func(*args)
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
14:10:06 return func(self, options, args)
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_internal/commands/install.py", line 419, in run
14:10:06 requirement_set = resolver.resolve(
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
14:10:06 result = self._result = resolver.resolve(
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
14:10:06 state = resolution.resolve(requirements, max_rounds=max_rounds)
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 348, in resolve
14:10:06 self._add_to_criteria(self.state.criteria, r, parent=None)
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/resolvers.py", line 172, in _add_to_criteria
14:10:06 if not criterion.candidates:
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
14:10:06 return bool(self._sequence)
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
14:10:06 return any(self)
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
14:10:06 return (c for c in iterator if id(c) not in self._incompatible_ids)
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
14:10:06 candidate = func()
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
14:10:06 self._link_candidate_cache[link] = LinkCandidate(
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 297, in __init__
14:10:06 super().__init__(
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 162, in __init__
14:10:06 self.dist = self._prepare()
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 231, in _prepare
14:10:06 dist = self._prepare_distribution()
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
14:10:06 return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 491, in prepare_linked_requirement
14:10:06 return self._prepare_linked_requirement(req, parallel_builds)
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 536, in _prepare_linked_requirement
14:10:06 local_file = unpack_url(
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 166, in unpack_url
14:10:06 file = get_http_url(
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_internal/operations/prepare.py", line 107, in get_http_url
14:10:06 from_path, content_type = download(link, temp_dir.path)
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_internal/network/download.py", line 147, in __call__
14:10:06 for chunk in chunks:
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_internal/cli/progress_bars.py", line 52, in _rich_progress_bar
14:10:06 with progress:
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/rich/progress.py", line 1169, in __enter__
14:10:06 self.start()
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/rich/progress.py", line 1160, in start
14:10:06 self.live.start(refresh=True)
14:10:06 File "/usr/local/lib/python3.9/site-packages/pip/_vendor/rich/live.py", line 132, in start
14:10:06 self._refresh_thread.start()
14:10:06 File "/usr/local/lib/python3.9/threading.py", line 899, in start
14:10:06 _start_new_thread(self._bootstrap, ())
14:10:06 RuntimeError: can't start new thread
14:10:06 [0m[91m
14:10:06 [notice] A new release of pip is available: 23.0.1 -> 23.1.2
14:10:06 [notice] To update, run: pip install --upgrade pip
14:10:06 [0mThe command '/bin/sh -c pip install -r 'requirements.txt'' returned a non-zero code: 2 Here is how docker version looked like –
Bash
docker version
Client: Docker Engine - Community
Version: 19.03.14
API version: 1.40
Go version: go1.13.15
Git commit: 5eb3275
Built: Tue Dec 1 19:14:24 2020
OS/Arch: linux/amd64
Experimental: false OS Version
Bash
[pocguru@workernode ~]$ uname -a
Linux workernode 3.10.0-1160.88.1.el7.x86_64 #1 SMP Sat Feb 18 13:27:00 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux Dockerfile
Bash
# Set base image
FROM registry.access.redhat.com/ubi9/python-39
#FROM python:3.9
ADD src/ ./
RUN pip install --upgrade pip
RUN pip install --verbose --debug -r 'requirements.txt'
CMD [ "tail", "-f", "/dev/null" ] Resolution
After some troubleshooting, we decided to update docker client version as it looked pretty outdated. And after updating docker version to 24.0.2, issue got resolved.
The reason for this seems to be old docker client’s incompatibility with latest OS libraries, but we didn’t go very deep into knowing the cause of this issue. Hope this will help someone out there.
Thank you.