Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prevent slowdown in yum due to high file descriptor count #1960

Merged
merged 3 commits into from
Jun 24, 2024
Merged

Prevent slowdown in yum due to high file descriptor count #1960

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4f909fd
Prevent slowdown in yum due to high file descriptor count
vepadulano Jun 24, 2024
d2e3666
Update build_steps.sh.tmpl
beckermr Jun 24, 2024
fb81f23
Update build_steps.sh.tmpl
beckermr Jun 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions conda_smithy/templates/build_steps.sh.tmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,10 @@ setup_conda_rc "${FEEDSTOCK_ROOT}" "${RECIPE_ROOT}" "${CONFIG_FILE}"
{% if build_setup -%}
{{ build_setup }}{% endif -%}
{% if yum_build_setup is defined -%}
# Due to https://bugzilla.redhat.com/show_bug.cgi?id=1537564 old versions of rpm
# are drastically slowed down when the number of file descriptors is very high.
# This can be visible during a `yum install` step of a feedstock build.
ulimit -n 1024
{{ yum_build_setup }}{% endif -%}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's put this in a subshell to make this as locally as possible.

Suggested change
# Due to https://bugzilla.redhat.com/show_bug.cgi?id=1537564 old versions of rpm
# are drastically slowed down when the number of file descriptors is very high.
# This can be visible during a `yum install` step of a feedstock build.
ulimit -n 1024
{{ yum_build_setup }}{% endif -%}
(
# Due to https://bugzilla.redhat.com/show_bug.cgi?id=1537564 old versions of rpm
# are drastically slowed down when the number of file descriptors is very high.
# This can be visible during a `yum install` step of a feedstock build.
# => Set a lower limit in a subshell for the `yum install`s only.
ulimit -n 1024
{{ yum_build_setup }}
){% endif -%}

(Alternatively, we could also put this is in yum_build_setup directly; IDK what the preference would be.)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(unresolved for visibility)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, we could also put this is in yum_build_setup directly

Handling in PR: #1961

This also fixed a formatting issue Jinja blank space cleanup otherwise introduces


# make the build number clobber
Expand Down
26 changes: 26 additions & 0 deletions news/1958_yum_fix_ulimit.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
**Added:**

* <news item>

**Changed:**

* Changed the build_steps.sh template so that it sets the number of maximum file
descriptors to 1024. This is done to mitigate a bug in old rpm versions (such
as the one shipped with the Centos7 container) that cause the yum install step
to take tremendously longer than necessary. See https://bugzilla.redhat.com/show_bug.cgi?id=1537564

**Deprecated:**

* <news item>

**Removed:**

* <news item>

**Fixed:**

* <news item>

**Security:**

* <news item>
Loading