Which distributions enable transparent huge pages “for all applications”?
I am puzzled by LWN again.
Huge pages, slow drives, and long delays -- LWN.net, 2011
It is a rare event, but it is no fun when it strikes. Plug in a slow storage device - a USB stick or a music player, for example - and run something like rsync to move a lot of data to that device. The operation takes a while, which is unsurprising; more surprising is when random processes begin to stall...
A process (that web browser, say) is doing its job when it incurs a page fault ... If the transparent huge pages feature is built into the kernel (and most distributors do enable this feature), the page fault handler will attempt to allocate a huge page.
I can see that on Red Hat Enterprise Linux, transparent hugepages are enabled as described by LWN.
How to use, monitor, and disable transparent hugepages in Red Hat Enterprise Linux 6 and 7?
Transparent Huge Pages (THP) are enabled by default in RHEL 6 for all applications...
For RHEL 7 see How to disable transparent hugepages (THP) on Red Hat Enterprise Linux 7.
For RHEL 8 see How to disable transparent hugepages (THP) on Red Hat Enterprise Linux 8. [Links to content for subscribers only].
However, I looked on my Fedora Workstation 29 laptop, and I see transparent hugepages are not enabled "for all applications".
$ cat /sys/kernel/mm/transparent_hugepage/enabled
always [madvise] never
$ uname -r
4.19.13-300.fc29.x86_64
I also checked in some VMs: I see the same on Ubuntu Desktop 18.04 (kernel 4.15.0-43-generic), and on a Debian 9 Desktop install (kernel 4.9.0-8-amd64). (And I don't believe Debian has a "Server" install, spin, or package. It just has a few different "tasks" to install the most common server softwares. So I don't think there is any one place in Debian that could be enabling a server-specific configuration).
- Does Fedora Server 29 enable transparent hugepages for all apps?
- Does Ubuntu Server 18.04 enable transparent hugepages for all apps?
- Did a different version number of one of these distributions enable transparent hugepages for all apps, e.g. around 2011?
[By implication: was LWN.net correct here, or not? And if it was correct to say that most distributors enabled transparent hugepages, and imply that it affected some common "desktop" (or laptop) setups, do we know why current versions of Fedora Workstation, Debian, and Ubuntu Desktop are not enabling transparent hugepages for all?]
linux linux-kernel huge-pages
asked Jan 21 at 16:24
sourcejedisourcejedi
23.9k439106
add a comment |
I am puzzled by LWN again.
Huge pages, slow drives, and long delays -- LWN.net, 2011
It is a rare event, but it is no fun when it strikes. Plug in a slow storage device - a USB stick or a music player, for example - and run something like rsync to move a lot of data to that device. The operation takes a while, which is unsurprising; more surprising is when random processes begin to stall...
A process (that web browser, say) is doing its job when it incurs a page fault ... If the transparent huge pages feature is built into the kernel (and most distributors do enable this feature), the page fault handler will attempt to allocate a huge page.
I can see that on Red Hat Enterprise Linux, transparent hugepages are enabled as described by LWN.
How to use, monitor, and disable transparent hugepages in Red Hat Enterprise Linux 6 and 7?
Transparent Huge Pages (THP) are enabled by default in RHEL 6 for all applications...
For RHEL 7 see How to disable transparent hugepages (THP) on Red Hat Enterprise Linux 7.
For RHEL 8 see How to disable transparent hugepages (THP) on Red Hat Enterprise Linux 8. [Links to content for subscribers only].
However, I looked on my Fedora Workstation 29 laptop, and I see transparent hugepages are not enabled "for all applications".
$ cat /sys/kernel/mm/transparent_hugepage/enabled
always [madvise] never
$ uname -r
4.19.13-300.fc29.x86_64
I also checked in some VMs: I see the same on Ubuntu Desktop 18.04 (kernel 4.15.0-43-generic), and on a Debian 9 Desktop install (kernel 4.9.0-8-amd64). (And I don't believe Debian has a "Server" install, spin, or package. It just has a few different "tasks" to install the most common server softwares. So I don't think there is any one place in Debian that could be enabling a server-specific configuration).
- Does Fedora Server 29 enable transparent hugepages for all apps?
- Does Ubuntu Server 18.04 enable transparent hugepages for all apps?
- Did a different version number of one of these distributions enable transparent hugepages for all apps, e.g. around 2011?
[By implication: was LWN.net correct here, or not? And if it was correct to say that most distributors enabled transparent hugepages, and imply that it affected some common "desktop" (or laptop) setups, do we know why current versions of Fedora Workstation, Debian, and Ubuntu Desktop are not enabling transparent hugepages for all?]
linux linux-kernel huge-pages
asked Jan 21 at 16:24
sourcejedisourcejedi
23.9k439106
madviseis still enabled. But it's not "system wide". See kernel.org/doc/Documentation/vm/transhuge.txt FYI, I just checked my ubuntu server on AWS and it is also set tomadvise.
– couling
Jan 21 at 16:43
@couling right, presumably an app can ask for transparent hugepages, using the madvise() call if it is enabled. But I don't have any hugepages. So my browser clearly isn't asking for them. Contra the LWN example, a page fault in my browser will not try to allocate a hugepage. So I am trying to guess what made LWN think this was a good example to illustrate the problem...
– sourcejedi
Jan 21 at 16:47
@couling Thanks for pointing out the ambiguity in the title I used! Particularly since it looks like this might be what LWN was confused about. I have edited my question to use the specific phrasing from the RHEL doc.
– sourcejedi
Jan 21 at 16:54
1
Debian had THP enabled by default from kernel 2.6.38 to 4.13, so the kernels for Debian 7 to 9 had it set to “always”. Debian 10 will have it set to “madvise”. Debian 6 was current in 2011, with kernel 2.6.32, and didn’t have transparent hugepages, it had hugetlbfs.
– Stephen Kitt
Jan 21 at 16:54
1
Sorry, I got things mixed up; Debian used “madvise” from 2.6.38 to 4.13, where it switched to “always”. Thus Debian 7 through 9 use “madvise”, while Debian 10 will use “always”.
– Stephen Kitt
Jan 21 at 17:04
add a comment |
I am puzzled by LWN again.
Huge pages, slow drives, and long delays -- LWN.net, 2011
It is a rare event, but it is no fun when it strikes. Plug in a slow storage device - a USB stick or a music player, for example - and run something like rsync to move a lot of data to that device. The operation takes a while, which is unsurprising; more surprising is when random processes begin to stall...
A process (that web browser, say) is doing its job when it incurs a page fault ... If the transparent huge pages feature is built into the kernel (and most distributors do enable this feature), the page fault handler will attempt to allocate a huge page.
I can see that on Red Hat Enterprise Linux, transparent hugepages are enabled as described by LWN.
How to use, monitor, and disable transparent hugepages in Red Hat Enterprise Linux 6 and 7?
Transparent Huge Pages (THP) are enabled by default in RHEL 6 for all applications...
For RHEL 7 see How to disable transparent hugepages (THP) on Red Hat Enterprise Linux 7.
For RHEL 8 see How to disable transparent hugepages (THP) on Red Hat Enterprise Linux 8. [Links to content for subscribers only].
However, I looked on my Fedora Workstation 29 laptop, and I see transparent hugepages are not enabled "for all applications".
$ cat /sys/kernel/mm/transparent_hugepage/enabled
always [madvise] never
$ uname -r
4.19.13-300.fc29.x86_64
I also checked in some VMs: I see the same on Ubuntu Desktop 18.04 (kernel 4.15.0-43-generic), and on a Debian 9 Desktop install (kernel 4.9.0-8-amd64). (And I don't believe Debian has a "Server" install, spin, or package. It just has a few different "tasks" to install the most common server softwares. So I don't think there is any one place in Debian that could be enabling a server-specific configuration).
- Does Fedora Server 29 enable transparent hugepages for all apps?
- Does Ubuntu Server 18.04 enable transparent hugepages for all apps?
- Did a different version number of one of these distributions enable transparent hugepages for all apps, e.g. around 2011?
[By implication: was LWN.net correct here, or not? And if it was correct to say that most distributors enabled transparent hugepages, and imply that it affected some common "desktop" (or laptop) setups, do we know why current versions of Fedora Workstation, Debian, and Ubuntu Desktop are not enabling transparent hugepages for all?]
linux linux-kernel huge-pages
asked Jan 21 at 16:24
sourcejedisourcejedi
23.9k439106
I am puzzled by LWN again.
Huge pages, slow drives, and long delays -- LWN.net, 2011
It is a rare event, but it is no fun when it strikes. Plug in a slow storage device - a USB stick or a music player, for example - and run something like rsync to move a lot of data to that device. The operation takes a while, which is unsurprising; more surprising is when random processes begin to stall...
A process (that web browser, say) is doing its job when it incurs a page fault ... If the transparent huge pages feature is built into the kernel (and most distributors do enable this feature), the page fault handler will attempt to allocate a huge page.
I can see that on Red Hat Enterprise Linux, transparent hugepages are enabled as described by LWN.
How to use, monitor, and disable transparent hugepages in Red Hat Enterprise Linux 6 and 7?
Transparent Huge Pages (THP) are enabled by default in RHEL 6 for all applications...
For RHEL 7 see How to disable transparent hugepages (THP) on Red Hat Enterprise Linux 7.
For RHEL 8 see How to disable transparent hugepages (THP) on Red Hat Enterprise Linux 8. [Links to content for subscribers only].
However, I looked on my Fedora Workstation 29 laptop, and I see transparent hugepages are not enabled "for all applications".
$ cat /sys/kernel/mm/transparent_hugepage/enabled
always [madvise] never
$ uname -r
4.19.13-300.fc29.x86_64
I also checked in some VMs: I see the same on Ubuntu Desktop 18.04 (kernel 4.15.0-43-generic), and on a Debian 9 Desktop install (kernel 4.9.0-8-amd64). (And I don't believe Debian has a "Server" install, spin, or package. It just has a few different "tasks" to install the most common server softwares. So I don't think there is any one place in Debian that could be enabling a server-specific configuration).
- Does Fedora Server 29 enable transparent hugepages for all apps?
- Does Ubuntu Server 18.04 enable transparent hugepages for all apps?
- Did a different version number of one of these distributions enable transparent hugepages for all apps, e.g. around 2011?
[By implication: was LWN.net correct here, or not? And if it was correct to say that most distributors enabled transparent hugepages, and imply that it affected some common "desktop" (or laptop) setups, do we know why current versions of Fedora Workstation, Debian, and Ubuntu Desktop are not enabling transparent hugepages for all?]
linux linux-kernel huge-pages
linux linux-kernel huge-pages
asked Jan 21 at 16:24
sourcejedisourcejedi
23.9k439106
asked Jan 21 at 16:24
sourcejedisourcejedi
23.9k439106
edited Jan 21 at 17:09
sourcejedi
asked Jan 21 at 16:24
sourcejedisourcejedi
23.9k439106
asked Jan 21 at 16:24
sourcejedisourcejedi
23.9k439106
asked Jan 21 at 16:24
sourcejedisourcejedi
23.9k439106
23.9k439106
madviseis still enabled. But it's not "system wide". See kernel.org/doc/Documentation/vm/transhuge.txt FYI, I just checked my ubuntu server on AWS and it is also set tomadvise.
– couling
Jan 21 at 16:43
@couling right, presumably an app can ask for transparent hugepages, using the madvise() call if it is enabled. But I don't have any hugepages. So my browser clearly isn't asking for them. Contra the LWN example, a page fault in my browser will not try to allocate a hugepage. So I am trying to guess what made LWN think this was a good example to illustrate the problem...
– sourcejedi
Jan 21 at 16:47
@couling Thanks for pointing out the ambiguity in the title I used! Particularly since it looks like this might be what LWN was confused about. I have edited my question to use the specific phrasing from the RHEL doc.
– sourcejedi
Jan 21 at 16:54
1
Debian had THP enabled by default from kernel 2.6.38 to 4.13, so the kernels for Debian 7 to 9 had it set to “always”. Debian 10 will have it set to “madvise”. Debian 6 was current in 2011, with kernel 2.6.32, and didn’t have transparent hugepages, it had hugetlbfs.
– Stephen Kitt
Jan 21 at 16:54
1
Sorry, I got things mixed up; Debian used “madvise” from 2.6.38 to 4.13, where it switched to “always”. Thus Debian 7 through 9 use “madvise”, while Debian 10 will use “always”.
– Stephen Kitt
Jan 21 at 17:04
add a comment |
madviseis still enabled. But it's not "system wide". See kernel.org/doc/Documentation/vm/transhuge.txt FYI, I just checked my ubuntu server on AWS and it is also set tomadvise.
– couling
Jan 21 at 16:43
@couling right, presumably an app can ask for transparent hugepages, using the madvise() call if it is enabled. But I don't have any hugepages. So my browser clearly isn't asking for them. Contra the LWN example, a page fault in my browser will not try to allocate a hugepage. So I am trying to guess what made LWN think this was a good example to illustrate the problem...
– sourcejedi
Jan 21 at 16:47
@couling Thanks for pointing out the ambiguity in the title I used! Particularly since it looks like this might be what LWN was confused about. I have edited my question to use the specific phrasing from the RHEL doc.
– sourcejedi
Jan 21 at 16:54
1
Debian had THP enabled by default from kernel 2.6.38 to 4.13, so the kernels for Debian 7 to 9 had it set to “always”. Debian 10 will have it set to “madvise”. Debian 6 was current in 2011, with kernel 2.6.32, and didn’t have transparent hugepages, it had hugetlbfs.
– Stephen Kitt
Jan 21 at 16:54
1
Sorry, I got things mixed up; Debian used “madvise” from 2.6.38 to 4.13, where it switched to “always”. Thus Debian 7 through 9 use “madvise”, while Debian 10 will use “always”.
– Stephen Kitt
Jan 21 at 17:04
madvise is still enabled. But it's not "system wide". See kernel.org/doc/Documentation/vm/transhuge.txt FYI, I just checked my ubuntu server on AWS and it is also set to madvise.– couling
Jan 21 at 16:43
madvise is still enabled. But it's not "system wide". See kernel.org/doc/Documentation/vm/transhuge.txt FYI, I just checked my ubuntu server on AWS and it is also set to madvise.– couling
Jan 21 at 16:43
@couling right, presumably an app can ask for transparent hugepages, using the madvise() call if it is enabled. But I don't have any hugepages. So my browser clearly isn't asking for them. Contra the LWN example, a page fault in my browser will not try to allocate a hugepage. So I am trying to guess what made LWN think this was a good example to illustrate the problem...
– sourcejedi
Jan 21 at 16:47
@couling right, presumably an app can ask for transparent hugepages, using the madvise() call if it is enabled. But I don't have any hugepages. So my browser clearly isn't asking for them. Contra the LWN example, a page fault in my browser will not try to allocate a hugepage. So I am trying to guess what made LWN think this was a good example to illustrate the problem...
– sourcejedi
Jan 21 at 16:47
@couling Thanks for pointing out the ambiguity in the title I used! Particularly since it looks like this might be what LWN was confused about. I have edited my question to use the specific phrasing from the RHEL doc.
– sourcejedi
Jan 21 at 16:54
@couling Thanks for pointing out the ambiguity in the title I used! Particularly since it looks like this might be what LWN was confused about. I have edited my question to use the specific phrasing from the RHEL doc.
– sourcejedi
Jan 21 at 16:54
1
1
Debian had THP enabled by default from kernel 2.6.38 to 4.13, so the kernels for Debian 7 to 9 had it set to “always”. Debian 10 will have it set to “madvise”. Debian 6 was current in 2011, with kernel 2.6.32, and didn’t have transparent hugepages, it had hugetlbfs.
– Stephen Kitt
Jan 21 at 16:54
Debian had THP enabled by default from kernel 2.6.38 to 4.13, so the kernels for Debian 7 to 9 had it set to “always”. Debian 10 will have it set to “madvise”. Debian 6 was current in 2011, with kernel 2.6.32, and didn’t have transparent hugepages, it had hugetlbfs.
– Stephen Kitt
Jan 21 at 16:54
1
1
Sorry, I got things mixed up; Debian used “madvise” from 2.6.38 to 4.13, where it switched to “always”. Thus Debian 7 through 9 use “madvise”, while Debian 10 will use “always”.
– Stephen Kitt
Jan 21 at 17:04
Sorry, I got things mixed up; Debian used “madvise” from 2.6.38 to 4.13, where it switched to “always”. Thus Debian 7 through 9 use “madvise”, while Debian 10 will use “always”.
– Stephen Kitt
Jan 21 at 17:04
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f495816%2fwhich-distributions-enable-transparent-huge-pages-for-all-applications%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f495816%2fwhich-distributions-enable-transparent-huge-pages-for-all-applications%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
madviseis still enabled. But it's not "system wide". See kernel.org/doc/Documentation/vm/transhuge.txt FYI, I just checked my ubuntu server on AWS and it is also set tomadvise.– couling
Jan 21 at 16:43
@couling right, presumably an app can ask for transparent hugepages, using the madvise() call if it is enabled. But I don't have any hugepages. So my browser clearly isn't asking for them. Contra the LWN example, a page fault in my browser will not try to allocate a hugepage. So I am trying to guess what made LWN think this was a good example to illustrate the problem...
– sourcejedi
Jan 21 at 16:47
@couling Thanks for pointing out the ambiguity in the title I used! Particularly since it looks like this might be what LWN was confused about. I have edited my question to use the specific phrasing from the RHEL doc.
– sourcejedi
Jan 21 at 16:54
1
Debian had THP enabled by default from kernel 2.6.38 to 4.13, so the kernels for Debian 7 to 9 had it set to “always”. Debian 10 will have it set to “madvise”. Debian 6 was current in 2011, with kernel 2.6.32, and didn’t have transparent hugepages, it had hugetlbfs.
– Stephen Kitt
Jan 21 at 16:54
1
Sorry, I got things mixed up; Debian used “madvise” from 2.6.38 to 4.13, where it switched to “always”. Thus Debian 7 through 9 use “madvise”, while Debian 10 will use “always”.
– Stephen Kitt
Jan 21 at 17:04