Why don't reads from /dev/zero count as IO_RBYTES?
I am emptying out a hard drive on some Linux 4.x OS using this command:
sudo sh -c 'pv -pterb /dev/zero > /dev/sda'
And I opened another tty and started sudo htop and noticed this:
PID USER PRI NI CPU% RES SHR IO_RBYTES IO_WBYTES S TIME+ Command
4598 root 20 0 15.5 1820 1596 4096 17223823 D 1:14.11 pv -pterb /dev/zero
The value for IO_WBYTES seems quite normal, but IO_RBYTES remains at 4 KiB and never changes.
I ran a few other programs, for example
dd if=/dev/zero of=/dev/zero
cat /dev/zero > /dev/zero
and was surprised to see none of them generates a lot of IO_RBYTES or IO_WBYTES.
I think this is not specific to any program, but why don't reads from /dev/zero and writes to /dev/{zero,null} count as I/O bytes?
linux devices io null
asked Feb 20 at 10:38
iBugiBug
956928
|
show 1 more comment
I am emptying out a hard drive on some Linux 4.x OS using this command:
sudo sh -c 'pv -pterb /dev/zero > /dev/sda'
And I opened another tty and started sudo htop and noticed this:
PID USER PRI NI CPU% RES SHR IO_RBYTES IO_WBYTES S TIME+ Command
4598 root 20 0 15.5 1820 1596 4096 17223823 D 1:14.11 pv -pterb /dev/zero
The value for IO_WBYTES seems quite normal, but IO_RBYTES remains at 4 KiB and never changes.
I ran a few other programs, for example
dd if=/dev/zero of=/dev/zero
cat /dev/zero > /dev/zero
and was surprised to see none of them generates a lot of IO_RBYTES or IO_WBYTES.
I think this is not specific to any program, but why don't reads from /dev/zero and writes to /dev/{zero,null} count as I/O bytes?
linux devices io null
asked Feb 20 at 10:38
iBugiBug
956928
5
I'm curious, why do you think they should count as I/O?
– marcelm
Feb 20 at 19:58
1
@marcelm I think any input/output should count as I/O, including file R/W, network I/O and many more.
– iBug
Feb 21 at 0:56
but those operations perform I/O to hardware (disk, and the network card, respectively), and have to travel over some I/O bus (such as PCI-express), all of which can be a significant bottleneck. Writes to, say,/dev/nulldon't end up interfacing wich such hardware and don't clog I/O buses. Taken to the extreme; are reads/writes to/from memory also I/O? Of course, there is no hard delineation for these things, and it all depends on which perspective you take in these things, and how useful that perspective ends up being to you.
– marcelm
Feb 21 at 11:39
1
Note, my first comment was intended to provoke you (and others) to think about those perspectives, and find out why you're taking your perspective. I don't mean to insinuate you're wrong; I don't even think the situation is that black and white. But personally, I would be a lot more interested in I/O statistics to actual hardware (which may very well be a bottleneck) than to/dev/{null,zero}(which usually isn't a bottleneck). That's just my perspective though :)
– marcelm
Feb 21 at 11:44
1
@marcelm But I was initially thinking that anyread(2)andwrite(2)counts as I/O, which is very reasonable in its own sense.
– iBug
Feb 21 at 11:51
|
show 1 more comment
I am emptying out a hard drive on some Linux 4.x OS using this command:
sudo sh -c 'pv -pterb /dev/zero > /dev/sda'
And I opened another tty and started sudo htop and noticed this:
PID USER PRI NI CPU% RES SHR IO_RBYTES IO_WBYTES S TIME+ Command
4598 root 20 0 15.5 1820 1596 4096 17223823 D 1:14.11 pv -pterb /dev/zero
The value for IO_WBYTES seems quite normal, but IO_RBYTES remains at 4 KiB and never changes.
I ran a few other programs, for example
dd if=/dev/zero of=/dev/zero
cat /dev/zero > /dev/zero
and was surprised to see none of them generates a lot of IO_RBYTES or IO_WBYTES.
I think this is not specific to any program, but why don't reads from /dev/zero and writes to /dev/{zero,null} count as I/O bytes?
linux devices io null
asked Feb 20 at 10:38
iBugiBug
956928
I am emptying out a hard drive on some Linux 4.x OS using this command:
sudo sh -c 'pv -pterb /dev/zero > /dev/sda'
And I opened another tty and started sudo htop and noticed this:
PID USER PRI NI CPU% RES SHR IO_RBYTES IO_WBYTES S TIME+ Command
4598 root 20 0 15.5 1820 1596 4096 17223823 D 1:14.11 pv -pterb /dev/zero
The value for IO_WBYTES seems quite normal, but IO_RBYTES remains at 4 KiB and never changes.
I ran a few other programs, for example
dd if=/dev/zero of=/dev/zero
cat /dev/zero > /dev/zero
and was surprised to see none of them generates a lot of IO_RBYTES or IO_WBYTES.
I think this is not specific to any program, but why don't reads from /dev/zero and writes to /dev/{zero,null} count as I/O bytes?
linux devices io null
linux devices io null
asked Feb 20 at 10:38
iBugiBug
956928
asked Feb 20 at 10:38
iBugiBug
956928
edited 15 hours ago
iBug
asked Feb 20 at 10:38
iBugiBug
956928
asked Feb 20 at 10:38
iBugiBug
956928
asked Feb 20 at 10:38
iBugiBug
956928
956928
5
I'm curious, why do you think they should count as I/O?
– marcelm
Feb 20 at 19:58
1
@marcelm I think any input/output should count as I/O, including file R/W, network I/O and many more.
– iBug
Feb 21 at 0:56
but those operations perform I/O to hardware (disk, and the network card, respectively), and have to travel over some I/O bus (such as PCI-express), all of which can be a significant bottleneck. Writes to, say,/dev/nulldon't end up interfacing wich such hardware and don't clog I/O buses. Taken to the extreme; are reads/writes to/from memory also I/O? Of course, there is no hard delineation for these things, and it all depends on which perspective you take in these things, and how useful that perspective ends up being to you.
– marcelm
Feb 21 at 11:39
1
Note, my first comment was intended to provoke you (and others) to think about those perspectives, and find out why you're taking your perspective. I don't mean to insinuate you're wrong; I don't even think the situation is that black and white. But personally, I would be a lot more interested in I/O statistics to actual hardware (which may very well be a bottleneck) than to/dev/{null,zero}(which usually isn't a bottleneck). That's just my perspective though :)
– marcelm
Feb 21 at 11:44
1
@marcelm But I was initially thinking that anyread(2)andwrite(2)counts as I/O, which is very reasonable in its own sense.
– iBug
Feb 21 at 11:51
|
show 1 more comment
5
I'm curious, why do you think they should count as I/O?
– marcelm
Feb 20 at 19:58
1
@marcelm I think any input/output should count as I/O, including file R/W, network I/O and many more.
– iBug
Feb 21 at 0:56
but those operations perform I/O to hardware (disk, and the network card, respectively), and have to travel over some I/O bus (such as PCI-express), all of which can be a significant bottleneck. Writes to, say,/dev/nulldon't end up interfacing wich such hardware and don't clog I/O buses. Taken to the extreme; are reads/writes to/from memory also I/O? Of course, there is no hard delineation for these things, and it all depends on which perspective you take in these things, and how useful that perspective ends up being to you.
– marcelm
Feb 21 at 11:39
1
Note, my first comment was intended to provoke you (and others) to think about those perspectives, and find out why you're taking your perspective. I don't mean to insinuate you're wrong; I don't even think the situation is that black and white. But personally, I would be a lot more interested in I/O statistics to actual hardware (which may very well be a bottleneck) than to/dev/{null,zero}(which usually isn't a bottleneck). That's just my perspective though :)
– marcelm
Feb 21 at 11:44
1
@marcelm But I was initially thinking that anyread(2)andwrite(2)counts as I/O, which is very reasonable in its own sense.
– iBug
Feb 21 at 11:51
5
5
I'm curious, why do you think they should count as I/O?
– marcelm
Feb 20 at 19:58
I'm curious, why do you think they should count as I/O?
– marcelm
Feb 20 at 19:58
1
1
@marcelm I think any input/output should count as I/O, including file R/W, network I/O and many more.
– iBug
Feb 21 at 0:56
@marcelm I think any input/output should count as I/O, including file R/W, network I/O and many more.
– iBug
Feb 21 at 0:56
but those operations perform I/O to hardware (disk, and the network card, respectively), and have to travel over some I/O bus (such as PCI-express), all of which can be a significant bottleneck. Writes to, say,
/dev/null don't end up interfacing wich such hardware and don't clog I/O buses. Taken to the extreme; are reads/writes to/from memory also I/O? Of course, there is no hard delineation for these things, and it all depends on which perspective you take in these things, and how useful that perspective ends up being to you.– marcelm
Feb 21 at 11:39
but those operations perform I/O to hardware (disk, and the network card, respectively), and have to travel over some I/O bus (such as PCI-express), all of which can be a significant bottleneck. Writes to, say,
/dev/null don't end up interfacing wich such hardware and don't clog I/O buses. Taken to the extreme; are reads/writes to/from memory also I/O? Of course, there is no hard delineation for these things, and it all depends on which perspective you take in these things, and how useful that perspective ends up being to you.– marcelm
Feb 21 at 11:39
1
1
Note, my first comment was intended to provoke you (and others) to think about those perspectives, and find out why you're taking your perspective. I don't mean to insinuate you're wrong; I don't even think the situation is that black and white. But personally, I would be a lot more interested in I/O statistics to actual hardware (which may very well be a bottleneck) than to
/dev/{null,zero} (which usually isn't a bottleneck). That's just my perspective though :)– marcelm
Feb 21 at 11:44
Note, my first comment was intended to provoke you (and others) to think about those perspectives, and find out why you're taking your perspective. I don't mean to insinuate you're wrong; I don't even think the situation is that black and white. But personally, I would be a lot more interested in I/O statistics to actual hardware (which may very well be a bottleneck) than to
/dev/{null,zero} (which usually isn't a bottleneck). That's just my perspective though :)– marcelm
Feb 21 at 11:44
1
1
@marcelm But I was initially thinking that any
read(2) and write(2) counts as I/O, which is very reasonable in its own sense.– iBug
Feb 21 at 11:51
@marcelm But I was initially thinking that any
read(2) and write(2) counts as I/O, which is very reasonable in its own sense.– iBug
Feb 21 at 11:51
|
show 1 more comment
1 Answer
1
active
oldest
votes
They do count as I/O, but not of the type measured by the fields you’re looking at.
In htop, IO_RBYTES and IO_WBYTES show the read_bytes and write_bytes fields from /proc/<pid>/io, and those fields measure bytes which go through the block layer. /dev/zero doesn’t involve the block layer, so reads from it don’t show up there.
To see I/O from /dev/zero, you need to look at the rchar and wchar fields in /proc/<pid>/io, which show up in htop as RCHAR and WCHAR:
rchar: characters read
The number of bytes which this task has caused to be
read from storage. This is simply the sum of bytes
which this process passed toread(2)and similar system
calls. It includes things such as terminal I/O and is
unaffected by whether or not actual physical disk I/O
was required (the read might have been satisfied from
pagecache).
wchar: characters written
The number of bytes which this task has caused, or
shall cause to be written to disk. Similar caveats
apply here as with rchar.
See man 5 proc and man 1 htop for details.
answered Feb 20 at 10:44
Stephen KittStephen Kitt
175k24400477
So it'srcharandwcharthat count bytes from calls toread(2)andwrite(2), right?
– iBug
Feb 20 at 11:00
Yes, that’s right.
– Stephen Kitt
Feb 20 at 11:06
9
Talk about misleading phrasing on the description of rchar. Everything passed throughread()most definitely isn't "read from storage"!
– ilkkachu
Feb 20 at 11:32
2
@ilkkachu bystoragethey mean "any conceivable bus line", regardless of whether the storage in question is physical or virtual or mmap'd or a virtual socket or in L1 cache -- it's just anything outside of that program's mapped memory including shared
– cat
Feb 20 at 23:53
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f501814%2fwhy-dont-reads-from-dev-zero-count-as-io-rbytes%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
They do count as I/O, but not of the type measured by the fields you’re looking at.
In htop, IO_RBYTES and IO_WBYTES show the read_bytes and write_bytes fields from /proc/<pid>/io, and those fields measure bytes which go through the block layer. /dev/zero doesn’t involve the block layer, so reads from it don’t show up there.
To see I/O from /dev/zero, you need to look at the rchar and wchar fields in /proc/<pid>/io, which show up in htop as RCHAR and WCHAR:
rchar: characters read
The number of bytes which this task has caused to be
read from storage. This is simply the sum of bytes
which this process passed toread(2)and similar system
calls. It includes things such as terminal I/O and is
unaffected by whether or not actual physical disk I/O
was required (the read might have been satisfied from
pagecache).
wchar: characters written
The number of bytes which this task has caused, or
shall cause to be written to disk. Similar caveats
apply here as with rchar.
See man 5 proc and man 1 htop for details.
answered Feb 20 at 10:44
Stephen KittStephen Kitt
175k24400477
So it'srcharandwcharthat count bytes from calls toread(2)andwrite(2), right?
– iBug
Feb 20 at 11:00
Yes, that’s right.
– Stephen Kitt
Feb 20 at 11:06
9
Talk about misleading phrasing on the description of rchar. Everything passed throughread()most definitely isn't "read from storage"!
– ilkkachu
Feb 20 at 11:32
2
@ilkkachu bystoragethey mean "any conceivable bus line", regardless of whether the storage in question is physical or virtual or mmap'd or a virtual socket or in L1 cache -- it's just anything outside of that program's mapped memory including shared
– cat
Feb 20 at 23:53
add a comment |
They do count as I/O, but not of the type measured by the fields you’re looking at.
In htop, IO_RBYTES and IO_WBYTES show the read_bytes and write_bytes fields from /proc/<pid>/io, and those fields measure bytes which go through the block layer. /dev/zero doesn’t involve the block layer, so reads from it don’t show up there.
To see I/O from /dev/zero, you need to look at the rchar and wchar fields in /proc/<pid>/io, which show up in htop as RCHAR and WCHAR:
rchar: characters read
The number of bytes which this task has caused to be
read from storage. This is simply the sum of bytes
which this process passed toread(2)and similar system
calls. It includes things such as terminal I/O and is
unaffected by whether or not actual physical disk I/O
was required (the read might have been satisfied from
pagecache).
wchar: characters written
The number of bytes which this task has caused, or
shall cause to be written to disk. Similar caveats
apply here as with rchar.
See man 5 proc and man 1 htop for details.
answered Feb 20 at 10:44
Stephen KittStephen Kitt
175k24400477
So it'srcharandwcharthat count bytes from calls toread(2)andwrite(2), right?
– iBug
Feb 20 at 11:00
Yes, that’s right.
– Stephen Kitt
Feb 20 at 11:06
9
Talk about misleading phrasing on the description of rchar. Everything passed throughread()most definitely isn't "read from storage"!
– ilkkachu
Feb 20 at 11:32
2
@ilkkachu bystoragethey mean "any conceivable bus line", regardless of whether the storage in question is physical or virtual or mmap'd or a virtual socket or in L1 cache -- it's just anything outside of that program's mapped memory including shared
– cat
Feb 20 at 23:53
add a comment |
They do count as I/O, but not of the type measured by the fields you’re looking at.
In htop, IO_RBYTES and IO_WBYTES show the read_bytes and write_bytes fields from /proc/<pid>/io, and those fields measure bytes which go through the block layer. /dev/zero doesn’t involve the block layer, so reads from it don’t show up there.
To see I/O from /dev/zero, you need to look at the rchar and wchar fields in /proc/<pid>/io, which show up in htop as RCHAR and WCHAR:
rchar: characters read
The number of bytes which this task has caused to be
read from storage. This is simply the sum of bytes
which this process passed toread(2)and similar system
calls. It includes things such as terminal I/O and is
unaffected by whether or not actual physical disk I/O
was required (the read might have been satisfied from
pagecache).
wchar: characters written
The number of bytes which this task has caused, or
shall cause to be written to disk. Similar caveats
apply here as with rchar.
See man 5 proc and man 1 htop for details.
answered Feb 20 at 10:44
Stephen KittStephen Kitt
175k24400477
They do count as I/O, but not of the type measured by the fields you’re looking at.
In htop, IO_RBYTES and IO_WBYTES show the read_bytes and write_bytes fields from /proc/<pid>/io, and those fields measure bytes which go through the block layer. /dev/zero doesn’t involve the block layer, so reads from it don’t show up there.
To see I/O from /dev/zero, you need to look at the rchar and wchar fields in /proc/<pid>/io, which show up in htop as RCHAR and WCHAR:
rchar: characters read
The number of bytes which this task has caused to be
read from storage. This is simply the sum of bytes
which this process passed toread(2)and similar system
calls. It includes things such as terminal I/O and is
unaffected by whether or not actual physical disk I/O
was required (the read might have been satisfied from
pagecache).
wchar: characters written
The number of bytes which this task has caused, or
shall cause to be written to disk. Similar caveats
apply here as with rchar.
See man 5 proc and man 1 htop for details.
answered Feb 20 at 10:44
Stephen KittStephen Kitt
175k24400477
edited Feb 20 at 11:06
answered Feb 20 at 10:44
Stephen KittStephen Kitt
175k24400477
answered Feb 20 at 10:44
Stephen KittStephen Kitt
175k24400477
answered Feb 20 at 10:44
Stephen KittStephen Kitt
175k24400477
175k24400477
So it'srcharandwcharthat count bytes from calls toread(2)andwrite(2), right?
– iBug
Feb 20 at 11:00
Yes, that’s right.
– Stephen Kitt
Feb 20 at 11:06
9
Talk about misleading phrasing on the description of rchar. Everything passed throughread()most definitely isn't "read from storage"!
– ilkkachu
Feb 20 at 11:32
2
@ilkkachu bystoragethey mean "any conceivable bus line", regardless of whether the storage in question is physical or virtual or mmap'd or a virtual socket or in L1 cache -- it's just anything outside of that program's mapped memory including shared
– cat
Feb 20 at 23:53
add a comment |
So it'srcharandwcharthat count bytes from calls toread(2)andwrite(2), right?
– iBug
Feb 20 at 11:00
Yes, that’s right.
– Stephen Kitt
Feb 20 at 11:06
9
Talk about misleading phrasing on the description of rchar. Everything passed throughread()most definitely isn't "read from storage"!
– ilkkachu
Feb 20 at 11:32
2
@ilkkachu bystoragethey mean "any conceivable bus line", regardless of whether the storage in question is physical or virtual or mmap'd or a virtual socket or in L1 cache -- it's just anything outside of that program's mapped memory including shared
– cat
Feb 20 at 23:53
So it's
rchar and wchar that count bytes from calls to read(2) and write(2), right?– iBug
Feb 20 at 11:00
So it's
rchar and wchar that count bytes from calls to read(2) and write(2), right?– iBug
Feb 20 at 11:00
Yes, that’s right.
– Stephen Kitt
Feb 20 at 11:06
Yes, that’s right.
– Stephen Kitt
Feb 20 at 11:06
9
9
Talk about misleading phrasing on the description of rchar. Everything passed through
read() most definitely isn't "read from storage"!– ilkkachu
Feb 20 at 11:32
Talk about misleading phrasing on the description of rchar. Everything passed through
read() most definitely isn't "read from storage"!– ilkkachu
Feb 20 at 11:32
2
2
@ilkkachu by
storage they mean "any conceivable bus line", regardless of whether the storage in question is physical or virtual or mmap'd or a virtual socket or in L1 cache -- it's just anything outside of that program's mapped memory including shared– cat
Feb 20 at 23:53
@ilkkachu by
storage they mean "any conceivable bus line", regardless of whether the storage in question is physical or virtual or mmap'd or a virtual socket or in L1 cache -- it's just anything outside of that program's mapped memory including shared– cat
Feb 20 at 23:53
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f501814%2fwhy-dont-reads-from-dev-zero-count-as-io-rbytes%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
5
I'm curious, why do you think they should count as I/O?
– marcelm
Feb 20 at 19:58
1
@marcelm I think any input/output should count as I/O, including file R/W, network I/O and many more.
– iBug
Feb 21 at 0:56
but those operations perform I/O to hardware (disk, and the network card, respectively), and have to travel over some I/O bus (such as PCI-express), all of which can be a significant bottleneck. Writes to, say,
/dev/nulldon't end up interfacing wich such hardware and don't clog I/O buses. Taken to the extreme; are reads/writes to/from memory also I/O? Of course, there is no hard delineation for these things, and it all depends on which perspective you take in these things, and how useful that perspective ends up being to you.– marcelm
Feb 21 at 11:39
1
Note, my first comment was intended to provoke you (and others) to think about those perspectives, and find out why you're taking your perspective. I don't mean to insinuate you're wrong; I don't even think the situation is that black and white. But personally, I would be a lot more interested in I/O statistics to actual hardware (which may very well be a bottleneck) than to
/dev/{null,zero}(which usually isn't a bottleneck). That's just my perspective though :)– marcelm
Feb 21 at 11:44
1
@marcelm But I was initially thinking that any
read(2)andwrite(2)counts as I/O, which is very reasonable in its own sense.– iBug
Feb 21 at 11:51