How does 200 million function call of fgets translate into only 25,224 system calls?
A I/O efficiency test from APUE:
Test file is said to be "98.5 MB with 3 million lines."
Code used in "Figure 3.6":
#include "apue.h"
#define BUFFSIZE 4096
int
main(void)
{
int n;
char buf[BUFFSIZE];
while ((n = read(STDIN_FILENO, buf, BUFFSIZE)) > 0)
if (write(STDOUT_FILENO, buf, n) != n)
err_sys("write error");
if (n < 0)
err_sys("read error");
exit(0);
}
best time from Figure 3.6: Use different BUFFSIZE and choose the best time
single byte time from Figure 3.6: Use BUFFSIZE=1
err_sys is merely a wrapper function for error handling.
apue.h is a also wrapper header file.
Code uses fgets:
#include "apue.h"
int
main(void)
{
int c;
while ((c = fgetc(stdin)) != EOF){
if (fputc(c, stdout) == EOF)
err_sys("output error");
}
if (ferror(stdin))
err_sys("input error");
exit(0);
}
The question comes from:
The last point of interest with these timing numbers is that the
fgetcversion is so much faster than theBUFFSIZE=1version from Figure 3.6. Both involve the same number of function calls—about 200 million—yet thefgetcversion is more than 16 times faster in terms of user CPU time and almost 39 times faster in terms of clock time.
The difference is that the version using read executes 200 million function calls, which in turn execute 200 million system calls. With thefgetcversion, we still execute 200 million function calls, but this translates into only 25,224 system calls. System calls are
usually much more expensive than ordinary function calls.
The test file is 98.5 MB ≈ 100 M, so 100 million calls for each I/O function call == 200 million calls, no problem here.
But how does 200 million fgets translate into 25,224 system call?
With the
fgetcversion, we still execute 200 million function calls, but this translates into only 25,224 system calls.
How is 25,224 calculated?
files io
edited Mar 5 at 5:56
Olorin
3,9481723
asked Mar 5 at 5:42
RickRick
262310
|
show 1 more comment
A I/O efficiency test from APUE:
Test file is said to be "98.5 MB with 3 million lines."
Code used in "Figure 3.6":
#include "apue.h"
#define BUFFSIZE 4096
int
main(void)
{
int n;
char buf[BUFFSIZE];
while ((n = read(STDIN_FILENO, buf, BUFFSIZE)) > 0)
if (write(STDOUT_FILENO, buf, n) != n)
err_sys("write error");
if (n < 0)
err_sys("read error");
exit(0);
}
best time from Figure 3.6: Use different BUFFSIZE and choose the best time
single byte time from Figure 3.6: Use BUFFSIZE=1
err_sys is merely a wrapper function for error handling.
apue.h is a also wrapper header file.
Code uses fgets:
#include "apue.h"
int
main(void)
{
int c;
while ((c = fgetc(stdin)) != EOF){
if (fputc(c, stdout) == EOF)
err_sys("output error");
}
if (ferror(stdin))
err_sys("input error");
exit(0);
}
The question comes from:
The last point of interest with these timing numbers is that the
fgetcversion is so much faster than theBUFFSIZE=1version from Figure 3.6. Both involve the same number of function calls—about 200 million—yet thefgetcversion is more than 16 times faster in terms of user CPU time and almost 39 times faster in terms of clock time.
The difference is that the version using read executes 200 million function calls, which in turn execute 200 million system calls. With thefgetcversion, we still execute 200 million function calls, but this translates into only 25,224 system calls. System calls are
usually much more expensive than ordinary function calls.
The test file is 98.5 MB ≈ 100 M, so 100 million calls for each I/O function call == 200 million calls, no problem here.
But how does 200 million fgets translate into 25,224 system call?
With the
fgetcversion, we still execute 200 million function calls, but this translates into only 25,224 system calls.
How is 25,224 calculated?
files io
edited Mar 5 at 5:56
Olorin
3,9481723
asked Mar 5 at 5:42
RickRick
262310
3
Presumablyfgetcandfputcdo internal buffering of ~4k, dividing the whole count by 8k (98.5*1024*1024/4096 = 25216, which is remarkably close to the given number).
– Olorin
Mar 5 at 6:01
1
If so, I think internal buffering should be ~8K. Read function call == write function call = 25224 / 2 = 12612. 98.5*1024*1024/12612 = 8189 B. ~8K data transmitted for each read or write operation. Am I right here?
– Rick
Mar 5 at 6:37
1
Ah, yes, I forgot to double the 98.5 before.
– Olorin
Mar 5 at 7:21
1
You can set the stdio buffer size to whatever you want withsetvbuf()(eg. to 1G), and then calculate yet another value ;-)
– mosvy
Mar 5 at 11:02
1
at the contrary, it's functions like fgetc or fputc (which may even be inlined) which need buffering the most, as the example from that book triea to demonstrate. In any case, they use full buffering by default (when not reading from some interactive device) in any implementation I may think of.
– mosvy
Mar 5 at 16:41
|
show 1 more comment
A I/O efficiency test from APUE:
Test file is said to be "98.5 MB with 3 million lines."
Code used in "Figure 3.6":
#include "apue.h"
#define BUFFSIZE 4096
int
main(void)
{
int n;
char buf[BUFFSIZE];
while ((n = read(STDIN_FILENO, buf, BUFFSIZE)) > 0)
if (write(STDOUT_FILENO, buf, n) != n)
err_sys("write error");
if (n < 0)
err_sys("read error");
exit(0);
}
best time from Figure 3.6: Use different BUFFSIZE and choose the best time
single byte time from Figure 3.6: Use BUFFSIZE=1
err_sys is merely a wrapper function for error handling.
apue.h is a also wrapper header file.
Code uses fgets:
#include "apue.h"
int
main(void)
{
int c;
while ((c = fgetc(stdin)) != EOF){
if (fputc(c, stdout) == EOF)
err_sys("output error");
}
if (ferror(stdin))
err_sys("input error");
exit(0);
}
The question comes from:
The last point of interest with these timing numbers is that the
fgetcversion is so much faster than theBUFFSIZE=1version from Figure 3.6. Both involve the same number of function calls—about 200 million—yet thefgetcversion is more than 16 times faster in terms of user CPU time and almost 39 times faster in terms of clock time.
The difference is that the version using read executes 200 million function calls, which in turn execute 200 million system calls. With thefgetcversion, we still execute 200 million function calls, but this translates into only 25,224 system calls. System calls are
usually much more expensive than ordinary function calls.
The test file is 98.5 MB ≈ 100 M, so 100 million calls for each I/O function call == 200 million calls, no problem here.
But how does 200 million fgets translate into 25,224 system call?
With the
fgetcversion, we still execute 200 million function calls, but this translates into only 25,224 system calls.
How is 25,224 calculated?
files io
edited Mar 5 at 5:56
Olorin
3,9481723
asked Mar 5 at 5:42
RickRick
262310
A I/O efficiency test from APUE:
Test file is said to be "98.5 MB with 3 million lines."
Code used in "Figure 3.6":
#include "apue.h"
#define BUFFSIZE 4096
int
main(void)
{
int n;
char buf[BUFFSIZE];
while ((n = read(STDIN_FILENO, buf, BUFFSIZE)) > 0)
if (write(STDOUT_FILENO, buf, n) != n)
err_sys("write error");
if (n < 0)
err_sys("read error");
exit(0);
}
best time from Figure 3.6: Use different BUFFSIZE and choose the best time
single byte time from Figure 3.6: Use BUFFSIZE=1
err_sys is merely a wrapper function for error handling.
apue.h is a also wrapper header file.
Code uses fgets:
#include "apue.h"
int
main(void)
{
int c;
while ((c = fgetc(stdin)) != EOF){
if (fputc(c, stdout) == EOF)
err_sys("output error");
}
if (ferror(stdin))
err_sys("input error");
exit(0);
}
The question comes from:
The last point of interest with these timing numbers is that the
fgetcversion is so much faster than theBUFFSIZE=1version from Figure 3.6. Both involve the same number of function calls—about 200 million—yet thefgetcversion is more than 16 times faster in terms of user CPU time and almost 39 times faster in terms of clock time.
The difference is that the version using read executes 200 million function calls, which in turn execute 200 million system calls. With thefgetcversion, we still execute 200 million function calls, but this translates into only 25,224 system calls. System calls are
usually much more expensive than ordinary function calls.
The test file is 98.5 MB ≈ 100 M, so 100 million calls for each I/O function call == 200 million calls, no problem here.
But how does 200 million fgets translate into 25,224 system call?
With the
fgetcversion, we still execute 200 million function calls, but this translates into only 25,224 system calls.
How is 25,224 calculated?
files io
files io
edited Mar 5 at 5:56
Olorin
3,9481723
asked Mar 5 at 5:42
RickRick
262310
edited Mar 5 at 5:56
Olorin
3,9481723
asked Mar 5 at 5:42
RickRick
262310
edited Mar 5 at 5:56
Olorin
3,9481723
edited Mar 5 at 5:56
Olorin
3,9481723
edited Mar 5 at 5:56
Olorin
3,9481723
3,9481723
asked Mar 5 at 5:42
RickRick
262310
asked Mar 5 at 5:42
RickRick
262310
asked Mar 5 at 5:42
RickRick
262310
262310
3
Presumablyfgetcandfputcdo internal buffering of ~4k, dividing the whole count by 8k (98.5*1024*1024/4096 = 25216, which is remarkably close to the given number).
– Olorin
Mar 5 at 6:01
1
If so, I think internal buffering should be ~8K. Read function call == write function call = 25224 / 2 = 12612. 98.5*1024*1024/12612 = 8189 B. ~8K data transmitted for each read or write operation. Am I right here?
– Rick
Mar 5 at 6:37
1
Ah, yes, I forgot to double the 98.5 before.
– Olorin
Mar 5 at 7:21
1
You can set the stdio buffer size to whatever you want withsetvbuf()(eg. to 1G), and then calculate yet another value ;-)
– mosvy
Mar 5 at 11:02
1
at the contrary, it's functions like fgetc or fputc (which may even be inlined) which need buffering the most, as the example from that book triea to demonstrate. In any case, they use full buffering by default (when not reading from some interactive device) in any implementation I may think of.
– mosvy
Mar 5 at 16:41
|
show 1 more comment
3
Presumablyfgetcandfputcdo internal buffering of ~4k, dividing the whole count by 8k (98.5*1024*1024/4096 = 25216, which is remarkably close to the given number).
– Olorin
Mar 5 at 6:01
1
If so, I think internal buffering should be ~8K. Read function call == write function call = 25224 / 2 = 12612. 98.5*1024*1024/12612 = 8189 B. ~8K data transmitted for each read or write operation. Am I right here?
– Rick
Mar 5 at 6:37
1
Ah, yes, I forgot to double the 98.5 before.
– Olorin
Mar 5 at 7:21
1
You can set the stdio buffer size to whatever you want withsetvbuf()(eg. to 1G), and then calculate yet another value ;-)
– mosvy
Mar 5 at 11:02
1
at the contrary, it's functions like fgetc or fputc (which may even be inlined) which need buffering the most, as the example from that book triea to demonstrate. In any case, they use full buffering by default (when not reading from some interactive device) in any implementation I may think of.
– mosvy
Mar 5 at 16:41
3
3
Presumably
fgetc and fputc do internal buffering of ~4k, dividing the whole count by 8k (98.5*1024*1024/4096 = 25216, which is remarkably close to the given number).– Olorin
Mar 5 at 6:01
Presumably
fgetc and fputc do internal buffering of ~4k, dividing the whole count by 8k (98.5*1024*1024/4096 = 25216, which is remarkably close to the given number).– Olorin
Mar 5 at 6:01
1
1
If so, I think internal buffering should be ~8K. Read function call == write function call = 25224 / 2 = 12612. 98.5*1024*1024/12612 = 8189 B. ~8K data transmitted for each read or write operation. Am I right here?
– Rick
Mar 5 at 6:37
If so, I think internal buffering should be ~8K. Read function call == write function call = 25224 / 2 = 12612. 98.5*1024*1024/12612 = 8189 B. ~8K data transmitted for each read or write operation. Am I right here?
– Rick
Mar 5 at 6:37
1
1
Ah, yes, I forgot to double the 98.5 before.
– Olorin
Mar 5 at 7:21
Ah, yes, I forgot to double the 98.5 before.
– Olorin
Mar 5 at 7:21
1
1
You can set the stdio buffer size to whatever you want with
setvbuf() (eg. to 1G), and then calculate yet another value ;-)– mosvy
Mar 5 at 11:02
You can set the stdio buffer size to whatever you want with
setvbuf() (eg. to 1G), and then calculate yet another value ;-)– mosvy
Mar 5 at 11:02
1
1
at the contrary, it's functions like fgetc or fputc (which may even be inlined) which need buffering the most, as the example from that book triea to demonstrate. In any case, they use full buffering by default (when not reading from some interactive device) in any implementation I may think of.
– mosvy
Mar 5 at 16:41
at the contrary, it's functions like fgetc or fputc (which may even be inlined) which need buffering the most, as the example from that book triea to demonstrate. In any case, they use full buffering by default (when not reading from some interactive device) in any implementation I may think of.
– mosvy
Mar 5 at 16:41
|
show 1 more comment
0
active
oldest
votes
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f504406%2fhow-does-200-million-function-call-of-fgets-translate-into-only-25-224-system-ca%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f504406%2fhow-does-200-million-function-call-of-fgets-translate-into-only-25-224-system-ca%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
3
Presumably
fgetcandfputcdo internal buffering of ~4k, dividing the whole count by 8k (98.5*1024*1024/4096 = 25216, which is remarkably close to the given number).– Olorin
Mar 5 at 6:01
1
If so, I think internal buffering should be ~8K. Read function call == write function call = 25224 / 2 = 12612. 98.5*1024*1024/12612 = 8189 B. ~8K data transmitted for each read or write operation. Am I right here?
– Rick
Mar 5 at 6:37
1
Ah, yes, I forgot to double the 98.5 before.
– Olorin
Mar 5 at 7:21
1
You can set the stdio buffer size to whatever you want with
setvbuf()(eg. to 1G), and then calculate yet another value ;-)– mosvy
Mar 5 at 11:02
1
at the contrary, it's functions like fgetc or fputc (which may even be inlined) which need buffering the most, as the example from that book triea to demonstrate. In any case, they use full buffering by default (when not reading from some interactive device) in any implementation I may think of.
– mosvy
Mar 5 at 16:41