Bash - Execute background process whilst reading output
1
I'm trying to start a process (target_executable) and have it run in the background. I know I could do that via ./target_executable & but in the bash script that's running the show, I want to read the output of the process looking for a particular output. Then once the output is . found, I want to let the the script complete whilst the target process is left running in the background.
This is what I have so far but there are a number of things wrong (its not running the process in the background, and it never hits "Finished Reading" even though the ID is found):
echo "Starting Process..."
TARGET_ID=""
./target_executable | while read -r line || [[ "$TARGET_ID" == "" ]] ; do
TARGET_ID=$(echo "$line" | grep -oE 'Id = [0-9A-Z]+' | grep -oE '[0-9A-Z]{10,}')
if [ "$TARGET_ID" != "" ]
then
echo "Processing $line '$TARGET_ID'"
fi
done
echo "Finished Reading..."
Any thoughts?
bash background-process
edited Jan 30 at 7:13
Rui F Ribeiro
40k1479135
asked Jan 30 at 6:14
anthonyvanthonyv
1061
add a comment |
1
I'm trying to start a process (target_executable) and have it run in the background. I know I could do that via ./target_executable & but in the bash script that's running the show, I want to read the output of the process looking for a particular output. Then once the output is . found, I want to let the the script complete whilst the target process is left running in the background.
This is what I have so far but there are a number of things wrong (its not running the process in the background, and it never hits "Finished Reading" even though the ID is found):
echo "Starting Process..."
TARGET_ID=""
./target_executable | while read -r line || [[ "$TARGET_ID" == "" ]] ; do
TARGET_ID=$(echo "$line" | grep -oE 'Id = [0-9A-Z]+' | grep -oE '[0-9A-Z]{10,}')
if [ "$TARGET_ID" != "" ]
then
echo "Processing $line '$TARGET_ID'"
fi
done
echo "Finished Reading..."
Any thoughts?
bash background-process
edited Jan 30 at 7:13
Rui F Ribeiro
40k1479135
asked Jan 30 at 6:14
anthonyvanthonyv
1061
add a comment |
1
1
1
I'm trying to start a process (target_executable) and have it run in the background. I know I could do that via ./target_executable & but in the bash script that's running the show, I want to read the output of the process looking for a particular output. Then once the output is . found, I want to let the the script complete whilst the target process is left running in the background.
This is what I have so far but there are a number of things wrong (its not running the process in the background, and it never hits "Finished Reading" even though the ID is found):
echo "Starting Process..."
TARGET_ID=""
./target_executable | while read -r line || [[ "$TARGET_ID" == "" ]] ; do
TARGET_ID=$(echo "$line" | grep -oE 'Id = [0-9A-Z]+' | grep -oE '[0-9A-Z]{10,}')
if [ "$TARGET_ID" != "" ]
then
echo "Processing $line '$TARGET_ID'"
fi
done
echo "Finished Reading..."
Any thoughts?
bash background-process
edited Jan 30 at 7:13
Rui F Ribeiro
40k1479135
asked Jan 30 at 6:14
anthonyvanthonyv
1061
I'm trying to start a process (target_executable) and have it run in the background. I know I could do that via ./target_executable & but in the bash script that's running the show, I want to read the output of the process looking for a particular output. Then once the output is . found, I want to let the the script complete whilst the target process is left running in the background.
This is what I have so far but there are a number of things wrong (its not running the process in the background, and it never hits "Finished Reading" even though the ID is found):
echo "Starting Process..."
TARGET_ID=""
./target_executable | while read -r line || [[ "$TARGET_ID" == "" ]] ; do
TARGET_ID=$(echo "$line" | grep -oE 'Id = [0-9A-Z]+' | grep -oE '[0-9A-Z]{10,}')
if [ "$TARGET_ID" != "" ]
then
echo "Processing $line '$TARGET_ID'"
fi
done
echo "Finished Reading..."
Any thoughts?
bash background-process
bash background-process
edited Jan 30 at 7:13
Rui F Ribeiro
40k1479135
asked Jan 30 at 6:14
anthonyvanthonyv
1061
edited Jan 30 at 7:13
Rui F Ribeiro
40k1479135
asked Jan 30 at 6:14
anthonyvanthonyv
1061
edited Jan 30 at 7:13
Rui F Ribeiro
40k1479135
edited Jan 30 at 7:13
Rui F Ribeiro
40k1479135
edited Jan 30 at 7:13
Rui F Ribeiro
40k1479135
40k1479135
asked Jan 30 at 6:14
anthonyvanthonyv
1061
asked Jan 30 at 6:14
anthonyvanthonyv
1061
asked Jan 30 at 6:14
anthonyvanthonyv
1061
1061
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
1
This looks like a job for coproc. From the help:
coproc: coproc [NAME] command [redirections]
Create a coprocess named NAME.
Execute COMMAND asynchronously, with the standard output and standard
input of the command connected via a pipe to file descriptors assigned
to indices 0 and 1 of an array variable NAME in the executing shell.
The default NAME is "COPROC".
So it would look something like:
echo "Starting Process..."
TARGET_ID=""
coproc (trap '' PIPE; ./target_executable < /dev/null & disown) # since it's in the bg, input won't be useful
while read -r line || [[ "$TARGET_ID" == "" ]] ; do
TARGET_ID=$(echo "$line" | grep -oE 'Id = [0-9A-Z]+' | grep -oE '[0-9A-Z]{10,}')
if [ "$TARGET_ID" != "" ]
then
echo "Processing $line '$TARGET_ID'"
break
fi
done <&${COPROC[0]} # redirect in from coprocess output
Note that bash sets up a pipe for the input/output of the coprocess, so the application must be able to handle a broken output pipe. Not all commands can. (That's why I trap SIGPIPE in the subshell.)
answered Jan 30 at 7:03
OlorinOlorin
3,3331418
Would you not have to read the rest of the output (stdout/stderr) from the co-process (if it produces anything)? It would pause otherwise. You can combine thosegrepinvocations and get rid ofread+echo, I think.
– Kusalananda
Jan 30 at 7:35
@Kusalananda I think when then script ends, the pipes will be closed (and so writes will start failing anyway). At least when I tested with a loopedecho, my terminal started displayingfoo.sh: line 5: echo: write error: Broken pipeafter the script that started coproc ended ... and I didn't really look at what OP was doing in the loop, TBH.
– Olorin
Jan 30 at 7:45
So, for the background process, output has to be directed to the terminal or a file for it to continue executing properly (not a pipe). If it produces output that is. Unfortunately it's not clear whether the process is expected to survive only to the ed of the controlling script or longer. If it should stick around for longer, it will need to be protected from a HUP signal as well I suppose.
– Kusalananda
Jan 30 at 7:48
That background task/process needs to live indefinitely even after the script process is terminated. Also there is no definitive "end" to the background processes output, the controlling script here should just "listen" until it sees what its looking for and then continue on its own execution and let the background task continue running in a fully detached manner. Does that make sense?
– anthonyv
Jan 30 at 15:59
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f497614%2fbash-execute-background-process-whilst-reading-output%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
1
This looks like a job for coproc. From the help:
coproc: coproc [NAME] command [redirections]
Create a coprocess named NAME.
Execute COMMAND asynchronously, with the standard output and standard
input of the command connected via a pipe to file descriptors assigned
to indices 0 and 1 of an array variable NAME in the executing shell.
The default NAME is "COPROC".
So it would look something like:
echo "Starting Process..."
TARGET_ID=""
coproc (trap '' PIPE; ./target_executable < /dev/null & disown) # since it's in the bg, input won't be useful
while read -r line || [[ "$TARGET_ID" == "" ]] ; do
TARGET_ID=$(echo "$line" | grep -oE 'Id = [0-9A-Z]+' | grep -oE '[0-9A-Z]{10,}')
if [ "$TARGET_ID" != "" ]
then
echo "Processing $line '$TARGET_ID'"
break
fi
done <&${COPROC[0]} # redirect in from coprocess output
Note that bash sets up a pipe for the input/output of the coprocess, so the application must be able to handle a broken output pipe. Not all commands can. (That's why I trap SIGPIPE in the subshell.)
answered Jan 30 at 7:03
OlorinOlorin
3,3331418
Would you not have to read the rest of the output (stdout/stderr) from the co-process (if it produces anything)? It would pause otherwise. You can combine thosegrepinvocations and get rid ofread+echo, I think.
– Kusalananda
Jan 30 at 7:35
@Kusalananda I think when then script ends, the pipes will be closed (and so writes will start failing anyway). At least when I tested with a loopedecho, my terminal started displayingfoo.sh: line 5: echo: write error: Broken pipeafter the script that started coproc ended ... and I didn't really look at what OP was doing in the loop, TBH.
– Olorin
Jan 30 at 7:45
So, for the background process, output has to be directed to the terminal or a file for it to continue executing properly (not a pipe). If it produces output that is. Unfortunately it's not clear whether the process is expected to survive only to the ed of the controlling script or longer. If it should stick around for longer, it will need to be protected from a HUP signal as well I suppose.
– Kusalananda
Jan 30 at 7:48
That background task/process needs to live indefinitely even after the script process is terminated. Also there is no definitive "end" to the background processes output, the controlling script here should just "listen" until it sees what its looking for and then continue on its own execution and let the background task continue running in a fully detached manner. Does that make sense?
– anthonyv
Jan 30 at 15:59
add a comment |
1
This looks like a job for coproc. From the help:
coproc: coproc [NAME] command [redirections]
Create a coprocess named NAME.
Execute COMMAND asynchronously, with the standard output and standard
input of the command connected via a pipe to file descriptors assigned
to indices 0 and 1 of an array variable NAME in the executing shell.
The default NAME is "COPROC".
So it would look something like:
echo "Starting Process..."
TARGET_ID=""
coproc (trap '' PIPE; ./target_executable < /dev/null & disown) # since it's in the bg, input won't be useful
while read -r line || [[ "$TARGET_ID" == "" ]] ; do
TARGET_ID=$(echo "$line" | grep -oE 'Id = [0-9A-Z]+' | grep -oE '[0-9A-Z]{10,}')
if [ "$TARGET_ID" != "" ]
then
echo "Processing $line '$TARGET_ID'"
break
fi
done <&${COPROC[0]} # redirect in from coprocess output
Note that bash sets up a pipe for the input/output of the coprocess, so the application must be able to handle a broken output pipe. Not all commands can. (That's why I trap SIGPIPE in the subshell.)
answered Jan 30 at 7:03
OlorinOlorin
3,3331418
Would you not have to read the rest of the output (stdout/stderr) from the co-process (if it produces anything)? It would pause otherwise. You can combine thosegrepinvocations and get rid ofread+echo, I think.
– Kusalananda
Jan 30 at 7:35
@Kusalananda I think when then script ends, the pipes will be closed (and so writes will start failing anyway). At least when I tested with a loopedecho, my terminal started displayingfoo.sh: line 5: echo: write error: Broken pipeafter the script that started coproc ended ... and I didn't really look at what OP was doing in the loop, TBH.
– Olorin
Jan 30 at 7:45
So, for the background process, output has to be directed to the terminal or a file for it to continue executing properly (not a pipe). If it produces output that is. Unfortunately it's not clear whether the process is expected to survive only to the ed of the controlling script or longer. If it should stick around for longer, it will need to be protected from a HUP signal as well I suppose.
– Kusalananda
Jan 30 at 7:48
That background task/process needs to live indefinitely even after the script process is terminated. Also there is no definitive "end" to the background processes output, the controlling script here should just "listen" until it sees what its looking for and then continue on its own execution and let the background task continue running in a fully detached manner. Does that make sense?
– anthonyv
Jan 30 at 15:59
add a comment |
1
1
1
This looks like a job for coproc. From the help:
coproc: coproc [NAME] command [redirections]
Create a coprocess named NAME.
Execute COMMAND asynchronously, with the standard output and standard
input of the command connected via a pipe to file descriptors assigned
to indices 0 and 1 of an array variable NAME in the executing shell.
The default NAME is "COPROC".
So it would look something like:
echo "Starting Process..."
TARGET_ID=""
coproc (trap '' PIPE; ./target_executable < /dev/null & disown) # since it's in the bg, input won't be useful
while read -r line || [[ "$TARGET_ID" == "" ]] ; do
TARGET_ID=$(echo "$line" | grep -oE 'Id = [0-9A-Z]+' | grep -oE '[0-9A-Z]{10,}')
if [ "$TARGET_ID" != "" ]
then
echo "Processing $line '$TARGET_ID'"
break
fi
done <&${COPROC[0]} # redirect in from coprocess output
Note that bash sets up a pipe for the input/output of the coprocess, so the application must be able to handle a broken output pipe. Not all commands can. (That's why I trap SIGPIPE in the subshell.)
answered Jan 30 at 7:03
OlorinOlorin
3,3331418
This looks like a job for coproc. From the help:
coproc: coproc [NAME] command [redirections]
Create a coprocess named NAME.
Execute COMMAND asynchronously, with the standard output and standard
input of the command connected via a pipe to file descriptors assigned
to indices 0 and 1 of an array variable NAME in the executing shell.
The default NAME is "COPROC".
So it would look something like:
echo "Starting Process..."
TARGET_ID=""
coproc (trap '' PIPE; ./target_executable < /dev/null & disown) # since it's in the bg, input won't be useful
while read -r line || [[ "$TARGET_ID" == "" ]] ; do
TARGET_ID=$(echo "$line" | grep -oE 'Id = [0-9A-Z]+' | grep -oE '[0-9A-Z]{10,}')
if [ "$TARGET_ID" != "" ]
then
echo "Processing $line '$TARGET_ID'"
break
fi
done <&${COPROC[0]} # redirect in from coprocess output
Note that bash sets up a pipe for the input/output of the coprocess, so the application must be able to handle a broken output pipe. Not all commands can. (That's why I trap SIGPIPE in the subshell.)
answered Jan 30 at 7:03
OlorinOlorin
3,3331418
answered Jan 30 at 7:03
OlorinOlorin
3,3331418
answered Jan 30 at 7:03
OlorinOlorin
3,3331418
answered Jan 30 at 7:03
OlorinOlorin
3,3331418
3,3331418
Would you not have to read the rest of the output (stdout/stderr) from the co-process (if it produces anything)? It would pause otherwise. You can combine thosegrepinvocations and get rid ofread+echo, I think.
– Kusalananda
Jan 30 at 7:35
@Kusalananda I think when then script ends, the pipes will be closed (and so writes will start failing anyway). At least when I tested with a loopedecho, my terminal started displayingfoo.sh: line 5: echo: write error: Broken pipeafter the script that started coproc ended ... and I didn't really look at what OP was doing in the loop, TBH.
– Olorin
Jan 30 at 7:45
So, for the background process, output has to be directed to the terminal or a file for it to continue executing properly (not a pipe). If it produces output that is. Unfortunately it's not clear whether the process is expected to survive only to the ed of the controlling script or longer. If it should stick around for longer, it will need to be protected from a HUP signal as well I suppose.
– Kusalananda
Jan 30 at 7:48
That background task/process needs to live indefinitely even after the script process is terminated. Also there is no definitive "end" to the background processes output, the controlling script here should just "listen" until it sees what its looking for and then continue on its own execution and let the background task continue running in a fully detached manner. Does that make sense?
– anthonyv
Jan 30 at 15:59
add a comment |
Would you not have to read the rest of the output (stdout/stderr) from the co-process (if it produces anything)? It would pause otherwise. You can combine thosegrepinvocations and get rid ofread+echo, I think.
– Kusalananda
Jan 30 at 7:35
@Kusalananda I think when then script ends, the pipes will be closed (and so writes will start failing anyway). At least when I tested with a loopedecho, my terminal started displayingfoo.sh: line 5: echo: write error: Broken pipeafter the script that started coproc ended ... and I didn't really look at what OP was doing in the loop, TBH.
– Olorin
Jan 30 at 7:45
So, for the background process, output has to be directed to the terminal or a file for it to continue executing properly (not a pipe). If it produces output that is. Unfortunately it's not clear whether the process is expected to survive only to the ed of the controlling script or longer. If it should stick around for longer, it will need to be protected from a HUP signal as well I suppose.
– Kusalananda
Jan 30 at 7:48
That background task/process needs to live indefinitely even after the script process is terminated. Also there is no definitive "end" to the background processes output, the controlling script here should just "listen" until it sees what its looking for and then continue on its own execution and let the background task continue running in a fully detached manner. Does that make sense?
– anthonyv
Jan 30 at 15:59
Would you not have to read the rest of the output (stdout/stderr) from the co-process (if it produces anything)? It would pause otherwise. You can combine those
grep invocations and get rid of read+echo, I think.– Kusalananda
Jan 30 at 7:35
Would you not have to read the rest of the output (stdout/stderr) from the co-process (if it produces anything)? It would pause otherwise. You can combine those
grep invocations and get rid of read+echo, I think.– Kusalananda
Jan 30 at 7:35
@Kusalananda I think when then script ends, the pipes will be closed (and so writes will start failing anyway). At least when I tested with a looped
echo, my terminal started displaying foo.sh: line 5: echo: write error: Broken pipe after the script that started coproc ended ... and I didn't really look at what OP was doing in the loop, TBH.– Olorin
Jan 30 at 7:45
@Kusalananda I think when then script ends, the pipes will be closed (and so writes will start failing anyway). At least when I tested with a looped
echo, my terminal started displaying foo.sh: line 5: echo: write error: Broken pipe after the script that started coproc ended ... and I didn't really look at what OP was doing in the loop, TBH.– Olorin
Jan 30 at 7:45
So, for the background process, output has to be directed to the terminal or a file for it to continue executing properly (not a pipe). If it produces output that is. Unfortunately it's not clear whether the process is expected to survive only to the ed of the controlling script or longer. If it should stick around for longer, it will need to be protected from a HUP signal as well I suppose.
– Kusalananda
Jan 30 at 7:48
So, for the background process, output has to be directed to the terminal or a file for it to continue executing properly (not a pipe). If it produces output that is. Unfortunately it's not clear whether the process is expected to survive only to the ed of the controlling script or longer. If it should stick around for longer, it will need to be protected from a HUP signal as well I suppose.
– Kusalananda
Jan 30 at 7:48
That background task/process needs to live indefinitely even after the script process is terminated. Also there is no definitive "end" to the background processes output, the controlling script here should just "listen" until it sees what its looking for and then continue on its own execution and let the background task continue running in a fully detached manner. Does that make sense?
– anthonyv
Jan 30 at 15:59
That background task/process needs to live indefinitely even after the script process is terminated. Also there is no definitive "end" to the background processes output, the controlling script here should just "listen" until it sees what its looking for and then continue on its own execution and let the background task continue running in a fully detached manner. Does that make sense?
– anthonyv
Jan 30 at 15:59
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f497614%2fbash-execute-background-process-whilst-reading-output%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
