How can I iterate through each line of a file using each line to replace a string in other files, creating a...
I have file 1 which contains:
AAAAAA
BBBBBB
I have file 2 which contains:
11111 22222 33333
and file 3 which contains:
22222 11111 33333
I want to insert each line in file 1 into file 2 and 3 replacing '11111' and creating a new file each time.
So the end result is 4 files:
File 1:
AAAAAA2222233333
File 2:
BBBBBB2222233333
File 3:
22222AAAAAA33333
File 4:
22222BBBBBB33333
I have attempted to use a nested for loop and sed to replace the values, at the moment I have this; however, it does not work:
(directory/* contains file 2 and file 3)
FILES=/directory/*
cat file1 | while read line ;do
for f in $FILES; do
cat $f | sed 's/11111/$i/g' > newfiles/$f$i.txt
done
done
Each time I run the script there may be a different number of files to edit, so I would like to iterate over the directory to make sure all files are covered. There will also be different numbers of strings in file 1.
bash shell-script sed
add a comment |
I have file 1 which contains:
AAAAAA
BBBBBB
I have file 2 which contains:
11111 22222 33333
and file 3 which contains:
22222 11111 33333
I want to insert each line in file 1 into file 2 and 3 replacing '11111' and creating a new file each time.
So the end result is 4 files:
File 1:
AAAAAA2222233333
File 2:
BBBBBB2222233333
File 3:
22222AAAAAA33333
File 4:
22222BBBBBB33333
I have attempted to use a nested for loop and sed to replace the values, at the moment I have this; however, it does not work:
(directory/* contains file 2 and file 3)
FILES=/directory/*
cat file1 | while read line ;do
for f in $FILES; do
cat $f | sed 's/11111/$i/g' > newfiles/$f$i.txt
done
done
Each time I run the script there may be a different number of files to edit, so I would like to iterate over the directory to make sure all files are covered. There will also be different numbers of strings in file 1.
bash shell-script sed
There will be a different number of files in the directory at each time, so i require the script to itterate through all files in the directory. Also, the format of each file may be different so i think sed replace is suitable.
– jonny b
Jan 20 at 0:14
1
And why are file1 and file2 the same and file3 and file4 the same? Please edit your question and clarify what you need to do.
– terdon♦
Jan 20 at 0:53
Apologies, my mistake, i have corrected them.
– jonny b
Jan 20 at 1:09
add a comment |
I have file 1 which contains:
AAAAAA
BBBBBB
I have file 2 which contains:
11111 22222 33333
and file 3 which contains:
22222 11111 33333
I want to insert each line in file 1 into file 2 and 3 replacing '11111' and creating a new file each time.
So the end result is 4 files:
File 1:
AAAAAA2222233333
File 2:
BBBBBB2222233333
File 3:
22222AAAAAA33333
File 4:
22222BBBBBB33333
I have attempted to use a nested for loop and sed to replace the values, at the moment I have this; however, it does not work:
(directory/* contains file 2 and file 3)
FILES=/directory/*
cat file1 | while read line ;do
for f in $FILES; do
cat $f | sed 's/11111/$i/g' > newfiles/$f$i.txt
done
done
Each time I run the script there may be a different number of files to edit, so I would like to iterate over the directory to make sure all files are covered. There will also be different numbers of strings in file 1.
bash shell-script sed
I have file 1 which contains:
AAAAAA
BBBBBB
I have file 2 which contains:
11111 22222 33333
and file 3 which contains:
22222 11111 33333
I want to insert each line in file 1 into file 2 and 3 replacing '11111' and creating a new file each time.
So the end result is 4 files:
File 1:
AAAAAA2222233333
File 2:
BBBBBB2222233333
File 3:
22222AAAAAA33333
File 4:
22222BBBBBB33333
I have attempted to use a nested for loop and sed to replace the values, at the moment I have this; however, it does not work:
(directory/* contains file 2 and file 3)
FILES=/directory/*
cat file1 | while read line ;do
for f in $FILES; do
cat $f | sed 's/11111/$i/g' > newfiles/$f$i.txt
done
done
Each time I run the script there may be a different number of files to edit, so I would like to iterate over the directory to make sure all files are covered. There will also be different numbers of strings in file 1.
bash shell-script sed
bash shell-script sed
edited Jan 20 at 2:04
Jeff Schaller
40.1k1054126
40.1k1054126
asked Jan 19 at 23:54
jonny bjonny b
336
336
There will be a different number of files in the directory at each time, so i require the script to itterate through all files in the directory. Also, the format of each file may be different so i think sed replace is suitable.
– jonny b
Jan 20 at 0:14
1
And why are file1 and file2 the same and file3 and file4 the same? Please edit your question and clarify what you need to do.
– terdon♦
Jan 20 at 0:53
Apologies, my mistake, i have corrected them.
– jonny b
Jan 20 at 1:09
add a comment |
There will be a different number of files in the directory at each time, so i require the script to itterate through all files in the directory. Also, the format of each file may be different so i think sed replace is suitable.
– jonny b
Jan 20 at 0:14
1
And why are file1 and file2 the same and file3 and file4 the same? Please edit your question and clarify what you need to do.
– terdon♦
Jan 20 at 0:53
Apologies, my mistake, i have corrected them.
– jonny b
Jan 20 at 1:09
There will be a different number of files in the directory at each time, so i require the script to itterate through all files in the directory. Also, the format of each file may be different so i think sed replace is suitable.
– jonny b
Jan 20 at 0:14
There will be a different number of files in the directory at each time, so i require the script to itterate through all files in the directory. Also, the format of each file may be different so i think sed replace is suitable.
– jonny b
Jan 20 at 0:14
1
1
And why are file1 and file2 the same and file3 and file4 the same? Please edit your question and clarify what you need to do.
– terdon♦
Jan 20 at 0:53
And why are file1 and file2 the same and file3 and file4 the same? Please edit your question and clarify what you need to do.
– terdon♦
Jan 20 at 0:53
Apologies, my mistake, i have corrected them.
– jonny b
Jan 20 at 1:09
Apologies, my mistake, i have corrected them.
– jonny b
Jan 20 at 1:09
add a comment |
3 Answers
3
active
oldest
votes
You need to iterate through the substitution file and then loop the target files. I think you are almost there. I have something like the following (I subbed some file names so that the globbing would work):
!/bin/bash
i=1
FILES=./other_file*
while read r; do
for f in $FILES; do
sed "s/11111/$r/g" > newfiles/file$i.txt < $f
i=$(( i + 1 ))
done
done <file1
Here we first indicate that we are using bash as the shell. Then set a counter i
to 1. Next glob
the files into a variable. The loop through file1 and assign each line to variable r
. Then for each of the files sub the 11111
string with whatever we saw in the line. Output it to a new file using the counter in the name and then increment the counter. Repeat until the loops are done.
Perfect thankyou!
– jonny b
Jan 20 at 2:39
add a comment |
How about Awk? save the mappings from file1
into an associative array, and then loop over the array for each line of the other files:
awk -v OFS= '
NR==FNR {t[NR]=$0; next}
{s = $0}
{
for(i in t) {
$0 = s;
sub(/11111/,t[i]);
$1=$1;
print > "newfile" ++n
}
}' file1 file2 file3
(the OFS=
and $1=$1
are just to force the record to be re-evaluated with en empty field separator, so as to remove the spaces between fields). Results:
$ head newfile*
==> newfile1 <==
AAAAAA2222233333
==> newfile2 <==
BBBBBB2222233333
==> newfile3 <==
22222AAAAAA33333
==> newfile4 <==
22222BBBBBB33333
Thankyou! this works great
– jonny b
Jan 20 at 2:39
add a comment |
I have used below script to achieve the same
for i in `cat mainfile`; do sed "s/11111/$i/g" file1 |sed -r "s/s+//g" >file1_$i; sed "s/11111/$i/g" file2|sed -r "s/s+//g" >file2_$i;done
output
Please find the output of all 4 files
AAAAAA2222233333
============================
BBBBBB2222233333
============================
22222AAAAAA33333
============================
22222BBBBBB33333
==========================
==
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f495539%2fhow-can-i-iterate-through-each-line-of-a-file-using-each-line-to-replace-a-strin%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
You need to iterate through the substitution file and then loop the target files. I think you are almost there. I have something like the following (I subbed some file names so that the globbing would work):
!/bin/bash
i=1
FILES=./other_file*
while read r; do
for f in $FILES; do
sed "s/11111/$r/g" > newfiles/file$i.txt < $f
i=$(( i + 1 ))
done
done <file1
Here we first indicate that we are using bash as the shell. Then set a counter i
to 1. Next glob
the files into a variable. The loop through file1 and assign each line to variable r
. Then for each of the files sub the 11111
string with whatever we saw in the line. Output it to a new file using the counter in the name and then increment the counter. Repeat until the loops are done.
Perfect thankyou!
– jonny b
Jan 20 at 2:39
add a comment |
You need to iterate through the substitution file and then loop the target files. I think you are almost there. I have something like the following (I subbed some file names so that the globbing would work):
!/bin/bash
i=1
FILES=./other_file*
while read r; do
for f in $FILES; do
sed "s/11111/$r/g" > newfiles/file$i.txt < $f
i=$(( i + 1 ))
done
done <file1
Here we first indicate that we are using bash as the shell. Then set a counter i
to 1. Next glob
the files into a variable. The loop through file1 and assign each line to variable r
. Then for each of the files sub the 11111
string with whatever we saw in the line. Output it to a new file using the counter in the name and then increment the counter. Repeat until the loops are done.
Perfect thankyou!
– jonny b
Jan 20 at 2:39
add a comment |
You need to iterate through the substitution file and then loop the target files. I think you are almost there. I have something like the following (I subbed some file names so that the globbing would work):
!/bin/bash
i=1
FILES=./other_file*
while read r; do
for f in $FILES; do
sed "s/11111/$r/g" > newfiles/file$i.txt < $f
i=$(( i + 1 ))
done
done <file1
Here we first indicate that we are using bash as the shell. Then set a counter i
to 1. Next glob
the files into a variable. The loop through file1 and assign each line to variable r
. Then for each of the files sub the 11111
string with whatever we saw in the line. Output it to a new file using the counter in the name and then increment the counter. Repeat until the loops are done.
You need to iterate through the substitution file and then loop the target files. I think you are almost there. I have something like the following (I subbed some file names so that the globbing would work):
!/bin/bash
i=1
FILES=./other_file*
while read r; do
for f in $FILES; do
sed "s/11111/$r/g" > newfiles/file$i.txt < $f
i=$(( i + 1 ))
done
done <file1
Here we first indicate that we are using bash as the shell. Then set a counter i
to 1. Next glob
the files into a variable. The loop through file1 and assign each line to variable r
. Then for each of the files sub the 11111
string with whatever we saw in the line. Output it to a new file using the counter in the name and then increment the counter. Repeat until the loops are done.
answered Jan 20 at 1:34
user1794469user1794469
1,5841822
1,5841822
Perfect thankyou!
– jonny b
Jan 20 at 2:39
add a comment |
Perfect thankyou!
– jonny b
Jan 20 at 2:39
Perfect thankyou!
– jonny b
Jan 20 at 2:39
Perfect thankyou!
– jonny b
Jan 20 at 2:39
add a comment |
How about Awk? save the mappings from file1
into an associative array, and then loop over the array for each line of the other files:
awk -v OFS= '
NR==FNR {t[NR]=$0; next}
{s = $0}
{
for(i in t) {
$0 = s;
sub(/11111/,t[i]);
$1=$1;
print > "newfile" ++n
}
}' file1 file2 file3
(the OFS=
and $1=$1
are just to force the record to be re-evaluated with en empty field separator, so as to remove the spaces between fields). Results:
$ head newfile*
==> newfile1 <==
AAAAAA2222233333
==> newfile2 <==
BBBBBB2222233333
==> newfile3 <==
22222AAAAAA33333
==> newfile4 <==
22222BBBBBB33333
Thankyou! this works great
– jonny b
Jan 20 at 2:39
add a comment |
How about Awk? save the mappings from file1
into an associative array, and then loop over the array for each line of the other files:
awk -v OFS= '
NR==FNR {t[NR]=$0; next}
{s = $0}
{
for(i in t) {
$0 = s;
sub(/11111/,t[i]);
$1=$1;
print > "newfile" ++n
}
}' file1 file2 file3
(the OFS=
and $1=$1
are just to force the record to be re-evaluated with en empty field separator, so as to remove the spaces between fields). Results:
$ head newfile*
==> newfile1 <==
AAAAAA2222233333
==> newfile2 <==
BBBBBB2222233333
==> newfile3 <==
22222AAAAAA33333
==> newfile4 <==
22222BBBBBB33333
Thankyou! this works great
– jonny b
Jan 20 at 2:39
add a comment |
How about Awk? save the mappings from file1
into an associative array, and then loop over the array for each line of the other files:
awk -v OFS= '
NR==FNR {t[NR]=$0; next}
{s = $0}
{
for(i in t) {
$0 = s;
sub(/11111/,t[i]);
$1=$1;
print > "newfile" ++n
}
}' file1 file2 file3
(the OFS=
and $1=$1
are just to force the record to be re-evaluated with en empty field separator, so as to remove the spaces between fields). Results:
$ head newfile*
==> newfile1 <==
AAAAAA2222233333
==> newfile2 <==
BBBBBB2222233333
==> newfile3 <==
22222AAAAAA33333
==> newfile4 <==
22222BBBBBB33333
How about Awk? save the mappings from file1
into an associative array, and then loop over the array for each line of the other files:
awk -v OFS= '
NR==FNR {t[NR]=$0; next}
{s = $0}
{
for(i in t) {
$0 = s;
sub(/11111/,t[i]);
$1=$1;
print > "newfile" ++n
}
}' file1 file2 file3
(the OFS=
and $1=$1
are just to force the record to be re-evaluated with en empty field separator, so as to remove the spaces between fields). Results:
$ head newfile*
==> newfile1 <==
AAAAAA2222233333
==> newfile2 <==
BBBBBB2222233333
==> newfile3 <==
22222AAAAAA33333
==> newfile4 <==
22222BBBBBB33333
answered Jan 20 at 1:39
steeldriversteeldriver
35.7k35286
35.7k35286
Thankyou! this works great
– jonny b
Jan 20 at 2:39
add a comment |
Thankyou! this works great
– jonny b
Jan 20 at 2:39
Thankyou! this works great
– jonny b
Jan 20 at 2:39
Thankyou! this works great
– jonny b
Jan 20 at 2:39
add a comment |
I have used below script to achieve the same
for i in `cat mainfile`; do sed "s/11111/$i/g" file1 |sed -r "s/s+//g" >file1_$i; sed "s/11111/$i/g" file2|sed -r "s/s+//g" >file2_$i;done
output
Please find the output of all 4 files
AAAAAA2222233333
============================
BBBBBB2222233333
============================
22222AAAAAA33333
============================
22222BBBBBB33333
==========================
==
add a comment |
I have used below script to achieve the same
for i in `cat mainfile`; do sed "s/11111/$i/g" file1 |sed -r "s/s+//g" >file1_$i; sed "s/11111/$i/g" file2|sed -r "s/s+//g" >file2_$i;done
output
Please find the output of all 4 files
AAAAAA2222233333
============================
BBBBBB2222233333
============================
22222AAAAAA33333
============================
22222BBBBBB33333
==========================
==
add a comment |
I have used below script to achieve the same
for i in `cat mainfile`; do sed "s/11111/$i/g" file1 |sed -r "s/s+//g" >file1_$i; sed "s/11111/$i/g" file2|sed -r "s/s+//g" >file2_$i;done
output
Please find the output of all 4 files
AAAAAA2222233333
============================
BBBBBB2222233333
============================
22222AAAAAA33333
============================
22222BBBBBB33333
==========================
==
I have used below script to achieve the same
for i in `cat mainfile`; do sed "s/11111/$i/g" file1 |sed -r "s/s+//g" >file1_$i; sed "s/11111/$i/g" file2|sed -r "s/s+//g" >file2_$i;done
output
Please find the output of all 4 files
AAAAAA2222233333
============================
BBBBBB2222233333
============================
22222AAAAAA33333
============================
22222BBBBBB33333
==========================
==
answered Jan 20 at 7:31
Praveen Kumar BSPraveen Kumar BS
1,366138
1,366138
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f495539%2fhow-can-i-iterate-through-each-line-of-a-file-using-each-line-to-replace-a-strin%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
There will be a different number of files in the directory at each time, so i require the script to itterate through all files in the directory. Also, the format of each file may be different so i think sed replace is suitable.
– jonny b
Jan 20 at 0:14
1
And why are file1 and file2 the same and file3 and file4 the same? Please edit your question and clarify what you need to do.
– terdon♦
Jan 20 at 0:53
Apologies, my mistake, i have corrected them.
– jonny b
Jan 20 at 1:09