How can I iterate through each line of a file using each line to replace a string in other files, creating a...












1















I have file 1 which contains:



AAAAAA
BBBBBB


I have file 2 which contains:



11111 22222 33333


and file 3 which contains:



22222 11111 33333


I want to insert each line in file 1 into file 2 and 3 replacing '11111' and creating a new file each time.
So the end result is 4 files:



File 1:



AAAAAA2222233333


File 2:



BBBBBB2222233333


File 3:



22222AAAAAA33333


File 4:



22222BBBBBB33333


I have attempted to use a nested for loop and sed to replace the values, at the moment I have this; however, it does not work:
(directory/* contains file 2 and file 3)



FILES=/directory/*
cat file1 | while read line ;do
for f in $FILES; do

cat $f | sed 's/11111/$i/g' > newfiles/$f$i.txt

done

done


Each time I run the script there may be a different number of files to edit, so I would like to iterate over the directory to make sure all files are covered. There will also be different numbers of strings in file 1.










share|improve this question

























  • There will be a different number of files in the directory at each time, so i require the script to itterate through all files in the directory. Also, the format of each file may be different so i think sed replace is suitable.

    – jonny b
    Jan 20 at 0:14






  • 1





    And why are file1 and file2 the same and file3 and file4 the same? Please edit your question and clarify what you need to do.

    – terdon
    Jan 20 at 0:53











  • Apologies, my mistake, i have corrected them.

    – jonny b
    Jan 20 at 1:09
















1















I have file 1 which contains:



AAAAAA
BBBBBB


I have file 2 which contains:



11111 22222 33333


and file 3 which contains:



22222 11111 33333


I want to insert each line in file 1 into file 2 and 3 replacing '11111' and creating a new file each time.
So the end result is 4 files:



File 1:



AAAAAA2222233333


File 2:



BBBBBB2222233333


File 3:



22222AAAAAA33333


File 4:



22222BBBBBB33333


I have attempted to use a nested for loop and sed to replace the values, at the moment I have this; however, it does not work:
(directory/* contains file 2 and file 3)



FILES=/directory/*
cat file1 | while read line ;do
for f in $FILES; do

cat $f | sed 's/11111/$i/g' > newfiles/$f$i.txt

done

done


Each time I run the script there may be a different number of files to edit, so I would like to iterate over the directory to make sure all files are covered. There will also be different numbers of strings in file 1.










share|improve this question

























  • There will be a different number of files in the directory at each time, so i require the script to itterate through all files in the directory. Also, the format of each file may be different so i think sed replace is suitable.

    – jonny b
    Jan 20 at 0:14






  • 1





    And why are file1 and file2 the same and file3 and file4 the same? Please edit your question and clarify what you need to do.

    – terdon
    Jan 20 at 0:53











  • Apologies, my mistake, i have corrected them.

    – jonny b
    Jan 20 at 1:09














1












1








1








I have file 1 which contains:



AAAAAA
BBBBBB


I have file 2 which contains:



11111 22222 33333


and file 3 which contains:



22222 11111 33333


I want to insert each line in file 1 into file 2 and 3 replacing '11111' and creating a new file each time.
So the end result is 4 files:



File 1:



AAAAAA2222233333


File 2:



BBBBBB2222233333


File 3:



22222AAAAAA33333


File 4:



22222BBBBBB33333


I have attempted to use a nested for loop and sed to replace the values, at the moment I have this; however, it does not work:
(directory/* contains file 2 and file 3)



FILES=/directory/*
cat file1 | while read line ;do
for f in $FILES; do

cat $f | sed 's/11111/$i/g' > newfiles/$f$i.txt

done

done


Each time I run the script there may be a different number of files to edit, so I would like to iterate over the directory to make sure all files are covered. There will also be different numbers of strings in file 1.










share|improve this question
















I have file 1 which contains:



AAAAAA
BBBBBB


I have file 2 which contains:



11111 22222 33333


and file 3 which contains:



22222 11111 33333


I want to insert each line in file 1 into file 2 and 3 replacing '11111' and creating a new file each time.
So the end result is 4 files:



File 1:



AAAAAA2222233333


File 2:



BBBBBB2222233333


File 3:



22222AAAAAA33333


File 4:



22222BBBBBB33333


I have attempted to use a nested for loop and sed to replace the values, at the moment I have this; however, it does not work:
(directory/* contains file 2 and file 3)



FILES=/directory/*
cat file1 | while read line ;do
for f in $FILES; do

cat $f | sed 's/11111/$i/g' > newfiles/$f$i.txt

done

done


Each time I run the script there may be a different number of files to edit, so I would like to iterate over the directory to make sure all files are covered. There will also be different numbers of strings in file 1.







bash shell-script sed






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Jan 20 at 2:04









Jeff Schaller

40.1k1054126




40.1k1054126










asked Jan 19 at 23:54









jonny bjonny b

336




336













  • There will be a different number of files in the directory at each time, so i require the script to itterate through all files in the directory. Also, the format of each file may be different so i think sed replace is suitable.

    – jonny b
    Jan 20 at 0:14






  • 1





    And why are file1 and file2 the same and file3 and file4 the same? Please edit your question and clarify what you need to do.

    – terdon
    Jan 20 at 0:53











  • Apologies, my mistake, i have corrected them.

    – jonny b
    Jan 20 at 1:09



















  • There will be a different number of files in the directory at each time, so i require the script to itterate through all files in the directory. Also, the format of each file may be different so i think sed replace is suitable.

    – jonny b
    Jan 20 at 0:14






  • 1





    And why are file1 and file2 the same and file3 and file4 the same? Please edit your question and clarify what you need to do.

    – terdon
    Jan 20 at 0:53











  • Apologies, my mistake, i have corrected them.

    – jonny b
    Jan 20 at 1:09

















There will be a different number of files in the directory at each time, so i require the script to itterate through all files in the directory. Also, the format of each file may be different so i think sed replace is suitable.

– jonny b
Jan 20 at 0:14





There will be a different number of files in the directory at each time, so i require the script to itterate through all files in the directory. Also, the format of each file may be different so i think sed replace is suitable.

– jonny b
Jan 20 at 0:14




1




1





And why are file1 and file2 the same and file3 and file4 the same? Please edit your question and clarify what you need to do.

– terdon
Jan 20 at 0:53





And why are file1 and file2 the same and file3 and file4 the same? Please edit your question and clarify what you need to do.

– terdon
Jan 20 at 0:53













Apologies, my mistake, i have corrected them.

– jonny b
Jan 20 at 1:09





Apologies, my mistake, i have corrected them.

– jonny b
Jan 20 at 1:09










3 Answers
3






active

oldest

votes


















2














You need to iterate through the substitution file and then loop the target files. I think you are almost there. I have something like the following (I subbed some file names so that the globbing would work):



!/bin/bash

i=1
FILES=./other_file*
while read r; do
for f in $FILES; do
sed "s/11111/$r/g" > newfiles/file$i.txt < $f
i=$(( i + 1 ))
done
done <file1


Here we first indicate that we are using bash as the shell. Then set a counter i to 1. Next glob the files into a variable. The loop through file1 and assign each line to variable r. Then for each of the files sub the 11111 string with whatever we saw in the line. Output it to a new file using the counter in the name and then increment the counter. Repeat until the loops are done.






share|improve this answer
























  • Perfect thankyou!

    – jonny b
    Jan 20 at 2:39



















2














How about Awk? save the mappings from file1 into an associative array, and then loop over the array for each line of the other files:



awk -v OFS= '
NR==FNR {t[NR]=$0; next}
{s = $0}
{
for(i in t) {
$0 = s;
sub(/11111/,t[i]);
$1=$1;
print > "newfile" ++n
}
}' file1 file2 file3


(the OFS= and $1=$1 are just to force the record to be re-evaluated with en empty field separator, so as to remove the spaces between fields). Results:



$ head newfile*
==> newfile1 <==
AAAAAA2222233333

==> newfile2 <==
BBBBBB2222233333

==> newfile3 <==
22222AAAAAA33333

==> newfile4 <==
22222BBBBBB33333





share|improve this answer
























  • Thankyou! this works great

    – jonny b
    Jan 20 at 2:39



















0














I have used below script to achieve the same



for i in `cat mainfile`; do sed "s/11111/$i/g" file1 |sed -r "s/s+//g" >file1_$i; sed "s/11111/$i/g" file2|sed -r "s/s+//g" >file2_$i;done


output



Please find the output of all 4 files



AAAAAA2222233333
============================
BBBBBB2222233333
============================
22222AAAAAA33333
============================
22222BBBBBB33333
==========================


==






share|improve this answer























    Your Answer








    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "106"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: false,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: null,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f495539%2fhow-can-i-iterate-through-each-line-of-a-file-using-each-line-to-replace-a-strin%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    3 Answers
    3






    active

    oldest

    votes








    3 Answers
    3






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    2














    You need to iterate through the substitution file and then loop the target files. I think you are almost there. I have something like the following (I subbed some file names so that the globbing would work):



    !/bin/bash

    i=1
    FILES=./other_file*
    while read r; do
    for f in $FILES; do
    sed "s/11111/$r/g" > newfiles/file$i.txt < $f
    i=$(( i + 1 ))
    done
    done <file1


    Here we first indicate that we are using bash as the shell. Then set a counter i to 1. Next glob the files into a variable. The loop through file1 and assign each line to variable r. Then for each of the files sub the 11111 string with whatever we saw in the line. Output it to a new file using the counter in the name and then increment the counter. Repeat until the loops are done.






    share|improve this answer
























    • Perfect thankyou!

      – jonny b
      Jan 20 at 2:39
















    2














    You need to iterate through the substitution file and then loop the target files. I think you are almost there. I have something like the following (I subbed some file names so that the globbing would work):



    !/bin/bash

    i=1
    FILES=./other_file*
    while read r; do
    for f in $FILES; do
    sed "s/11111/$r/g" > newfiles/file$i.txt < $f
    i=$(( i + 1 ))
    done
    done <file1


    Here we first indicate that we are using bash as the shell. Then set a counter i to 1. Next glob the files into a variable. The loop through file1 and assign each line to variable r. Then for each of the files sub the 11111 string with whatever we saw in the line. Output it to a new file using the counter in the name and then increment the counter. Repeat until the loops are done.






    share|improve this answer
























    • Perfect thankyou!

      – jonny b
      Jan 20 at 2:39














    2












    2








    2







    You need to iterate through the substitution file and then loop the target files. I think you are almost there. I have something like the following (I subbed some file names so that the globbing would work):



    !/bin/bash

    i=1
    FILES=./other_file*
    while read r; do
    for f in $FILES; do
    sed "s/11111/$r/g" > newfiles/file$i.txt < $f
    i=$(( i + 1 ))
    done
    done <file1


    Here we first indicate that we are using bash as the shell. Then set a counter i to 1. Next glob the files into a variable. The loop through file1 and assign each line to variable r. Then for each of the files sub the 11111 string with whatever we saw in the line. Output it to a new file using the counter in the name and then increment the counter. Repeat until the loops are done.






    share|improve this answer













    You need to iterate through the substitution file and then loop the target files. I think you are almost there. I have something like the following (I subbed some file names so that the globbing would work):



    !/bin/bash

    i=1
    FILES=./other_file*
    while read r; do
    for f in $FILES; do
    sed "s/11111/$r/g" > newfiles/file$i.txt < $f
    i=$(( i + 1 ))
    done
    done <file1


    Here we first indicate that we are using bash as the shell. Then set a counter i to 1. Next glob the files into a variable. The loop through file1 and assign each line to variable r. Then for each of the files sub the 11111 string with whatever we saw in the line. Output it to a new file using the counter in the name and then increment the counter. Repeat until the loops are done.







    share|improve this answer












    share|improve this answer



    share|improve this answer










    answered Jan 20 at 1:34









    user1794469user1794469

    1,5841822




    1,5841822













    • Perfect thankyou!

      – jonny b
      Jan 20 at 2:39



















    • Perfect thankyou!

      – jonny b
      Jan 20 at 2:39

















    Perfect thankyou!

    – jonny b
    Jan 20 at 2:39





    Perfect thankyou!

    – jonny b
    Jan 20 at 2:39













    2














    How about Awk? save the mappings from file1 into an associative array, and then loop over the array for each line of the other files:



    awk -v OFS= '
    NR==FNR {t[NR]=$0; next}
    {s = $0}
    {
    for(i in t) {
    $0 = s;
    sub(/11111/,t[i]);
    $1=$1;
    print > "newfile" ++n
    }
    }' file1 file2 file3


    (the OFS= and $1=$1 are just to force the record to be re-evaluated with en empty field separator, so as to remove the spaces between fields). Results:



    $ head newfile*
    ==> newfile1 <==
    AAAAAA2222233333

    ==> newfile2 <==
    BBBBBB2222233333

    ==> newfile3 <==
    22222AAAAAA33333

    ==> newfile4 <==
    22222BBBBBB33333





    share|improve this answer
























    • Thankyou! this works great

      – jonny b
      Jan 20 at 2:39
















    2














    How about Awk? save the mappings from file1 into an associative array, and then loop over the array for each line of the other files:



    awk -v OFS= '
    NR==FNR {t[NR]=$0; next}
    {s = $0}
    {
    for(i in t) {
    $0 = s;
    sub(/11111/,t[i]);
    $1=$1;
    print > "newfile" ++n
    }
    }' file1 file2 file3


    (the OFS= and $1=$1 are just to force the record to be re-evaluated with en empty field separator, so as to remove the spaces between fields). Results:



    $ head newfile*
    ==> newfile1 <==
    AAAAAA2222233333

    ==> newfile2 <==
    BBBBBB2222233333

    ==> newfile3 <==
    22222AAAAAA33333

    ==> newfile4 <==
    22222BBBBBB33333





    share|improve this answer
























    • Thankyou! this works great

      – jonny b
      Jan 20 at 2:39














    2












    2








    2







    How about Awk? save the mappings from file1 into an associative array, and then loop over the array for each line of the other files:



    awk -v OFS= '
    NR==FNR {t[NR]=$0; next}
    {s = $0}
    {
    for(i in t) {
    $0 = s;
    sub(/11111/,t[i]);
    $1=$1;
    print > "newfile" ++n
    }
    }' file1 file2 file3


    (the OFS= and $1=$1 are just to force the record to be re-evaluated with en empty field separator, so as to remove the spaces between fields). Results:



    $ head newfile*
    ==> newfile1 <==
    AAAAAA2222233333

    ==> newfile2 <==
    BBBBBB2222233333

    ==> newfile3 <==
    22222AAAAAA33333

    ==> newfile4 <==
    22222BBBBBB33333





    share|improve this answer













    How about Awk? save the mappings from file1 into an associative array, and then loop over the array for each line of the other files:



    awk -v OFS= '
    NR==FNR {t[NR]=$0; next}
    {s = $0}
    {
    for(i in t) {
    $0 = s;
    sub(/11111/,t[i]);
    $1=$1;
    print > "newfile" ++n
    }
    }' file1 file2 file3


    (the OFS= and $1=$1 are just to force the record to be re-evaluated with en empty field separator, so as to remove the spaces between fields). Results:



    $ head newfile*
    ==> newfile1 <==
    AAAAAA2222233333

    ==> newfile2 <==
    BBBBBB2222233333

    ==> newfile3 <==
    22222AAAAAA33333

    ==> newfile4 <==
    22222BBBBBB33333






    share|improve this answer












    share|improve this answer



    share|improve this answer










    answered Jan 20 at 1:39









    steeldriversteeldriver

    35.7k35286




    35.7k35286













    • Thankyou! this works great

      – jonny b
      Jan 20 at 2:39



















    • Thankyou! this works great

      – jonny b
      Jan 20 at 2:39

















    Thankyou! this works great

    – jonny b
    Jan 20 at 2:39





    Thankyou! this works great

    – jonny b
    Jan 20 at 2:39











    0














    I have used below script to achieve the same



    for i in `cat mainfile`; do sed "s/11111/$i/g" file1 |sed -r "s/s+//g" >file1_$i; sed "s/11111/$i/g" file2|sed -r "s/s+//g" >file2_$i;done


    output



    Please find the output of all 4 files



    AAAAAA2222233333
    ============================
    BBBBBB2222233333
    ============================
    22222AAAAAA33333
    ============================
    22222BBBBBB33333
    ==========================


    ==






    share|improve this answer




























      0














      I have used below script to achieve the same



      for i in `cat mainfile`; do sed "s/11111/$i/g" file1 |sed -r "s/s+//g" >file1_$i; sed "s/11111/$i/g" file2|sed -r "s/s+//g" >file2_$i;done


      output



      Please find the output of all 4 files



      AAAAAA2222233333
      ============================
      BBBBBB2222233333
      ============================
      22222AAAAAA33333
      ============================
      22222BBBBBB33333
      ==========================


      ==






      share|improve this answer


























        0












        0








        0







        I have used below script to achieve the same



        for i in `cat mainfile`; do sed "s/11111/$i/g" file1 |sed -r "s/s+//g" >file1_$i; sed "s/11111/$i/g" file2|sed -r "s/s+//g" >file2_$i;done


        output



        Please find the output of all 4 files



        AAAAAA2222233333
        ============================
        BBBBBB2222233333
        ============================
        22222AAAAAA33333
        ============================
        22222BBBBBB33333
        ==========================


        ==






        share|improve this answer













        I have used below script to achieve the same



        for i in `cat mainfile`; do sed "s/11111/$i/g" file1 |sed -r "s/s+//g" >file1_$i; sed "s/11111/$i/g" file2|sed -r "s/s+//g" >file2_$i;done


        output



        Please find the output of all 4 files



        AAAAAA2222233333
        ============================
        BBBBBB2222233333
        ============================
        22222AAAAAA33333
        ============================
        22222BBBBBB33333
        ==========================


        ==







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Jan 20 at 7:31









        Praveen Kumar BSPraveen Kumar BS

        1,366138




        1,366138






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Unix & Linux Stack Exchange!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f495539%2fhow-can-i-iterate-through-each-line-of-a-file-using-each-line-to-replace-a-strin%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            How to reconfigure Docker Trusted Registry 2.x.x to use CEPH FS mount instead of NFS and other traditional...

            is 'sed' thread safe

            How to make a Squid Proxy server?