Use Sed Regex Capture Group in Replace Section Method












2















I have a list of timestamped filenames in this following format:



cat files



  • ..201807010112.log

  • ..201807020112.log

  • ..201807022359.log

  • ..201807030112.log

  • ..201807010412.log


I need to get a certain range and in order to do this, I'd like to use SED.



sed -n '/201807010112/,/201807030112/p' 


Was my first attempt but sed treats the hour and minute strangely, and to make a long story short it's unable to understand this format correctly.



I may be getting in the weeds but in order to solve this, I've decided to convert the format to something SED understands.



cat files | sed -e "s/([0-9]{12}}/$(date -f '%Y%m%d%H%M' 1)/g"


My problem is that I cannot use the result of the match 1 in the date conversion block.



Is there a way to do this or better yet a better way to get the range of dates?










share|improve this question




















  • 3





    This is a very good example of a XY question...

    – don_crissti
    Aug 6 '18 at 22:23











  • cat 2018070{[1-2],301}

    – user1133275
    Aug 6 '18 at 22:49






  • 1





    like... sycamore?

    – mikeserv
    Aug 7 '18 at 0:52






  • 1





    Toxic comments, even I mentioned there could be a better way. User1133275, thanks for the suggestion but I need more flexibility when specifying the range. I'll answer my own question for the benefit of the community.

    – Proximo
    Aug 7 '18 at 1:47











  • Where is your files contents coming from? There might be a better way to get the filenames than from that file, e.g. using find against the actual files in the filesystem (and their timestamps). That is what @don_crissti meant by XY problem. You are may be focusing on an issue which is down a path that you don't need to take while solving another problem.

    – Kusalananda
    Aug 7 '18 at 6:14


















2















I have a list of timestamped filenames in this following format:



cat files



  • ..201807010112.log

  • ..201807020112.log

  • ..201807022359.log

  • ..201807030112.log

  • ..201807010412.log


I need to get a certain range and in order to do this, I'd like to use SED.



sed -n '/201807010112/,/201807030112/p' 


Was my first attempt but sed treats the hour and minute strangely, and to make a long story short it's unable to understand this format correctly.



I may be getting in the weeds but in order to solve this, I've decided to convert the format to something SED understands.



cat files | sed -e "s/([0-9]{12}}/$(date -f '%Y%m%d%H%M' 1)/g"


My problem is that I cannot use the result of the match 1 in the date conversion block.



Is there a way to do this or better yet a better way to get the range of dates?










share|improve this question




















  • 3





    This is a very good example of a XY question...

    – don_crissti
    Aug 6 '18 at 22:23











  • cat 2018070{[1-2],301}

    – user1133275
    Aug 6 '18 at 22:49






  • 1





    like... sycamore?

    – mikeserv
    Aug 7 '18 at 0:52






  • 1





    Toxic comments, even I mentioned there could be a better way. User1133275, thanks for the suggestion but I need more flexibility when specifying the range. I'll answer my own question for the benefit of the community.

    – Proximo
    Aug 7 '18 at 1:47











  • Where is your files contents coming from? There might be a better way to get the filenames than from that file, e.g. using find against the actual files in the filesystem (and their timestamps). That is what @don_crissti meant by XY problem. You are may be focusing on an issue which is down a path that you don't need to take while solving another problem.

    – Kusalananda
    Aug 7 '18 at 6:14
















2












2








2








I have a list of timestamped filenames in this following format:



cat files



  • ..201807010112.log

  • ..201807020112.log

  • ..201807022359.log

  • ..201807030112.log

  • ..201807010412.log


I need to get a certain range and in order to do this, I'd like to use SED.



sed -n '/201807010112/,/201807030112/p' 


Was my first attempt but sed treats the hour and minute strangely, and to make a long story short it's unable to understand this format correctly.



I may be getting in the weeds but in order to solve this, I've decided to convert the format to something SED understands.



cat files | sed -e "s/([0-9]{12}}/$(date -f '%Y%m%d%H%M' 1)/g"


My problem is that I cannot use the result of the match 1 in the date conversion block.



Is there a way to do this or better yet a better way to get the range of dates?










share|improve this question
















I have a list of timestamped filenames in this following format:



cat files



  • ..201807010112.log

  • ..201807020112.log

  • ..201807022359.log

  • ..201807030112.log

  • ..201807010412.log


I need to get a certain range and in order to do this, I'd like to use SED.



sed -n '/201807010112/,/201807030112/p' 


Was my first attempt but sed treats the hour and minute strangely, and to make a long story short it's unable to understand this format correctly.



I may be getting in the weeds but in order to solve this, I've decided to convert the format to something SED understands.



cat files | sed -e "s/([0-9]{12}}/$(date -f '%Y%m%d%H%M' 1)/g"


My problem is that I cannot use the result of the match 1 in the date conversion block.



Is there a way to do this or better yet a better way to get the range of dates?







linux awk sed grep regular-expression






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Aug 7 '18 at 1:23







Proximo

















asked Aug 6 '18 at 21:55









ProximoProximo

1305




1305








  • 3





    This is a very good example of a XY question...

    – don_crissti
    Aug 6 '18 at 22:23











  • cat 2018070{[1-2],301}

    – user1133275
    Aug 6 '18 at 22:49






  • 1





    like... sycamore?

    – mikeserv
    Aug 7 '18 at 0:52






  • 1





    Toxic comments, even I mentioned there could be a better way. User1133275, thanks for the suggestion but I need more flexibility when specifying the range. I'll answer my own question for the benefit of the community.

    – Proximo
    Aug 7 '18 at 1:47











  • Where is your files contents coming from? There might be a better way to get the filenames than from that file, e.g. using find against the actual files in the filesystem (and their timestamps). That is what @don_crissti meant by XY problem. You are may be focusing on an issue which is down a path that you don't need to take while solving another problem.

    – Kusalananda
    Aug 7 '18 at 6:14
















  • 3





    This is a very good example of a XY question...

    – don_crissti
    Aug 6 '18 at 22:23











  • cat 2018070{[1-2],301}

    – user1133275
    Aug 6 '18 at 22:49






  • 1





    like... sycamore?

    – mikeserv
    Aug 7 '18 at 0:52






  • 1





    Toxic comments, even I mentioned there could be a better way. User1133275, thanks for the suggestion but I need more flexibility when specifying the range. I'll answer my own question for the benefit of the community.

    – Proximo
    Aug 7 '18 at 1:47











  • Where is your files contents coming from? There might be a better way to get the filenames than from that file, e.g. using find against the actual files in the filesystem (and their timestamps). That is what @don_crissti meant by XY problem. You are may be focusing on an issue which is down a path that you don't need to take while solving another problem.

    – Kusalananda
    Aug 7 '18 at 6:14










3




3





This is a very good example of a XY question...

– don_crissti
Aug 6 '18 at 22:23





This is a very good example of a XY question...

– don_crissti
Aug 6 '18 at 22:23













cat 2018070{[1-2],301}

– user1133275
Aug 6 '18 at 22:49





cat 2018070{[1-2],301}

– user1133275
Aug 6 '18 at 22:49




1




1





like... sycamore?

– mikeserv
Aug 7 '18 at 0:52





like... sycamore?

– mikeserv
Aug 7 '18 at 0:52




1




1





Toxic comments, even I mentioned there could be a better way. User1133275, thanks for the suggestion but I need more flexibility when specifying the range. I'll answer my own question for the benefit of the community.

– Proximo
Aug 7 '18 at 1:47





Toxic comments, even I mentioned there could be a better way. User1133275, thanks for the suggestion but I need more flexibility when specifying the range. I'll answer my own question for the benefit of the community.

– Proximo
Aug 7 '18 at 1:47













Where is your files contents coming from? There might be a better way to get the filenames than from that file, e.g. using find against the actual files in the filesystem (and their timestamps). That is what @don_crissti meant by XY problem. You are may be focusing on an issue which is down a path that you don't need to take while solving another problem.

– Kusalananda
Aug 7 '18 at 6:14







Where is your files contents coming from? There might be a better way to get the filenames than from that file, e.g. using find against the actual files in the filesystem (and their timestamps). That is what @don_crissti meant by XY problem. You are may be focusing on an issue which is down a path that you don't need to take while solving another problem.

– Kusalananda
Aug 7 '18 at 6:14












2 Answers
2






active

oldest

votes


















1














Your question states that you need to get the names of some log files in a certain date range.



Let's disregard the fact that you have the filenames in a text file and instead assume that you have direct access to the files in some directory $logdir.



The format of the filenames are *_YYYYMMDDhhmmss.log where the end bit is a standard timestamp string.



To get the files between *_201807010112.log and *_201807030112.log in a loop (to do something to each of them), use (in bash),



process_flag=0

for pathname in "$logdir"/*_??????????????.log
do
if [ "$process_flag" -eq 0 ]; then
if [[ "$pathname" == *_201807010112.log ]]; then
process_flag=1
else
continue
fi
fi


# Do some sort of processing of
# the logfile in "$pathname" here.


# When done...

if [[ "$pathname" == *_201807030112.log ]]; then
break
fi
done


This loop loops over all logfiles that have a similar filename format. The loop will iterate over the pathnames in lexicographical order. It is assumed that all files have the same filename prefix (you say nothing about this).



The first part of the loop detects the first file in the range and sets process_flag to 1 when that file is found. Setting process_flag to 1 enables the loop to get into the middle bit where you actually use "$pathname" for whatever processing of that file that you need doing.



Before continuing with the next iteration, the last if statement checks to see whether the current $pathname matches the last file that we wanted to process. If it does, the loop ends by a break statement.






share|improve this answer































    0














    So I decided not to try to use date function to transform the form and instead use more SED!




    cat files






    • ..._201807010112.log




    • ..._201807010132.log




    • ..._201807010152.log




    • ..._201807010202.log




    The date is in the format YYYYmmddHHSS and I transformed it to the following format:



    YYYY-mm-dd HH:SS using this regex expression with SED.



    cat files | sed -e 's/(_[0-9]{4})([0-9]{2}) 
    ([0-9]{2})([0-9]{2})([0-9]{2}) /1-2-3 4:5/g'


    Next, from there it was a piece of cake to specify the range based on date.



      | sed -n '/2018-07-01 01:20/,/2018-07-02 02:01/p'


    The biggest take away for me was knowing the proper date-time format to utilize SED's range pattern option.






    share|improve this answer























      Your Answer








      StackExchange.ready(function() {
      var channelOptions = {
      tags: "".split(" "),
      id: "106"
      };
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function() {
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled) {
      StackExchange.using("snippets", function() {
      createEditor();
      });
      }
      else {
      createEditor();
      }
      });

      function createEditor() {
      StackExchange.prepareEditor({
      heartbeatType: 'answer',
      autoActivateHeartbeat: false,
      convertImagesToLinks: false,
      noModals: true,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: null,
      bindNavPrevention: true,
      postfix: "",
      imageUploader: {
      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
      allowUrls: true
      },
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      });


      }
      });














      draft saved

      draft discarded


















      StackExchange.ready(
      function () {
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f460912%2fuse-sed-regex-capture-group-in-replace-section-method%23new-answer', 'question_page');
      }
      );

      Post as a guest















      Required, but never shown

























      2 Answers
      2






      active

      oldest

      votes








      2 Answers
      2






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes









      1














      Your question states that you need to get the names of some log files in a certain date range.



      Let's disregard the fact that you have the filenames in a text file and instead assume that you have direct access to the files in some directory $logdir.



      The format of the filenames are *_YYYYMMDDhhmmss.log where the end bit is a standard timestamp string.



      To get the files between *_201807010112.log and *_201807030112.log in a loop (to do something to each of them), use (in bash),



      process_flag=0

      for pathname in "$logdir"/*_??????????????.log
      do
      if [ "$process_flag" -eq 0 ]; then
      if [[ "$pathname" == *_201807010112.log ]]; then
      process_flag=1
      else
      continue
      fi
      fi


      # Do some sort of processing of
      # the logfile in "$pathname" here.


      # When done...

      if [[ "$pathname" == *_201807030112.log ]]; then
      break
      fi
      done


      This loop loops over all logfiles that have a similar filename format. The loop will iterate over the pathnames in lexicographical order. It is assumed that all files have the same filename prefix (you say nothing about this).



      The first part of the loop detects the first file in the range and sets process_flag to 1 when that file is found. Setting process_flag to 1 enables the loop to get into the middle bit where you actually use "$pathname" for whatever processing of that file that you need doing.



      Before continuing with the next iteration, the last if statement checks to see whether the current $pathname matches the last file that we wanted to process. If it does, the loop ends by a break statement.






      share|improve this answer




























        1














        Your question states that you need to get the names of some log files in a certain date range.



        Let's disregard the fact that you have the filenames in a text file and instead assume that you have direct access to the files in some directory $logdir.



        The format of the filenames are *_YYYYMMDDhhmmss.log where the end bit is a standard timestamp string.



        To get the files between *_201807010112.log and *_201807030112.log in a loop (to do something to each of them), use (in bash),



        process_flag=0

        for pathname in "$logdir"/*_??????????????.log
        do
        if [ "$process_flag" -eq 0 ]; then
        if [[ "$pathname" == *_201807010112.log ]]; then
        process_flag=1
        else
        continue
        fi
        fi


        # Do some sort of processing of
        # the logfile in "$pathname" here.


        # When done...

        if [[ "$pathname" == *_201807030112.log ]]; then
        break
        fi
        done


        This loop loops over all logfiles that have a similar filename format. The loop will iterate over the pathnames in lexicographical order. It is assumed that all files have the same filename prefix (you say nothing about this).



        The first part of the loop detects the first file in the range and sets process_flag to 1 when that file is found. Setting process_flag to 1 enables the loop to get into the middle bit where you actually use "$pathname" for whatever processing of that file that you need doing.



        Before continuing with the next iteration, the last if statement checks to see whether the current $pathname matches the last file that we wanted to process. If it does, the loop ends by a break statement.






        share|improve this answer


























          1












          1








          1







          Your question states that you need to get the names of some log files in a certain date range.



          Let's disregard the fact that you have the filenames in a text file and instead assume that you have direct access to the files in some directory $logdir.



          The format of the filenames are *_YYYYMMDDhhmmss.log where the end bit is a standard timestamp string.



          To get the files between *_201807010112.log and *_201807030112.log in a loop (to do something to each of them), use (in bash),



          process_flag=0

          for pathname in "$logdir"/*_??????????????.log
          do
          if [ "$process_flag" -eq 0 ]; then
          if [[ "$pathname" == *_201807010112.log ]]; then
          process_flag=1
          else
          continue
          fi
          fi


          # Do some sort of processing of
          # the logfile in "$pathname" here.


          # When done...

          if [[ "$pathname" == *_201807030112.log ]]; then
          break
          fi
          done


          This loop loops over all logfiles that have a similar filename format. The loop will iterate over the pathnames in lexicographical order. It is assumed that all files have the same filename prefix (you say nothing about this).



          The first part of the loop detects the first file in the range and sets process_flag to 1 when that file is found. Setting process_flag to 1 enables the loop to get into the middle bit where you actually use "$pathname" for whatever processing of that file that you need doing.



          Before continuing with the next iteration, the last if statement checks to see whether the current $pathname matches the last file that we wanted to process. If it does, the loop ends by a break statement.






          share|improve this answer













          Your question states that you need to get the names of some log files in a certain date range.



          Let's disregard the fact that you have the filenames in a text file and instead assume that you have direct access to the files in some directory $logdir.



          The format of the filenames are *_YYYYMMDDhhmmss.log where the end bit is a standard timestamp string.



          To get the files between *_201807010112.log and *_201807030112.log in a loop (to do something to each of them), use (in bash),



          process_flag=0

          for pathname in "$logdir"/*_??????????????.log
          do
          if [ "$process_flag" -eq 0 ]; then
          if [[ "$pathname" == *_201807010112.log ]]; then
          process_flag=1
          else
          continue
          fi
          fi


          # Do some sort of processing of
          # the logfile in "$pathname" here.


          # When done...

          if [[ "$pathname" == *_201807030112.log ]]; then
          break
          fi
          done


          This loop loops over all logfiles that have a similar filename format. The loop will iterate over the pathnames in lexicographical order. It is assumed that all files have the same filename prefix (you say nothing about this).



          The first part of the loop detects the first file in the range and sets process_flag to 1 when that file is found. Setting process_flag to 1 enables the loop to get into the middle bit where you actually use "$pathname" for whatever processing of that file that you need doing.



          Before continuing with the next iteration, the last if statement checks to see whether the current $pathname matches the last file that we wanted to process. If it does, the loop ends by a break statement.







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Feb 15 at 20:23









          KusalanandaKusalananda

          133k17254417




          133k17254417

























              0














              So I decided not to try to use date function to transform the form and instead use more SED!




              cat files






              • ..._201807010112.log




              • ..._201807010132.log




              • ..._201807010152.log




              • ..._201807010202.log




              The date is in the format YYYYmmddHHSS and I transformed it to the following format:



              YYYY-mm-dd HH:SS using this regex expression with SED.



              cat files | sed -e 's/(_[0-9]{4})([0-9]{2}) 
              ([0-9]{2})([0-9]{2})([0-9]{2}) /1-2-3 4:5/g'


              Next, from there it was a piece of cake to specify the range based on date.



                | sed -n '/2018-07-01 01:20/,/2018-07-02 02:01/p'


              The biggest take away for me was knowing the proper date-time format to utilize SED's range pattern option.






              share|improve this answer




























                0














                So I decided not to try to use date function to transform the form and instead use more SED!




                cat files






                • ..._201807010112.log




                • ..._201807010132.log




                • ..._201807010152.log




                • ..._201807010202.log




                The date is in the format YYYYmmddHHSS and I transformed it to the following format:



                YYYY-mm-dd HH:SS using this regex expression with SED.



                cat files | sed -e 's/(_[0-9]{4})([0-9]{2}) 
                ([0-9]{2})([0-9]{2})([0-9]{2}) /1-2-3 4:5/g'


                Next, from there it was a piece of cake to specify the range based on date.



                  | sed -n '/2018-07-01 01:20/,/2018-07-02 02:01/p'


                The biggest take away for me was knowing the proper date-time format to utilize SED's range pattern option.






                share|improve this answer


























                  0












                  0








                  0







                  So I decided not to try to use date function to transform the form and instead use more SED!




                  cat files






                  • ..._201807010112.log




                  • ..._201807010132.log




                  • ..._201807010152.log




                  • ..._201807010202.log




                  The date is in the format YYYYmmddHHSS and I transformed it to the following format:



                  YYYY-mm-dd HH:SS using this regex expression with SED.



                  cat files | sed -e 's/(_[0-9]{4})([0-9]{2}) 
                  ([0-9]{2})([0-9]{2})([0-9]{2}) /1-2-3 4:5/g'


                  Next, from there it was a piece of cake to specify the range based on date.



                    | sed -n '/2018-07-01 01:20/,/2018-07-02 02:01/p'


                  The biggest take away for me was knowing the proper date-time format to utilize SED's range pattern option.






                  share|improve this answer













                  So I decided not to try to use date function to transform the form and instead use more SED!




                  cat files






                  • ..._201807010112.log




                  • ..._201807010132.log




                  • ..._201807010152.log




                  • ..._201807010202.log




                  The date is in the format YYYYmmddHHSS and I transformed it to the following format:



                  YYYY-mm-dd HH:SS using this regex expression with SED.



                  cat files | sed -e 's/(_[0-9]{4})([0-9]{2}) 
                  ([0-9]{2})([0-9]{2})([0-9]{2}) /1-2-3 4:5/g'


                  Next, from there it was a piece of cake to specify the range based on date.



                    | sed -n '/2018-07-01 01:20/,/2018-07-02 02:01/p'


                  The biggest take away for me was knowing the proper date-time format to utilize SED's range pattern option.







                  share|improve this answer












                  share|improve this answer



                  share|improve this answer










                  answered Aug 7 '18 at 1:59









                  ProximoProximo

                  1305




                  1305






























                      draft saved

                      draft discarded




















































                      Thanks for contributing an answer to Unix & Linux Stack Exchange!


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid



                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.


                      To learn more, see our tips on writing great answers.




                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function () {
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f460912%2fuse-sed-regex-capture-group-in-replace-section-method%23new-answer', 'question_page');
                      }
                      );

                      Post as a guest















                      Required, but never shown





















































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown

































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown







                      Popular posts from this blog

                      How to reconfigure Docker Trusted Registry 2.x.x to use CEPH FS mount instead of NFS and other traditional...

                      is 'sed' thread safe

                      How to make a Squid Proxy server?