Summary statistics in shell script












0















I have multiple files (records) across multiple directories. Each record lies in its own path based on the date it was created.



Eg a record on 12 Nov 2016 is



~/records/2016/11/12/record.


In each record, there is a number in the final line and I am trying to calculate summary statistics for each year.



How do I write a script that takes two user inputs:




  • year

  • statistic


where statistic could be average, max, min or all (avg, max
and min)?










share|improve this question





























    0















    I have multiple files (records) across multiple directories. Each record lies in its own path based on the date it was created.



    Eg a record on 12 Nov 2016 is



    ~/records/2016/11/12/record.


    In each record, there is a number in the final line and I am trying to calculate summary statistics for each year.



    How do I write a script that takes two user inputs:




    • year

    • statistic


    where statistic could be average, max, min or all (avg, max
    and min)?










    share|improve this question



























      0












      0








      0








      I have multiple files (records) across multiple directories. Each record lies in its own path based on the date it was created.



      Eg a record on 12 Nov 2016 is



      ~/records/2016/11/12/record.


      In each record, there is a number in the final line and I am trying to calculate summary statistics for each year.



      How do I write a script that takes two user inputs:




      • year

      • statistic


      where statistic could be average, max, min or all (avg, max
      and min)?










      share|improve this question
















      I have multiple files (records) across multiple directories. Each record lies in its own path based on the date it was created.



      Eg a record on 12 Nov 2016 is



      ~/records/2016/11/12/record.


      In each record, there is a number in the final line and I am trying to calculate summary statistics for each year.



      How do I write a script that takes two user inputs:




      • year

      • statistic


      where statistic could be average, max, min or all (avg, max
      and min)?







      shell-script awk files statistics






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Jan 29 at 13:04









      msp9011

      4,31844065




      4,31844065










      asked Jan 29 at 12:40









      marzomarzo

      132




      132






















          2 Answers
          2






          active

          oldest

          votes


















          1














          You can use the year input to define the root directory for a find command that searches all record files and calls tail -1 for every file.



          find ~/records/"$YEAR" -type f -name record -exec tail -1 {} ;


          This would be sufficient to print all values of the specified year.



          To calculate min/max/average pipe the output into awk for the calculation.



          awk -v stat="$STATISTICS" 'BEGIN { count=0; sum=0; }
          { if(!count) { min=$0; max=$0; }
          count++;
          sum += $0;
          if($0 > max) max = $0;
          if($0 < min) min = $0;
          }
          END {
          if(count) {
          # TODO: use variable "stat" to select only one result to print
          print min;
          print sum/count; # average
          print max;
          } else {
          print "no data";
          }
          }'


          You can combine and extend these snippets to get a complete script.






          share|improve this answer

































            0














            I have used below command to find the same and it worked fine



            echo "enter the year"
            read year

            find /records/$year/ -maxdepth 3 -type f -iname "filename" -exec sed -n '$p' {} ;| awk 'BEGIN{sum=0}{sum=sum+$1}END{print sum


            }'






            share|improve this answer























              Your Answer








              StackExchange.ready(function() {
              var channelOptions = {
              tags: "".split(" "),
              id: "106"
              };
              initTagRenderer("".split(" "), "".split(" "), channelOptions);

              StackExchange.using("externalEditor", function() {
              // Have to fire editor after snippets, if snippets enabled
              if (StackExchange.settings.snippets.snippetsEnabled) {
              StackExchange.using("snippets", function() {
              createEditor();
              });
              }
              else {
              createEditor();
              }
              });

              function createEditor() {
              StackExchange.prepareEditor({
              heartbeatType: 'answer',
              autoActivateHeartbeat: false,
              convertImagesToLinks: false,
              noModals: true,
              showLowRepImageUploadWarning: true,
              reputationToPostImages: null,
              bindNavPrevention: true,
              postfix: "",
              imageUploader: {
              brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
              contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
              allowUrls: true
              },
              onDemand: true,
              discardSelector: ".discard-answer"
              ,immediatelyShowMarkdownHelp:true
              });


              }
              });














              draft saved

              draft discarded


















              StackExchange.ready(
              function () {
              StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f497443%2fsummary-statistics-in-shell-script%23new-answer', 'question_page');
              }
              );

              Post as a guest















              Required, but never shown

























              2 Answers
              2






              active

              oldest

              votes








              2 Answers
              2






              active

              oldest

              votes









              active

              oldest

              votes






              active

              oldest

              votes









              1














              You can use the year input to define the root directory for a find command that searches all record files and calls tail -1 for every file.



              find ~/records/"$YEAR" -type f -name record -exec tail -1 {} ;


              This would be sufficient to print all values of the specified year.



              To calculate min/max/average pipe the output into awk for the calculation.



              awk -v stat="$STATISTICS" 'BEGIN { count=0; sum=0; }
              { if(!count) { min=$0; max=$0; }
              count++;
              sum += $0;
              if($0 > max) max = $0;
              if($0 < min) min = $0;
              }
              END {
              if(count) {
              # TODO: use variable "stat" to select only one result to print
              print min;
              print sum/count; # average
              print max;
              } else {
              print "no data";
              }
              }'


              You can combine and extend these snippets to get a complete script.






              share|improve this answer






























                1














                You can use the year input to define the root directory for a find command that searches all record files and calls tail -1 for every file.



                find ~/records/"$YEAR" -type f -name record -exec tail -1 {} ;


                This would be sufficient to print all values of the specified year.



                To calculate min/max/average pipe the output into awk for the calculation.



                awk -v stat="$STATISTICS" 'BEGIN { count=0; sum=0; }
                { if(!count) { min=$0; max=$0; }
                count++;
                sum += $0;
                if($0 > max) max = $0;
                if($0 < min) min = $0;
                }
                END {
                if(count) {
                # TODO: use variable "stat" to select only one result to print
                print min;
                print sum/count; # average
                print max;
                } else {
                print "no data";
                }
                }'


                You can combine and extend these snippets to get a complete script.






                share|improve this answer




























                  1












                  1








                  1







                  You can use the year input to define the root directory for a find command that searches all record files and calls tail -1 for every file.



                  find ~/records/"$YEAR" -type f -name record -exec tail -1 {} ;


                  This would be sufficient to print all values of the specified year.



                  To calculate min/max/average pipe the output into awk for the calculation.



                  awk -v stat="$STATISTICS" 'BEGIN { count=0; sum=0; }
                  { if(!count) { min=$0; max=$0; }
                  count++;
                  sum += $0;
                  if($0 > max) max = $0;
                  if($0 < min) min = $0;
                  }
                  END {
                  if(count) {
                  # TODO: use variable "stat" to select only one result to print
                  print min;
                  print sum/count; # average
                  print max;
                  } else {
                  print "no data";
                  }
                  }'


                  You can combine and extend these snippets to get a complete script.






                  share|improve this answer















                  You can use the year input to define the root directory for a find command that searches all record files and calls tail -1 for every file.



                  find ~/records/"$YEAR" -type f -name record -exec tail -1 {} ;


                  This would be sufficient to print all values of the specified year.



                  To calculate min/max/average pipe the output into awk for the calculation.



                  awk -v stat="$STATISTICS" 'BEGIN { count=0; sum=0; }
                  { if(!count) { min=$0; max=$0; }
                  count++;
                  sum += $0;
                  if($0 > max) max = $0;
                  if($0 < min) min = $0;
                  }
                  END {
                  if(count) {
                  # TODO: use variable "stat" to select only one result to print
                  print min;
                  print sum/count; # average
                  print max;
                  } else {
                  print "no data";
                  }
                  }'


                  You can combine and extend these snippets to get a complete script.







                  share|improve this answer














                  share|improve this answer



                  share|improve this answer








                  edited Jan 31 at 16:14

























                  answered Jan 29 at 13:10









                  BodoBodo

                  84818




                  84818

























                      0














                      I have used below command to find the same and it worked fine



                      echo "enter the year"
                      read year

                      find /records/$year/ -maxdepth 3 -type f -iname "filename" -exec sed -n '$p' {} ;| awk 'BEGIN{sum=0}{sum=sum+$1}END{print sum


                      }'






                      share|improve this answer




























                        0














                        I have used below command to find the same and it worked fine



                        echo "enter the year"
                        read year

                        find /records/$year/ -maxdepth 3 -type f -iname "filename" -exec sed -n '$p' {} ;| awk 'BEGIN{sum=0}{sum=sum+$1}END{print sum


                        }'






                        share|improve this answer


























                          0












                          0








                          0







                          I have used below command to find the same and it worked fine



                          echo "enter the year"
                          read year

                          find /records/$year/ -maxdepth 3 -type f -iname "filename" -exec sed -n '$p' {} ;| awk 'BEGIN{sum=0}{sum=sum+$1}END{print sum


                          }'






                          share|improve this answer













                          I have used below command to find the same and it worked fine



                          echo "enter the year"
                          read year

                          find /records/$year/ -maxdepth 3 -type f -iname "filename" -exec sed -n '$p' {} ;| awk 'BEGIN{sum=0}{sum=sum+$1}END{print sum


                          }'







                          share|improve this answer












                          share|improve this answer



                          share|improve this answer










                          answered Jan 29 at 17:00









                          Praveen Kumar BSPraveen Kumar BS

                          1,464138




                          1,464138






























                              draft saved

                              draft discarded




















































                              Thanks for contributing an answer to Unix & Linux Stack Exchange!


                              • Please be sure to answer the question. Provide details and share your research!

                              But avoid



                              • Asking for help, clarification, or responding to other answers.

                              • Making statements based on opinion; back them up with references or personal experience.


                              To learn more, see our tips on writing great answers.




                              draft saved


                              draft discarded














                              StackExchange.ready(
                              function () {
                              StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f497443%2fsummary-statistics-in-shell-script%23new-answer', 'question_page');
                              }
                              );

                              Post as a guest















                              Required, but never shown





















































                              Required, but never shown














                              Required, but never shown












                              Required, but never shown







                              Required, but never shown

































                              Required, but never shown














                              Required, but never shown












                              Required, but never shown







                              Required, but never shown







                              Popular posts from this blog

                              How to make a Squid Proxy server?

                              Is this a new Fibonacci Identity?

                              19世紀