Removing control chars (including console codes / colours) from script output












63















I can use the "script" command to record an interactive session at the command line. However, this includes all control characters and colour codes. I can remove control characters (like backspace) with "col -b", but I can't find a simple way to remove the colour codes.



Note that I want to use the command line in the normal way, so don't want to disable colours there - I just want to remove them from the script output. Also, I know can play around and try find a regexp to fix things up, but I am hoping there is a simpler (and more reliable - what if there's a code I don't know about when I develop the regexp?) solution.



To show the problem:




spl62 tmp: script
Script started, file is typescript
spl62 lepl: ls
add-licence.sed build-example.sh commit-test push-docs.sh
add-licence.sh build.sh delete-licence.sed setup.py
asn build-test.sh delete-licence.sh src
build-doc.sh clean doc-src test.ini
spl62 lepl: exit
Script done, file is typescript
spl62 tmp: cat -v typescript
Script started on Thu 09 Jun 2011 09:47:27 AM CLT
spl62 lepl: ls^M
^[[0m^[[00madd-licence.sed^[[0m ^[[00;32mbuild-example.sh^[[0m ^[[00mcommit-test^[[0m ^[[00;32mpush-docs.sh^[[0m^M
^[[00;32madd-licence.sh^[[0m ^[[00;32mbuild.sh^[[0m ^[[00mdelete-licence.sed^[[0m ^[[00msetup.py^[[0m^M
^[[01;34masn^[[0m ^[[00;32mbuild-test.sh^[[0m ^[[00;32mdelete-licence.sh^[[0m ^[[01;34msrc^[[0m^M
^[[00;32mbuild-doc.sh^[[0m ^[[00;32mclean^[[0m ^[[01;34mdoc-src^[[0m ^[[00mtest.ini^[[0m^M
spl62 lepl: exit^M

Script done on Thu 09 Jun 2011 09:47:29 AM CLT
spl62 tmp: col -b < typescript
Script started on Thu 09 Jun 2011 09:47:27 AM CLT
spl62 lepl: ls
0m00madd-licence.sed0m 00;32mbuild-example.sh0m 00mcommit-test0m 00;32mpush-docs.sh0m
00;32madd-licence.sh0m 00;32mbuild.sh0m 00mdelete-licence.sed0m 00msetup.py0m
01;34masn0m 00;32mbuild-test.sh0m 00;32mdelete-licence.sh0m 01;34msrc0m
00;32mbuild-doc.sh0m 00;32mclean0m 01;34mdoc-src0m 00mtest.ini0m
spl62 lepl: exit

Script done on Thu 09 Jun 2011 09:47:29 AM CLT









share|improve this question





























    63















    I can use the "script" command to record an interactive session at the command line. However, this includes all control characters and colour codes. I can remove control characters (like backspace) with "col -b", but I can't find a simple way to remove the colour codes.



    Note that I want to use the command line in the normal way, so don't want to disable colours there - I just want to remove them from the script output. Also, I know can play around and try find a regexp to fix things up, but I am hoping there is a simpler (and more reliable - what if there's a code I don't know about when I develop the regexp?) solution.



    To show the problem:




    spl62 tmp: script
    Script started, file is typescript
    spl62 lepl: ls
    add-licence.sed build-example.sh commit-test push-docs.sh
    add-licence.sh build.sh delete-licence.sed setup.py
    asn build-test.sh delete-licence.sh src
    build-doc.sh clean doc-src test.ini
    spl62 lepl: exit
    Script done, file is typescript
    spl62 tmp: cat -v typescript
    Script started on Thu 09 Jun 2011 09:47:27 AM CLT
    spl62 lepl: ls^M
    ^[[0m^[[00madd-licence.sed^[[0m ^[[00;32mbuild-example.sh^[[0m ^[[00mcommit-test^[[0m ^[[00;32mpush-docs.sh^[[0m^M
    ^[[00;32madd-licence.sh^[[0m ^[[00;32mbuild.sh^[[0m ^[[00mdelete-licence.sed^[[0m ^[[00msetup.py^[[0m^M
    ^[[01;34masn^[[0m ^[[00;32mbuild-test.sh^[[0m ^[[00;32mdelete-licence.sh^[[0m ^[[01;34msrc^[[0m^M
    ^[[00;32mbuild-doc.sh^[[0m ^[[00;32mclean^[[0m ^[[01;34mdoc-src^[[0m ^[[00mtest.ini^[[0m^M
    spl62 lepl: exit^M

    Script done on Thu 09 Jun 2011 09:47:29 AM CLT
    spl62 tmp: col -b < typescript
    Script started on Thu 09 Jun 2011 09:47:27 AM CLT
    spl62 lepl: ls
    0m00madd-licence.sed0m 00;32mbuild-example.sh0m 00mcommit-test0m 00;32mpush-docs.sh0m
    00;32madd-licence.sh0m 00;32mbuild.sh0m 00mdelete-licence.sed0m 00msetup.py0m
    01;34masn0m 00;32mbuild-test.sh0m 00;32mdelete-licence.sh0m 01;34msrc0m
    00;32mbuild-doc.sh0m 00;32mclean0m 01;34mdoc-src0m 00mtest.ini0m
    spl62 lepl: exit

    Script done on Thu 09 Jun 2011 09:47:29 AM CLT









    share|improve this question



























      63












      63








      63


      22






      I can use the "script" command to record an interactive session at the command line. However, this includes all control characters and colour codes. I can remove control characters (like backspace) with "col -b", but I can't find a simple way to remove the colour codes.



      Note that I want to use the command line in the normal way, so don't want to disable colours there - I just want to remove them from the script output. Also, I know can play around and try find a regexp to fix things up, but I am hoping there is a simpler (and more reliable - what if there's a code I don't know about when I develop the regexp?) solution.



      To show the problem:




      spl62 tmp: script
      Script started, file is typescript
      spl62 lepl: ls
      add-licence.sed build-example.sh commit-test push-docs.sh
      add-licence.sh build.sh delete-licence.sed setup.py
      asn build-test.sh delete-licence.sh src
      build-doc.sh clean doc-src test.ini
      spl62 lepl: exit
      Script done, file is typescript
      spl62 tmp: cat -v typescript
      Script started on Thu 09 Jun 2011 09:47:27 AM CLT
      spl62 lepl: ls^M
      ^[[0m^[[00madd-licence.sed^[[0m ^[[00;32mbuild-example.sh^[[0m ^[[00mcommit-test^[[0m ^[[00;32mpush-docs.sh^[[0m^M
      ^[[00;32madd-licence.sh^[[0m ^[[00;32mbuild.sh^[[0m ^[[00mdelete-licence.sed^[[0m ^[[00msetup.py^[[0m^M
      ^[[01;34masn^[[0m ^[[00;32mbuild-test.sh^[[0m ^[[00;32mdelete-licence.sh^[[0m ^[[01;34msrc^[[0m^M
      ^[[00;32mbuild-doc.sh^[[0m ^[[00;32mclean^[[0m ^[[01;34mdoc-src^[[0m ^[[00mtest.ini^[[0m^M
      spl62 lepl: exit^M

      Script done on Thu 09 Jun 2011 09:47:29 AM CLT
      spl62 tmp: col -b < typescript
      Script started on Thu 09 Jun 2011 09:47:27 AM CLT
      spl62 lepl: ls
      0m00madd-licence.sed0m 00;32mbuild-example.sh0m 00mcommit-test0m 00;32mpush-docs.sh0m
      00;32madd-licence.sh0m 00;32mbuild.sh0m 00mdelete-licence.sed0m 00msetup.py0m
      01;34masn0m 00;32mbuild-test.sh0m 00;32mdelete-licence.sh0m 01;34msrc0m
      00;32mbuild-doc.sh0m 00;32mclean0m 01;34mdoc-src0m 00mtest.ini0m
      spl62 lepl: exit

      Script done on Thu 09 Jun 2011 09:47:29 AM CLT









      share|improve this question
















      I can use the "script" command to record an interactive session at the command line. However, this includes all control characters and colour codes. I can remove control characters (like backspace) with "col -b", but I can't find a simple way to remove the colour codes.



      Note that I want to use the command line in the normal way, so don't want to disable colours there - I just want to remove them from the script output. Also, I know can play around and try find a regexp to fix things up, but I am hoping there is a simpler (and more reliable - what if there's a code I don't know about when I develop the regexp?) solution.



      To show the problem:




      spl62 tmp: script
      Script started, file is typescript
      spl62 lepl: ls
      add-licence.sed build-example.sh commit-test push-docs.sh
      add-licence.sh build.sh delete-licence.sed setup.py
      asn build-test.sh delete-licence.sh src
      build-doc.sh clean doc-src test.ini
      spl62 lepl: exit
      Script done, file is typescript
      spl62 tmp: cat -v typescript
      Script started on Thu 09 Jun 2011 09:47:27 AM CLT
      spl62 lepl: ls^M
      ^[[0m^[[00madd-licence.sed^[[0m ^[[00;32mbuild-example.sh^[[0m ^[[00mcommit-test^[[0m ^[[00;32mpush-docs.sh^[[0m^M
      ^[[00;32madd-licence.sh^[[0m ^[[00;32mbuild.sh^[[0m ^[[00mdelete-licence.sed^[[0m ^[[00msetup.py^[[0m^M
      ^[[01;34masn^[[0m ^[[00;32mbuild-test.sh^[[0m ^[[00;32mdelete-licence.sh^[[0m ^[[01;34msrc^[[0m^M
      ^[[00;32mbuild-doc.sh^[[0m ^[[00;32mclean^[[0m ^[[01;34mdoc-src^[[0m ^[[00mtest.ini^[[0m^M
      spl62 lepl: exit^M

      Script done on Thu 09 Jun 2011 09:47:29 AM CLT
      spl62 tmp: col -b < typescript
      Script started on Thu 09 Jun 2011 09:47:27 AM CLT
      spl62 lepl: ls
      0m00madd-licence.sed0m 00;32mbuild-example.sh0m 00mcommit-test0m 00;32mpush-docs.sh0m
      00;32madd-licence.sh0m 00;32mbuild.sh0m 00mdelete-licence.sed0m 00msetup.py0m
      01;34masn0m 00;32mbuild-test.sh0m 00;32mdelete-licence.sh0m 01;34msrc0m
      00;32mbuild-doc.sh0m 00;32mclean0m 01;34mdoc-src0m 00mtest.ini0m
      spl62 lepl: exit

      Script done on Thu 09 Jun 2011 09:47:29 AM CLT






      text-processing terminal colors escape-characters typescript






      share|improve this question















      share|improve this question













      share|improve this question




      share|improve this question








      edited Jun 9 '11 at 20:45









      Gilles

      538k12810881606




      538k12810881606










      asked Jun 9 '11 at 13:51









      andrew cookeandrew cooke

      421147




      421147






















          10 Answers
          10






          active

          oldest

          votes


















          53














          The following script should filter out all ANSI/VT100/xterm control sequences for (based on ctlseqs). Minimally tested, please report any under- or over-match.



          #!/usr/bin/env perl
          ## uncolor — remove terminal escape sequences such as color changes
          while (<>) {
          s/ e[ #%()*+-./]. |
          e[ [ -?]* [@-~] | # CSI ... Cmd
          e] .*? (?:e\|[ax9c]) | # OSC ... (ST|BEL)
          e[P^_] .*? (?:e\|x9c) | # (DCS|PM|APC) ... ST
          e. //xg;
          print;
          }


          Known issues:




          • Doesn't complain about malformed sequences. That's not what this script is for.

          • Multi-line string arguments to DCS/PM/APC/OSC are not supported.

          • Bytes in the range 128–159 may be parsed as control characters, though this is rarely used. Here's a version which parses non-ASCII control characters (this will mangle non-ASCII text in some encodings including UTF-8).


          #!/usr/bin/env perl
          ## uncolor — remove terminal escape sequences such as color changes
          while (<>) {
          s/ e[ #%()*+-./]. |
          (?:e[|x9b) [ -?]* [@-~] | # CSI ... Cmd
          (?:e]|x9d) .*? (?:e\|[ax9c]) | # OSC ... (ST|BEL)
          (?:e[P^_]|[x90x9ex9f]) .*? (?:e\|x9c) | # (DCS|PM|APC) ... ST
          e.|[x80-x9f] //xg;
          print;
          }





          share|improve this answer


























          • thanks to both answers. i felt i should make something as a good answer, although both give regexps, which i wanted to avoid. chose this one as it gives a reference for the format.

            – andrew cooke
            Jun 16 '11 at 12:49











          • @andrew: My regexp is flexible enough that I expect it to work with pretty much any now-existing terminal, and probably with any tomorrow-existing terminal as well. I haven't tested it much, so there might be bugs, but the approach is sound as control sequences follow a few general patterns.

            – Gilles
            Jun 16 '11 at 13:53













          • please provide how to use this script. does it require pipe input? or positional arguments?

            – Trevor Boyd Smith
            Jan 5 '18 at 17:14











          • @TrevorBoydSmith Either will work for input, and the output is always on standard output, like typical text utilities.

            – Gilles
            Jan 5 '18 at 20:31













          • This mangles multibyte characters such as ☺ (xe2 x98 xba). The [x80-x9f] clause strips the middle byte.

            – Jeffrey
            Jun 9 '18 at 4:52



















          29














          Updating Gilles' answer to also remove carriage returns and do backspace-erasing of previous characters, which were both important to me for a typescript generated on Cygwin:



          #!/usr/bin/perl
          while (<>) {
          s/ e[ #%()*+-./]. |
          r | # Remove extra carriage returns also
          (?:e[|x9b) [ -?]* [@-~] | # CSI ... Cmd
          (?:e]|x9d) .*? (?:e\|[ax9c]) | # OSC ... (ST|BEL)
          (?:e[P^_]|[x90x9ex9f]) .*? (?:e\|x9c) | # (DCS|PM|APC) ... ST
          e.|[x80-x9f] //xg;
          1 while s/[^b][b]//g; # remove all non-backspace followed by backspace
          print;
          }





          share|improve this answer


























          • +1 I was already typing a post whith the same question as the OP when I fond this message with your script and that of @Gilles. +1 for both of you

            – miracle173
            Apr 29 '12 at 22:02



















          8














          I would use sed in this case.



          do:



          cat -v typescript | sed -e "s/x1b[.{1,5}m//g"



          sed -e "s/search/replace/g" is standard stuff. the regex is explained as below:



          x1b match the Escape preceeding the color code
          [ matches the first open bracket
          .{1,5} matches 1 to 5 of any single character. Have to the curly braces to keep the shell from mangling them.
          m last character in regex - usually trails the color code.
          // empty string for what to replace everything with.
          g match it multiple times per line.






          share|improve this answer



















          • 2





            This regex strips too much (fooe[1m(1m = { becomes foo = { instead of foo(m = {), replacing . by [0-9;] is more accurate.

            – Lekensteyn
            Dec 30 '13 at 15:14











          • Replace .{1,5} with [^m]{1,5} for that - but also note that this even then still only removes "graphics rendition" codes (those that end in an m) - basically color, reverse, bold and italics styles (where applicable).

            – Hannu
            Aug 23 '15 at 14:26













          • This doesn't remove x1b(B (included in rusts color output)

            – ideasman42
            Nov 11 '16 at 13:11











          • Why is it x1b and not 33?

            – atripes
            Aug 24 '17 at 15:21



















          7














          cat typescript | perl -pe 's/e([^[]]|[.*?[a-zA-Z]|].*?a)//g' | col -b > typescript-processed





          share|improve this answer
























          • The only one that actually worked, thanks!

            – Will
            Sep 29 '16 at 15:23



















          5














          # The "sed -r" trick does not work on every Linux, I still dunno why:
          DECOLORIZE='eval sed "s,${END}[[0-9;]*[m|K],,g"'


          => howto use:



          <commands that type colored output> | ${DECOLORIZE}


          tested on:
          - AIX 5.x / 6.1 / 7.1
          - Linux Mandrake / Mandriva / SLES / Fedora
          - SunOS






          share|improve this answer

































            2














            I solved the problem by running scriptreplay in a screen and the dumping the scrollback buffer to a file.



            The following expect script does this for you.



            It has been tested for logfiles with up to 250.000 lines. In the working directory you need your scriptlog, a file called "time" with 10.000.000 times the line "1 10" in it, and the script. I needs the name of your scriptfile as command line argument, like ./name_of_script name_of_scriptlog.



            #!/usr/bin/expect -f 

            set logfile [lindex $argv 0]

            if {$logfile == ""} {puts "Usage: ./script_to_readable.exp $logfile."; exit}

            set timestamp [clock format [clock sec] -format %Y-%m-%d,%H:%M:%S]
            set pwd [exec pwd]
            if {! [file exists ${pwd}/time]} {puts "ERROR: time file not found.nYou need a file named time with 10.000.000 times the line "1 10" in the working directory for this script to work. Please provide it."; exit}
            set wc [exec cat ${pwd}/$logfile | wc -l]
            set height [ expr "$wc" + "100" ]
            system cp $logfile ${logfile}.tmp
            system echo $timestamp >> ${logfile}.tmp
            set timeout -1
            spawn screen -h $height -S $timestamp
            send "scriptreplay -t time -s ${logfile}.tmp 100000 2>/dev/nullr"
            expect ${timestamp}
            send "x01:hardcopy -h readablelog.${timestamp}r"

            send "exitr"

            system sed '/^$/d' readablelog.$timestamp >> readablelog2.$timestamp
            system head -n-2 readablelog2.$timestamp >> ${logfile}.readable.$timestamp
            system rm -f readablelog.$timestamp readablelog2.$timestamp ${logfile}.tmp


            The time file can be generated by



            for i in $(seq 1 10000000); do echo "1 10" >> time; done





            share|improve this answer


























            • The command for generating time file generated 100% CPU usage for a few minutes and after it finished my memory usage was 100% and running command resulted in "fork: cannot allocate memory". And it didn't really work as expected.

              – Barteks2x
              May 27 '16 at 17:29











            • There's a far easier way to generate the timing file. The fields are "delay blocksize", so there's no reason not to just make it "0 <entirefile>" and dump the whole thing with no delay. You can do that by taking the size of the script minus the first line (tail -n +2 typescript|wc -c), and create the timing file with echo "0 "`tail -n +2 typescript|wc -c` > timing. That'll be basically instant, and scriptreplay will replay the entire script at fastest possible speed.

              – FeRD
              Jul 11 '18 at 5:27





















            1














            Found this question while looking for the a solution to the same problem. A little more digging and found this script over at Live Journal at this link. I worked perfectly for me. It's also a very good write up about this problem and how the solution works. Definitely worth a read.
            http://jdimpson.livejournal.com/7040.html



            #!/usr/bin/perl -wp

            # clean up control characters and other non-text detritus that shows up
            # when you run the "script" command.

            BEGIN {
            # xterm titlebar escape sequence
            $xtermesc = "x1bx5dx30x3b";

            # the occurence of a backspace event (e.g. cntrl H, cntrol W, or cntrl U)
            $backspaceevent = "x1b\x5bx4b"; # note escaping of third character

            # ANSI color escape sequence
            $ansiesc = qr/x1b[[d;]*?m/;

            # technically, this is arrow-right. For some reason, being used against
            # very long backspace jobs. I don't fully understand this, as evidenced
            # by the fact that is off by one sometimes.
            $bizarrebs = qr/x1b[C/;

            # used as part of the xterm titlebar mechanism, or when
            # a bell sounds, which might happen when you backspace too much.
            $bell = "x07"; # could use a

            $cr = "x0d"; # could use r

            $backspace = "x08"; # could use b
            }

            s/$xtermesc.+?$bell//g;
            s/[$cr$bell]//g;
            s/${backspaceevent}//g;
            s/$ansiesc//g;
            while (s/(.)(?=$backspace)//) { s/$backspace//; } # frickin' sweet
            # For every ^H delete the character immediately left of it, then delete the ^H.
            # Perl's RE's aren't R, so I wonder if I could do this in one expression.
            while (s/(..)(?=$bizarrebs)//) { s/$bizarrebs//; }





            share|improve this answer































              0














              I found that just using cat was all I needed to view the output of script in the terminal. This doesn't help when redirecting the output to another file, but does make the result readable, unlike cat -v, col -b, or a text editor.



              To eliminate colors or save the results to a file, manually copy and paste the output from cat into a text editor, or into another cat command, i.e.:



              cat > endResult << END
              <paste_copied_text_here>
              END





              share|improve this answer





















              • 1





                did your script run include output with color codes attached, as in the OP's case?

                – Jeff Schaller
                Feb 7 at 15:34











              • Using cat presents the original colors, which can be removed by manual copy-and-paste. The OP used cat -v and col -b, both of which present codes rather than a properly-formatted end result. I have edited my answer.

                – Roger Dueck
                Feb 7 at 15:53





















              -2














              Following up on the last answer which uses tr and :cntrl: could we maybe do



              sed "/^[[:cntrl:]]/d" output.txt



              This seems to work for me because all lines generated by vi start with a control character. It happens to also strip out blank lines and lines that start with a tab, although that works for what I'm doing. Maybe there is a way to match any control character except for n m t.



              Maybe we can search for the particular control character, and it looks like all junk lines generated by vi start with what looks like ^[. hexdump tells me the first character is 1b, so this seems to work too



              sed "/^x1b/d" output.txt



              This looks similar to an answer posted above, but it does not work properly because after running the command, some junk chars are already added to the command line as if the user had typed them.






              share|improve this answer



















              • 1





                There is no "last answer" as the answers can and do change order. You should use the "share" button underneath the answer you want to reference, and include that as a link in your answer. Assuming your answer is sufficient to be more than a comment, of course. Right now I can't identify which of the several answers you're referencing.

                – roaima
                Jun 23 '17 at 21:22













              • “could we maybe do …”  Yes, we could do that — but it would delete every line that begins with a control character.  On the output of, for example, ls --color (as shown in the question), your solution will delete almost every line that contains information.  Not good.  But thanks for leaving out the useless use of cat.    :-)    ⁠

                – G-Man
                Jun 23 '17 at 21:23













              • Is there a way to create a character class that is :iscntrl: but not :isspace:? Maybe some syntax like ^[[:iscntrl:]-[:isspace]]

                – snaran
                Jun 23 '17 at 22:09





















              -4














              tr - translate or delete characters



              cat typescript | tr -d [[:cntrl:]]





              share|improve this answer


























              • Welcome to Unix Stackexchange! When giving an answer it is preferable to give some explanation as to WHY your answer is the one.

                – Stephen Rauch
                Feb 19 '17 at 16:47











              • Brevity is acceptable, but fuller explanations are better., also doesn't quite solve the issue here.

                – Kusalananda
                Feb 19 '17 at 17:27








              • 3





                This actually will not work correctly, as it will not remove a 01;34m for example, and will remove the end of line newline (n).

                – sorontar
                Feb 19 '17 at 17:41











              Your Answer








              StackExchange.ready(function() {
              var channelOptions = {
              tags: "".split(" "),
              id: "106"
              };
              initTagRenderer("".split(" "), "".split(" "), channelOptions);

              StackExchange.using("externalEditor", function() {
              // Have to fire editor after snippets, if snippets enabled
              if (StackExchange.settings.snippets.snippetsEnabled) {
              StackExchange.using("snippets", function() {
              createEditor();
              });
              }
              else {
              createEditor();
              }
              });

              function createEditor() {
              StackExchange.prepareEditor({
              heartbeatType: 'answer',
              autoActivateHeartbeat: false,
              convertImagesToLinks: false,
              noModals: true,
              showLowRepImageUploadWarning: true,
              reputationToPostImages: null,
              bindNavPrevention: true,
              postfix: "",
              imageUploader: {
              brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
              contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
              allowUrls: true
              },
              onDemand: true,
              discardSelector: ".discard-answer"
              ,immediatelyShowMarkdownHelp:true
              });


              }
              });














              draft saved

              draft discarded


















              StackExchange.ready(
              function () {
              StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f14684%2fremoving-control-chars-including-console-codes-colours-from-script-output%23new-answer', 'question_page');
              }
              );

              Post as a guest















              Required, but never shown

























              10 Answers
              10






              active

              oldest

              votes








              10 Answers
              10






              active

              oldest

              votes









              active

              oldest

              votes






              active

              oldest

              votes









              53














              The following script should filter out all ANSI/VT100/xterm control sequences for (based on ctlseqs). Minimally tested, please report any under- or over-match.



              #!/usr/bin/env perl
              ## uncolor — remove terminal escape sequences such as color changes
              while (<>) {
              s/ e[ #%()*+-./]. |
              e[ [ -?]* [@-~] | # CSI ... Cmd
              e] .*? (?:e\|[ax9c]) | # OSC ... (ST|BEL)
              e[P^_] .*? (?:e\|x9c) | # (DCS|PM|APC) ... ST
              e. //xg;
              print;
              }


              Known issues:




              • Doesn't complain about malformed sequences. That's not what this script is for.

              • Multi-line string arguments to DCS/PM/APC/OSC are not supported.

              • Bytes in the range 128–159 may be parsed as control characters, though this is rarely used. Here's a version which parses non-ASCII control characters (this will mangle non-ASCII text in some encodings including UTF-8).


              #!/usr/bin/env perl
              ## uncolor — remove terminal escape sequences such as color changes
              while (<>) {
              s/ e[ #%()*+-./]. |
              (?:e[|x9b) [ -?]* [@-~] | # CSI ... Cmd
              (?:e]|x9d) .*? (?:e\|[ax9c]) | # OSC ... (ST|BEL)
              (?:e[P^_]|[x90x9ex9f]) .*? (?:e\|x9c) | # (DCS|PM|APC) ... ST
              e.|[x80-x9f] //xg;
              print;
              }





              share|improve this answer


























              • thanks to both answers. i felt i should make something as a good answer, although both give regexps, which i wanted to avoid. chose this one as it gives a reference for the format.

                – andrew cooke
                Jun 16 '11 at 12:49











              • @andrew: My regexp is flexible enough that I expect it to work with pretty much any now-existing terminal, and probably with any tomorrow-existing terminal as well. I haven't tested it much, so there might be bugs, but the approach is sound as control sequences follow a few general patterns.

                – Gilles
                Jun 16 '11 at 13:53













              • please provide how to use this script. does it require pipe input? or positional arguments?

                – Trevor Boyd Smith
                Jan 5 '18 at 17:14











              • @TrevorBoydSmith Either will work for input, and the output is always on standard output, like typical text utilities.

                – Gilles
                Jan 5 '18 at 20:31













              • This mangles multibyte characters such as ☺ (xe2 x98 xba). The [x80-x9f] clause strips the middle byte.

                – Jeffrey
                Jun 9 '18 at 4:52
















              53














              The following script should filter out all ANSI/VT100/xterm control sequences for (based on ctlseqs). Minimally tested, please report any under- or over-match.



              #!/usr/bin/env perl
              ## uncolor — remove terminal escape sequences such as color changes
              while (<>) {
              s/ e[ #%()*+-./]. |
              e[ [ -?]* [@-~] | # CSI ... Cmd
              e] .*? (?:e\|[ax9c]) | # OSC ... (ST|BEL)
              e[P^_] .*? (?:e\|x9c) | # (DCS|PM|APC) ... ST
              e. //xg;
              print;
              }


              Known issues:




              • Doesn't complain about malformed sequences. That's not what this script is for.

              • Multi-line string arguments to DCS/PM/APC/OSC are not supported.

              • Bytes in the range 128–159 may be parsed as control characters, though this is rarely used. Here's a version which parses non-ASCII control characters (this will mangle non-ASCII text in some encodings including UTF-8).


              #!/usr/bin/env perl
              ## uncolor — remove terminal escape sequences such as color changes
              while (<>) {
              s/ e[ #%()*+-./]. |
              (?:e[|x9b) [ -?]* [@-~] | # CSI ... Cmd
              (?:e]|x9d) .*? (?:e\|[ax9c]) | # OSC ... (ST|BEL)
              (?:e[P^_]|[x90x9ex9f]) .*? (?:e\|x9c) | # (DCS|PM|APC) ... ST
              e.|[x80-x9f] //xg;
              print;
              }





              share|improve this answer


























              • thanks to both answers. i felt i should make something as a good answer, although both give regexps, which i wanted to avoid. chose this one as it gives a reference for the format.

                – andrew cooke
                Jun 16 '11 at 12:49











              • @andrew: My regexp is flexible enough that I expect it to work with pretty much any now-existing terminal, and probably with any tomorrow-existing terminal as well. I haven't tested it much, so there might be bugs, but the approach is sound as control sequences follow a few general patterns.

                – Gilles
                Jun 16 '11 at 13:53













              • please provide how to use this script. does it require pipe input? or positional arguments?

                – Trevor Boyd Smith
                Jan 5 '18 at 17:14











              • @TrevorBoydSmith Either will work for input, and the output is always on standard output, like typical text utilities.

                – Gilles
                Jan 5 '18 at 20:31













              • This mangles multibyte characters such as ☺ (xe2 x98 xba). The [x80-x9f] clause strips the middle byte.

                – Jeffrey
                Jun 9 '18 at 4:52














              53












              53








              53







              The following script should filter out all ANSI/VT100/xterm control sequences for (based on ctlseqs). Minimally tested, please report any under- or over-match.



              #!/usr/bin/env perl
              ## uncolor — remove terminal escape sequences such as color changes
              while (<>) {
              s/ e[ #%()*+-./]. |
              e[ [ -?]* [@-~] | # CSI ... Cmd
              e] .*? (?:e\|[ax9c]) | # OSC ... (ST|BEL)
              e[P^_] .*? (?:e\|x9c) | # (DCS|PM|APC) ... ST
              e. //xg;
              print;
              }


              Known issues:




              • Doesn't complain about malformed sequences. That's not what this script is for.

              • Multi-line string arguments to DCS/PM/APC/OSC are not supported.

              • Bytes in the range 128–159 may be parsed as control characters, though this is rarely used. Here's a version which parses non-ASCII control characters (this will mangle non-ASCII text in some encodings including UTF-8).


              #!/usr/bin/env perl
              ## uncolor — remove terminal escape sequences such as color changes
              while (<>) {
              s/ e[ #%()*+-./]. |
              (?:e[|x9b) [ -?]* [@-~] | # CSI ... Cmd
              (?:e]|x9d) .*? (?:e\|[ax9c]) | # OSC ... (ST|BEL)
              (?:e[P^_]|[x90x9ex9f]) .*? (?:e\|x9c) | # (DCS|PM|APC) ... ST
              e.|[x80-x9f] //xg;
              print;
              }





              share|improve this answer















              The following script should filter out all ANSI/VT100/xterm control sequences for (based on ctlseqs). Minimally tested, please report any under- or over-match.



              #!/usr/bin/env perl
              ## uncolor — remove terminal escape sequences such as color changes
              while (<>) {
              s/ e[ #%()*+-./]. |
              e[ [ -?]* [@-~] | # CSI ... Cmd
              e] .*? (?:e\|[ax9c]) | # OSC ... (ST|BEL)
              e[P^_] .*? (?:e\|x9c) | # (DCS|PM|APC) ... ST
              e. //xg;
              print;
              }


              Known issues:




              • Doesn't complain about malformed sequences. That's not what this script is for.

              • Multi-line string arguments to DCS/PM/APC/OSC are not supported.

              • Bytes in the range 128–159 may be parsed as control characters, though this is rarely used. Here's a version which parses non-ASCII control characters (this will mangle non-ASCII text in some encodings including UTF-8).


              #!/usr/bin/env perl
              ## uncolor — remove terminal escape sequences such as color changes
              while (<>) {
              s/ e[ #%()*+-./]. |
              (?:e[|x9b) [ -?]* [@-~] | # CSI ... Cmd
              (?:e]|x9d) .*? (?:e\|[ax9c]) | # OSC ... (ST|BEL)
              (?:e[P^_]|[x90x9ex9f]) .*? (?:e\|x9c) | # (DCS|PM|APC) ... ST
              e.|[x80-x9f] //xg;
              print;
              }






              share|improve this answer














              share|improve this answer



              share|improve this answer








              edited Jun 10 '18 at 0:06

























              answered Jun 9 '11 at 20:43









              GillesGilles

              538k12810881606




              538k12810881606













              • thanks to both answers. i felt i should make something as a good answer, although both give regexps, which i wanted to avoid. chose this one as it gives a reference for the format.

                – andrew cooke
                Jun 16 '11 at 12:49











              • @andrew: My regexp is flexible enough that I expect it to work with pretty much any now-existing terminal, and probably with any tomorrow-existing terminal as well. I haven't tested it much, so there might be bugs, but the approach is sound as control sequences follow a few general patterns.

                – Gilles
                Jun 16 '11 at 13:53













              • please provide how to use this script. does it require pipe input? or positional arguments?

                – Trevor Boyd Smith
                Jan 5 '18 at 17:14











              • @TrevorBoydSmith Either will work for input, and the output is always on standard output, like typical text utilities.

                – Gilles
                Jan 5 '18 at 20:31













              • This mangles multibyte characters such as ☺ (xe2 x98 xba). The [x80-x9f] clause strips the middle byte.

                – Jeffrey
                Jun 9 '18 at 4:52



















              • thanks to both answers. i felt i should make something as a good answer, although both give regexps, which i wanted to avoid. chose this one as it gives a reference for the format.

                – andrew cooke
                Jun 16 '11 at 12:49











              • @andrew: My regexp is flexible enough that I expect it to work with pretty much any now-existing terminal, and probably with any tomorrow-existing terminal as well. I haven't tested it much, so there might be bugs, but the approach is sound as control sequences follow a few general patterns.

                – Gilles
                Jun 16 '11 at 13:53













              • please provide how to use this script. does it require pipe input? or positional arguments?

                – Trevor Boyd Smith
                Jan 5 '18 at 17:14











              • @TrevorBoydSmith Either will work for input, and the output is always on standard output, like typical text utilities.

                – Gilles
                Jan 5 '18 at 20:31













              • This mangles multibyte characters such as ☺ (xe2 x98 xba). The [x80-x9f] clause strips the middle byte.

                – Jeffrey
                Jun 9 '18 at 4:52

















              thanks to both answers. i felt i should make something as a good answer, although both give regexps, which i wanted to avoid. chose this one as it gives a reference for the format.

              – andrew cooke
              Jun 16 '11 at 12:49





              thanks to both answers. i felt i should make something as a good answer, although both give regexps, which i wanted to avoid. chose this one as it gives a reference for the format.

              – andrew cooke
              Jun 16 '11 at 12:49













              @andrew: My regexp is flexible enough that I expect it to work with pretty much any now-existing terminal, and probably with any tomorrow-existing terminal as well. I haven't tested it much, so there might be bugs, but the approach is sound as control sequences follow a few general patterns.

              – Gilles
              Jun 16 '11 at 13:53







              @andrew: My regexp is flexible enough that I expect it to work with pretty much any now-existing terminal, and probably with any tomorrow-existing terminal as well. I haven't tested it much, so there might be bugs, but the approach is sound as control sequences follow a few general patterns.

              – Gilles
              Jun 16 '11 at 13:53















              please provide how to use this script. does it require pipe input? or positional arguments?

              – Trevor Boyd Smith
              Jan 5 '18 at 17:14





              please provide how to use this script. does it require pipe input? or positional arguments?

              – Trevor Boyd Smith
              Jan 5 '18 at 17:14













              @TrevorBoydSmith Either will work for input, and the output is always on standard output, like typical text utilities.

              – Gilles
              Jan 5 '18 at 20:31







              @TrevorBoydSmith Either will work for input, and the output is always on standard output, like typical text utilities.

              – Gilles
              Jan 5 '18 at 20:31















              This mangles multibyte characters such as ☺ (xe2 x98 xba). The [x80-x9f] clause strips the middle byte.

              – Jeffrey
              Jun 9 '18 at 4:52





              This mangles multibyte characters such as ☺ (xe2 x98 xba). The [x80-x9f] clause strips the middle byte.

              – Jeffrey
              Jun 9 '18 at 4:52













              29














              Updating Gilles' answer to also remove carriage returns and do backspace-erasing of previous characters, which were both important to me for a typescript generated on Cygwin:



              #!/usr/bin/perl
              while (<>) {
              s/ e[ #%()*+-./]. |
              r | # Remove extra carriage returns also
              (?:e[|x9b) [ -?]* [@-~] | # CSI ... Cmd
              (?:e]|x9d) .*? (?:e\|[ax9c]) | # OSC ... (ST|BEL)
              (?:e[P^_]|[x90x9ex9f]) .*? (?:e\|x9c) | # (DCS|PM|APC) ... ST
              e.|[x80-x9f] //xg;
              1 while s/[^b][b]//g; # remove all non-backspace followed by backspace
              print;
              }





              share|improve this answer


























              • +1 I was already typing a post whith the same question as the OP when I fond this message with your script and that of @Gilles. +1 for both of you

                – miracle173
                Apr 29 '12 at 22:02
















              29














              Updating Gilles' answer to also remove carriage returns and do backspace-erasing of previous characters, which were both important to me for a typescript generated on Cygwin:



              #!/usr/bin/perl
              while (<>) {
              s/ e[ #%()*+-./]. |
              r | # Remove extra carriage returns also
              (?:e[|x9b) [ -?]* [@-~] | # CSI ... Cmd
              (?:e]|x9d) .*? (?:e\|[ax9c]) | # OSC ... (ST|BEL)
              (?:e[P^_]|[x90x9ex9f]) .*? (?:e\|x9c) | # (DCS|PM|APC) ... ST
              e.|[x80-x9f] //xg;
              1 while s/[^b][b]//g; # remove all non-backspace followed by backspace
              print;
              }





              share|improve this answer


























              • +1 I was already typing a post whith the same question as the OP when I fond this message with your script and that of @Gilles. +1 for both of you

                – miracle173
                Apr 29 '12 at 22:02














              29












              29








              29







              Updating Gilles' answer to also remove carriage returns and do backspace-erasing of previous characters, which were both important to me for a typescript generated on Cygwin:



              #!/usr/bin/perl
              while (<>) {
              s/ e[ #%()*+-./]. |
              r | # Remove extra carriage returns also
              (?:e[|x9b) [ -?]* [@-~] | # CSI ... Cmd
              (?:e]|x9d) .*? (?:e\|[ax9c]) | # OSC ... (ST|BEL)
              (?:e[P^_]|[x90x9ex9f]) .*? (?:e\|x9c) | # (DCS|PM|APC) ... ST
              e.|[x80-x9f] //xg;
              1 while s/[^b][b]//g; # remove all non-backspace followed by backspace
              print;
              }





              share|improve this answer















              Updating Gilles' answer to also remove carriage returns and do backspace-erasing of previous characters, which were both important to me for a typescript generated on Cygwin:



              #!/usr/bin/perl
              while (<>) {
              s/ e[ #%()*+-./]. |
              r | # Remove extra carriage returns also
              (?:e[|x9b) [ -?]* [@-~] | # CSI ... Cmd
              (?:e]|x9d) .*? (?:e\|[ax9c]) | # OSC ... (ST|BEL)
              (?:e[P^_]|[x90x9ex9f]) .*? (?:e\|x9c) | # (DCS|PM|APC) ... ST
              e.|[x80-x9f] //xg;
              1 while s/[^b][b]//g; # remove all non-backspace followed by backspace
              print;
              }






              share|improve this answer














              share|improve this answer



              share|improve this answer








              edited Aug 18 '11 at 22:20

























              answered Aug 18 '11 at 22:13









              dewtelldewtell

              39134




              39134













              • +1 I was already typing a post whith the same question as the OP when I fond this message with your script and that of @Gilles. +1 for both of you

                – miracle173
                Apr 29 '12 at 22:02



















              • +1 I was already typing a post whith the same question as the OP when I fond this message with your script and that of @Gilles. +1 for both of you

                – miracle173
                Apr 29 '12 at 22:02

















              +1 I was already typing a post whith the same question as the OP when I fond this message with your script and that of @Gilles. +1 for both of you

              – miracle173
              Apr 29 '12 at 22:02





              +1 I was already typing a post whith the same question as the OP when I fond this message with your script and that of @Gilles. +1 for both of you

              – miracle173
              Apr 29 '12 at 22:02











              8














              I would use sed in this case.



              do:



              cat -v typescript | sed -e "s/x1b[.{1,5}m//g"



              sed -e "s/search/replace/g" is standard stuff. the regex is explained as below:



              x1b match the Escape preceeding the color code
              [ matches the first open bracket
              .{1,5} matches 1 to 5 of any single character. Have to the curly braces to keep the shell from mangling them.
              m last character in regex - usually trails the color code.
              // empty string for what to replace everything with.
              g match it multiple times per line.






              share|improve this answer



















              • 2





                This regex strips too much (fooe[1m(1m = { becomes foo = { instead of foo(m = {), replacing . by [0-9;] is more accurate.

                – Lekensteyn
                Dec 30 '13 at 15:14











              • Replace .{1,5} with [^m]{1,5} for that - but also note that this even then still only removes "graphics rendition" codes (those that end in an m) - basically color, reverse, bold and italics styles (where applicable).

                – Hannu
                Aug 23 '15 at 14:26













              • This doesn't remove x1b(B (included in rusts color output)

                – ideasman42
                Nov 11 '16 at 13:11











              • Why is it x1b and not 33?

                – atripes
                Aug 24 '17 at 15:21
















              8














              I would use sed in this case.



              do:



              cat -v typescript | sed -e "s/x1b[.{1,5}m//g"



              sed -e "s/search/replace/g" is standard stuff. the regex is explained as below:



              x1b match the Escape preceeding the color code
              [ matches the first open bracket
              .{1,5} matches 1 to 5 of any single character. Have to the curly braces to keep the shell from mangling them.
              m last character in regex - usually trails the color code.
              // empty string for what to replace everything with.
              g match it multiple times per line.






              share|improve this answer



















              • 2





                This regex strips too much (fooe[1m(1m = { becomes foo = { instead of foo(m = {), replacing . by [0-9;] is more accurate.

                – Lekensteyn
                Dec 30 '13 at 15:14











              • Replace .{1,5} with [^m]{1,5} for that - but also note that this even then still only removes "graphics rendition" codes (those that end in an m) - basically color, reverse, bold and italics styles (where applicable).

                – Hannu
                Aug 23 '15 at 14:26













              • This doesn't remove x1b(B (included in rusts color output)

                – ideasman42
                Nov 11 '16 at 13:11











              • Why is it x1b and not 33?

                – atripes
                Aug 24 '17 at 15:21














              8












              8








              8







              I would use sed in this case.



              do:



              cat -v typescript | sed -e "s/x1b[.{1,5}m//g"



              sed -e "s/search/replace/g" is standard stuff. the regex is explained as below:



              x1b match the Escape preceeding the color code
              [ matches the first open bracket
              .{1,5} matches 1 to 5 of any single character. Have to the curly braces to keep the shell from mangling them.
              m last character in regex - usually trails the color code.
              // empty string for what to replace everything with.
              g match it multiple times per line.






              share|improve this answer













              I would use sed in this case.



              do:



              cat -v typescript | sed -e "s/x1b[.{1,5}m//g"



              sed -e "s/search/replace/g" is standard stuff. the regex is explained as below:



              x1b match the Escape preceeding the color code
              [ matches the first open bracket
              .{1,5} matches 1 to 5 of any single character. Have to the curly braces to keep the shell from mangling them.
              m last character in regex - usually trails the color code.
              // empty string for what to replace everything with.
              g match it multiple times per line.







              share|improve this answer












              share|improve this answer



              share|improve this answer










              answered Jun 9 '11 at 18:41









              GlorytoadGlorytoad

              56224




              56224








              • 2





                This regex strips too much (fooe[1m(1m = { becomes foo = { instead of foo(m = {), replacing . by [0-9;] is more accurate.

                – Lekensteyn
                Dec 30 '13 at 15:14











              • Replace .{1,5} with [^m]{1,5} for that - but also note that this even then still only removes "graphics rendition" codes (those that end in an m) - basically color, reverse, bold and italics styles (where applicable).

                – Hannu
                Aug 23 '15 at 14:26













              • This doesn't remove x1b(B (included in rusts color output)

                – ideasman42
                Nov 11 '16 at 13:11











              • Why is it x1b and not 33?

                – atripes
                Aug 24 '17 at 15:21














              • 2





                This regex strips too much (fooe[1m(1m = { becomes foo = { instead of foo(m = {), replacing . by [0-9;] is more accurate.

                – Lekensteyn
                Dec 30 '13 at 15:14











              • Replace .{1,5} with [^m]{1,5} for that - but also note that this even then still only removes "graphics rendition" codes (those that end in an m) - basically color, reverse, bold and italics styles (where applicable).

                – Hannu
                Aug 23 '15 at 14:26













              • This doesn't remove x1b(B (included in rusts color output)

                – ideasman42
                Nov 11 '16 at 13:11











              • Why is it x1b and not 33?

                – atripes
                Aug 24 '17 at 15:21








              2




              2





              This regex strips too much (fooe[1m(1m = { becomes foo = { instead of foo(m = {), replacing . by [0-9;] is more accurate.

              – Lekensteyn
              Dec 30 '13 at 15:14





              This regex strips too much (fooe[1m(1m = { becomes foo = { instead of foo(m = {), replacing . by [0-9;] is more accurate.

              – Lekensteyn
              Dec 30 '13 at 15:14













              Replace .{1,5} with [^m]{1,5} for that - but also note that this even then still only removes "graphics rendition" codes (those that end in an m) - basically color, reverse, bold and italics styles (where applicable).

              – Hannu
              Aug 23 '15 at 14:26







              Replace .{1,5} with [^m]{1,5} for that - but also note that this even then still only removes "graphics rendition" codes (those that end in an m) - basically color, reverse, bold and italics styles (where applicable).

              – Hannu
              Aug 23 '15 at 14:26















              This doesn't remove x1b(B (included in rusts color output)

              – ideasman42
              Nov 11 '16 at 13:11





              This doesn't remove x1b(B (included in rusts color output)

              – ideasman42
              Nov 11 '16 at 13:11













              Why is it x1b and not 33?

              – atripes
              Aug 24 '17 at 15:21





              Why is it x1b and not 33?

              – atripes
              Aug 24 '17 at 15:21











              7














              cat typescript | perl -pe 's/e([^[]]|[.*?[a-zA-Z]|].*?a)//g' | col -b > typescript-processed





              share|improve this answer
























              • The only one that actually worked, thanks!

                – Will
                Sep 29 '16 at 15:23
















              7














              cat typescript | perl -pe 's/e([^[]]|[.*?[a-zA-Z]|].*?a)//g' | col -b > typescript-processed





              share|improve this answer
























              • The only one that actually worked, thanks!

                – Will
                Sep 29 '16 at 15:23














              7












              7








              7







              cat typescript | perl -pe 's/e([^[]]|[.*?[a-zA-Z]|].*?a)//g' | col -b > typescript-processed





              share|improve this answer













              cat typescript | perl -pe 's/e([^[]]|[.*?[a-zA-Z]|].*?a)//g' | col -b > typescript-processed






              share|improve this answer












              share|improve this answer



              share|improve this answer










              answered Sep 27 '13 at 0:44









              Peter NorePeter Nore

              17113




              17113













              • The only one that actually worked, thanks!

                – Will
                Sep 29 '16 at 15:23



















              • The only one that actually worked, thanks!

                – Will
                Sep 29 '16 at 15:23

















              The only one that actually worked, thanks!

              – Will
              Sep 29 '16 at 15:23





              The only one that actually worked, thanks!

              – Will
              Sep 29 '16 at 15:23











              5














              # The "sed -r" trick does not work on every Linux, I still dunno why:
              DECOLORIZE='eval sed "s,${END}[[0-9;]*[m|K],,g"'


              => howto use:



              <commands that type colored output> | ${DECOLORIZE}


              tested on:
              - AIX 5.x / 6.1 / 7.1
              - Linux Mandrake / Mandriva / SLES / Fedora
              - SunOS






              share|improve this answer






























                5














                # The "sed -r" trick does not work on every Linux, I still dunno why:
                DECOLORIZE='eval sed "s,${END}[[0-9;]*[m|K],,g"'


                => howto use:



                <commands that type colored output> | ${DECOLORIZE}


                tested on:
                - AIX 5.x / 6.1 / 7.1
                - Linux Mandrake / Mandriva / SLES / Fedora
                - SunOS






                share|improve this answer




























                  5












                  5








                  5







                  # The "sed -r" trick does not work on every Linux, I still dunno why:
                  DECOLORIZE='eval sed "s,${END}[[0-9;]*[m|K],,g"'


                  => howto use:



                  <commands that type colored output> | ${DECOLORIZE}


                  tested on:
                  - AIX 5.x / 6.1 / 7.1
                  - Linux Mandrake / Mandriva / SLES / Fedora
                  - SunOS






                  share|improve this answer















                  # The "sed -r" trick does not work on every Linux, I still dunno why:
                  DECOLORIZE='eval sed "s,${END}[[0-9;]*[m|K],,g"'


                  => howto use:



                  <commands that type colored output> | ${DECOLORIZE}


                  tested on:
                  - AIX 5.x / 6.1 / 7.1
                  - Linux Mandrake / Mandriva / SLES / Fedora
                  - SunOS







                  share|improve this answer














                  share|improve this answer



                  share|improve this answer








                  edited Sep 27 '14 at 10:24

























                  answered Sep 24 '14 at 15:28









                  scavengerscavenger

                  6114




                  6114























                      2














                      I solved the problem by running scriptreplay in a screen and the dumping the scrollback buffer to a file.



                      The following expect script does this for you.



                      It has been tested for logfiles with up to 250.000 lines. In the working directory you need your scriptlog, a file called "time" with 10.000.000 times the line "1 10" in it, and the script. I needs the name of your scriptfile as command line argument, like ./name_of_script name_of_scriptlog.



                      #!/usr/bin/expect -f 

                      set logfile [lindex $argv 0]

                      if {$logfile == ""} {puts "Usage: ./script_to_readable.exp $logfile."; exit}

                      set timestamp [clock format [clock sec] -format %Y-%m-%d,%H:%M:%S]
                      set pwd [exec pwd]
                      if {! [file exists ${pwd}/time]} {puts "ERROR: time file not found.nYou need a file named time with 10.000.000 times the line "1 10" in the working directory for this script to work. Please provide it."; exit}
                      set wc [exec cat ${pwd}/$logfile | wc -l]
                      set height [ expr "$wc" + "100" ]
                      system cp $logfile ${logfile}.tmp
                      system echo $timestamp >> ${logfile}.tmp
                      set timeout -1
                      spawn screen -h $height -S $timestamp
                      send "scriptreplay -t time -s ${logfile}.tmp 100000 2>/dev/nullr"
                      expect ${timestamp}
                      send "x01:hardcopy -h readablelog.${timestamp}r"

                      send "exitr"

                      system sed '/^$/d' readablelog.$timestamp >> readablelog2.$timestamp
                      system head -n-2 readablelog2.$timestamp >> ${logfile}.readable.$timestamp
                      system rm -f readablelog.$timestamp readablelog2.$timestamp ${logfile}.tmp


                      The time file can be generated by



                      for i in $(seq 1 10000000); do echo "1 10" >> time; done





                      share|improve this answer


























                      • The command for generating time file generated 100% CPU usage for a few minutes and after it finished my memory usage was 100% and running command resulted in "fork: cannot allocate memory". And it didn't really work as expected.

                        – Barteks2x
                        May 27 '16 at 17:29











                      • There's a far easier way to generate the timing file. The fields are "delay blocksize", so there's no reason not to just make it "0 <entirefile>" and dump the whole thing with no delay. You can do that by taking the size of the script minus the first line (tail -n +2 typescript|wc -c), and create the timing file with echo "0 "`tail -n +2 typescript|wc -c` > timing. That'll be basically instant, and scriptreplay will replay the entire script at fastest possible speed.

                        – FeRD
                        Jul 11 '18 at 5:27


















                      2














                      I solved the problem by running scriptreplay in a screen and the dumping the scrollback buffer to a file.



                      The following expect script does this for you.



                      It has been tested for logfiles with up to 250.000 lines. In the working directory you need your scriptlog, a file called "time" with 10.000.000 times the line "1 10" in it, and the script. I needs the name of your scriptfile as command line argument, like ./name_of_script name_of_scriptlog.



                      #!/usr/bin/expect -f 

                      set logfile [lindex $argv 0]

                      if {$logfile == ""} {puts "Usage: ./script_to_readable.exp $logfile."; exit}

                      set timestamp [clock format [clock sec] -format %Y-%m-%d,%H:%M:%S]
                      set pwd [exec pwd]
                      if {! [file exists ${pwd}/time]} {puts "ERROR: time file not found.nYou need a file named time with 10.000.000 times the line "1 10" in the working directory for this script to work. Please provide it."; exit}
                      set wc [exec cat ${pwd}/$logfile | wc -l]
                      set height [ expr "$wc" + "100" ]
                      system cp $logfile ${logfile}.tmp
                      system echo $timestamp >> ${logfile}.tmp
                      set timeout -1
                      spawn screen -h $height -S $timestamp
                      send "scriptreplay -t time -s ${logfile}.tmp 100000 2>/dev/nullr"
                      expect ${timestamp}
                      send "x01:hardcopy -h readablelog.${timestamp}r"

                      send "exitr"

                      system sed '/^$/d' readablelog.$timestamp >> readablelog2.$timestamp
                      system head -n-2 readablelog2.$timestamp >> ${logfile}.readable.$timestamp
                      system rm -f readablelog.$timestamp readablelog2.$timestamp ${logfile}.tmp


                      The time file can be generated by



                      for i in $(seq 1 10000000); do echo "1 10" >> time; done





                      share|improve this answer


























                      • The command for generating time file generated 100% CPU usage for a few minutes and after it finished my memory usage was 100% and running command resulted in "fork: cannot allocate memory". And it didn't really work as expected.

                        – Barteks2x
                        May 27 '16 at 17:29











                      • There's a far easier way to generate the timing file. The fields are "delay blocksize", so there's no reason not to just make it "0 <entirefile>" and dump the whole thing with no delay. You can do that by taking the size of the script minus the first line (tail -n +2 typescript|wc -c), and create the timing file with echo "0 "`tail -n +2 typescript|wc -c` > timing. That'll be basically instant, and scriptreplay will replay the entire script at fastest possible speed.

                        – FeRD
                        Jul 11 '18 at 5:27
















                      2












                      2








                      2







                      I solved the problem by running scriptreplay in a screen and the dumping the scrollback buffer to a file.



                      The following expect script does this for you.



                      It has been tested for logfiles with up to 250.000 lines. In the working directory you need your scriptlog, a file called "time" with 10.000.000 times the line "1 10" in it, and the script. I needs the name of your scriptfile as command line argument, like ./name_of_script name_of_scriptlog.



                      #!/usr/bin/expect -f 

                      set logfile [lindex $argv 0]

                      if {$logfile == ""} {puts "Usage: ./script_to_readable.exp $logfile."; exit}

                      set timestamp [clock format [clock sec] -format %Y-%m-%d,%H:%M:%S]
                      set pwd [exec pwd]
                      if {! [file exists ${pwd}/time]} {puts "ERROR: time file not found.nYou need a file named time with 10.000.000 times the line "1 10" in the working directory for this script to work. Please provide it."; exit}
                      set wc [exec cat ${pwd}/$logfile | wc -l]
                      set height [ expr "$wc" + "100" ]
                      system cp $logfile ${logfile}.tmp
                      system echo $timestamp >> ${logfile}.tmp
                      set timeout -1
                      spawn screen -h $height -S $timestamp
                      send "scriptreplay -t time -s ${logfile}.tmp 100000 2>/dev/nullr"
                      expect ${timestamp}
                      send "x01:hardcopy -h readablelog.${timestamp}r"

                      send "exitr"

                      system sed '/^$/d' readablelog.$timestamp >> readablelog2.$timestamp
                      system head -n-2 readablelog2.$timestamp >> ${logfile}.readable.$timestamp
                      system rm -f readablelog.$timestamp readablelog2.$timestamp ${logfile}.tmp


                      The time file can be generated by



                      for i in $(seq 1 10000000); do echo "1 10" >> time; done





                      share|improve this answer















                      I solved the problem by running scriptreplay in a screen and the dumping the scrollback buffer to a file.



                      The following expect script does this for you.



                      It has been tested for logfiles with up to 250.000 lines. In the working directory you need your scriptlog, a file called "time" with 10.000.000 times the line "1 10" in it, and the script. I needs the name of your scriptfile as command line argument, like ./name_of_script name_of_scriptlog.



                      #!/usr/bin/expect -f 

                      set logfile [lindex $argv 0]

                      if {$logfile == ""} {puts "Usage: ./script_to_readable.exp $logfile."; exit}

                      set timestamp [clock format [clock sec] -format %Y-%m-%d,%H:%M:%S]
                      set pwd [exec pwd]
                      if {! [file exists ${pwd}/time]} {puts "ERROR: time file not found.nYou need a file named time with 10.000.000 times the line "1 10" in the working directory for this script to work. Please provide it."; exit}
                      set wc [exec cat ${pwd}/$logfile | wc -l]
                      set height [ expr "$wc" + "100" ]
                      system cp $logfile ${logfile}.tmp
                      system echo $timestamp >> ${logfile}.tmp
                      set timeout -1
                      spawn screen -h $height -S $timestamp
                      send "scriptreplay -t time -s ${logfile}.tmp 100000 2>/dev/nullr"
                      expect ${timestamp}
                      send "x01:hardcopy -h readablelog.${timestamp}r"

                      send "exitr"

                      system sed '/^$/d' readablelog.$timestamp >> readablelog2.$timestamp
                      system head -n-2 readablelog2.$timestamp >> ${logfile}.readable.$timestamp
                      system rm -f readablelog.$timestamp readablelog2.$timestamp ${logfile}.tmp


                      The time file can be generated by



                      for i in $(seq 1 10000000); do echo "1 10" >> time; done






                      share|improve this answer














                      share|improve this answer



                      share|improve this answer








                      edited Jan 22 '15 at 15:40









                      Michael Mrozek

                      61.5k29192211




                      61.5k29192211










                      answered Jan 22 '15 at 12:37









                      hnkchnskhnkchnsk

                      211




                      211













                      • The command for generating time file generated 100% CPU usage for a few minutes and after it finished my memory usage was 100% and running command resulted in "fork: cannot allocate memory". And it didn't really work as expected.

                        – Barteks2x
                        May 27 '16 at 17:29











                      • There's a far easier way to generate the timing file. The fields are "delay blocksize", so there's no reason not to just make it "0 <entirefile>" and dump the whole thing with no delay. You can do that by taking the size of the script minus the first line (tail -n +2 typescript|wc -c), and create the timing file with echo "0 "`tail -n +2 typescript|wc -c` > timing. That'll be basically instant, and scriptreplay will replay the entire script at fastest possible speed.

                        – FeRD
                        Jul 11 '18 at 5:27





















                      • The command for generating time file generated 100% CPU usage for a few minutes and after it finished my memory usage was 100% and running command resulted in "fork: cannot allocate memory". And it didn't really work as expected.

                        – Barteks2x
                        May 27 '16 at 17:29











                      • There's a far easier way to generate the timing file. The fields are "delay blocksize", so there's no reason not to just make it "0 <entirefile>" and dump the whole thing with no delay. You can do that by taking the size of the script minus the first line (tail -n +2 typescript|wc -c), and create the timing file with echo "0 "`tail -n +2 typescript|wc -c` > timing. That'll be basically instant, and scriptreplay will replay the entire script at fastest possible speed.

                        – FeRD
                        Jul 11 '18 at 5:27



















                      The command for generating time file generated 100% CPU usage for a few minutes and after it finished my memory usage was 100% and running command resulted in "fork: cannot allocate memory". And it didn't really work as expected.

                      – Barteks2x
                      May 27 '16 at 17:29





                      The command for generating time file generated 100% CPU usage for a few minutes and after it finished my memory usage was 100% and running command resulted in "fork: cannot allocate memory". And it didn't really work as expected.

                      – Barteks2x
                      May 27 '16 at 17:29













                      There's a far easier way to generate the timing file. The fields are "delay blocksize", so there's no reason not to just make it "0 <entirefile>" and dump the whole thing with no delay. You can do that by taking the size of the script minus the first line (tail -n +2 typescript|wc -c), and create the timing file with echo "0 "`tail -n +2 typescript|wc -c` > timing. That'll be basically instant, and scriptreplay will replay the entire script at fastest possible speed.

                      – FeRD
                      Jul 11 '18 at 5:27







                      There's a far easier way to generate the timing file. The fields are "delay blocksize", so there's no reason not to just make it "0 <entirefile>" and dump the whole thing with no delay. You can do that by taking the size of the script minus the first line (tail -n +2 typescript|wc -c), and create the timing file with echo "0 "`tail -n +2 typescript|wc -c` > timing. That'll be basically instant, and scriptreplay will replay the entire script at fastest possible speed.

                      – FeRD
                      Jul 11 '18 at 5:27













                      1














                      Found this question while looking for the a solution to the same problem. A little more digging and found this script over at Live Journal at this link. I worked perfectly for me. It's also a very good write up about this problem and how the solution works. Definitely worth a read.
                      http://jdimpson.livejournal.com/7040.html



                      #!/usr/bin/perl -wp

                      # clean up control characters and other non-text detritus that shows up
                      # when you run the "script" command.

                      BEGIN {
                      # xterm titlebar escape sequence
                      $xtermesc = "x1bx5dx30x3b";

                      # the occurence of a backspace event (e.g. cntrl H, cntrol W, or cntrl U)
                      $backspaceevent = "x1b\x5bx4b"; # note escaping of third character

                      # ANSI color escape sequence
                      $ansiesc = qr/x1b[[d;]*?m/;

                      # technically, this is arrow-right. For some reason, being used against
                      # very long backspace jobs. I don't fully understand this, as evidenced
                      # by the fact that is off by one sometimes.
                      $bizarrebs = qr/x1b[C/;

                      # used as part of the xterm titlebar mechanism, or when
                      # a bell sounds, which might happen when you backspace too much.
                      $bell = "x07"; # could use a

                      $cr = "x0d"; # could use r

                      $backspace = "x08"; # could use b
                      }

                      s/$xtermesc.+?$bell//g;
                      s/[$cr$bell]//g;
                      s/${backspaceevent}//g;
                      s/$ansiesc//g;
                      while (s/(.)(?=$backspace)//) { s/$backspace//; } # frickin' sweet
                      # For every ^H delete the character immediately left of it, then delete the ^H.
                      # Perl's RE's aren't R, so I wonder if I could do this in one expression.
                      while (s/(..)(?=$bizarrebs)//) { s/$bizarrebs//; }





                      share|improve this answer




























                        1














                        Found this question while looking for the a solution to the same problem. A little more digging and found this script over at Live Journal at this link. I worked perfectly for me. It's also a very good write up about this problem and how the solution works. Definitely worth a read.
                        http://jdimpson.livejournal.com/7040.html



                        #!/usr/bin/perl -wp

                        # clean up control characters and other non-text detritus that shows up
                        # when you run the "script" command.

                        BEGIN {
                        # xterm titlebar escape sequence
                        $xtermesc = "x1bx5dx30x3b";

                        # the occurence of a backspace event (e.g. cntrl H, cntrol W, or cntrl U)
                        $backspaceevent = "x1b\x5bx4b"; # note escaping of third character

                        # ANSI color escape sequence
                        $ansiesc = qr/x1b[[d;]*?m/;

                        # technically, this is arrow-right. For some reason, being used against
                        # very long backspace jobs. I don't fully understand this, as evidenced
                        # by the fact that is off by one sometimes.
                        $bizarrebs = qr/x1b[C/;

                        # used as part of the xterm titlebar mechanism, or when
                        # a bell sounds, which might happen when you backspace too much.
                        $bell = "x07"; # could use a

                        $cr = "x0d"; # could use r

                        $backspace = "x08"; # could use b
                        }

                        s/$xtermesc.+?$bell//g;
                        s/[$cr$bell]//g;
                        s/${backspaceevent}//g;
                        s/$ansiesc//g;
                        while (s/(.)(?=$backspace)//) { s/$backspace//; } # frickin' sweet
                        # For every ^H delete the character immediately left of it, then delete the ^H.
                        # Perl's RE's aren't R, so I wonder if I could do this in one expression.
                        while (s/(..)(?=$bizarrebs)//) { s/$bizarrebs//; }





                        share|improve this answer


























                          1












                          1








                          1







                          Found this question while looking for the a solution to the same problem. A little more digging and found this script over at Live Journal at this link. I worked perfectly for me. It's also a very good write up about this problem and how the solution works. Definitely worth a read.
                          http://jdimpson.livejournal.com/7040.html



                          #!/usr/bin/perl -wp

                          # clean up control characters and other non-text detritus that shows up
                          # when you run the "script" command.

                          BEGIN {
                          # xterm titlebar escape sequence
                          $xtermesc = "x1bx5dx30x3b";

                          # the occurence of a backspace event (e.g. cntrl H, cntrol W, or cntrl U)
                          $backspaceevent = "x1b\x5bx4b"; # note escaping of third character

                          # ANSI color escape sequence
                          $ansiesc = qr/x1b[[d;]*?m/;

                          # technically, this is arrow-right. For some reason, being used against
                          # very long backspace jobs. I don't fully understand this, as evidenced
                          # by the fact that is off by one sometimes.
                          $bizarrebs = qr/x1b[C/;

                          # used as part of the xterm titlebar mechanism, or when
                          # a bell sounds, which might happen when you backspace too much.
                          $bell = "x07"; # could use a

                          $cr = "x0d"; # could use r

                          $backspace = "x08"; # could use b
                          }

                          s/$xtermesc.+?$bell//g;
                          s/[$cr$bell]//g;
                          s/${backspaceevent}//g;
                          s/$ansiesc//g;
                          while (s/(.)(?=$backspace)//) { s/$backspace//; } # frickin' sweet
                          # For every ^H delete the character immediately left of it, then delete the ^H.
                          # Perl's RE's aren't R, so I wonder if I could do this in one expression.
                          while (s/(..)(?=$bizarrebs)//) { s/$bizarrebs//; }





                          share|improve this answer













                          Found this question while looking for the a solution to the same problem. A little more digging and found this script over at Live Journal at this link. I worked perfectly for me. It's also a very good write up about this problem and how the solution works. Definitely worth a read.
                          http://jdimpson.livejournal.com/7040.html



                          #!/usr/bin/perl -wp

                          # clean up control characters and other non-text detritus that shows up
                          # when you run the "script" command.

                          BEGIN {
                          # xterm titlebar escape sequence
                          $xtermesc = "x1bx5dx30x3b";

                          # the occurence of a backspace event (e.g. cntrl H, cntrol W, or cntrl U)
                          $backspaceevent = "x1b\x5bx4b"; # note escaping of third character

                          # ANSI color escape sequence
                          $ansiesc = qr/x1b[[d;]*?m/;

                          # technically, this is arrow-right. For some reason, being used against
                          # very long backspace jobs. I don't fully understand this, as evidenced
                          # by the fact that is off by one sometimes.
                          $bizarrebs = qr/x1b[C/;

                          # used as part of the xterm titlebar mechanism, or when
                          # a bell sounds, which might happen when you backspace too much.
                          $bell = "x07"; # could use a

                          $cr = "x0d"; # could use r

                          $backspace = "x08"; # could use b
                          }

                          s/$xtermesc.+?$bell//g;
                          s/[$cr$bell]//g;
                          s/${backspaceevent}//g;
                          s/$ansiesc//g;
                          while (s/(.)(?=$backspace)//) { s/$backspace//; } # frickin' sweet
                          # For every ^H delete the character immediately left of it, then delete the ^H.
                          # Perl's RE's aren't R, so I wonder if I could do this in one expression.
                          while (s/(..)(?=$bizarrebs)//) { s/$bizarrebs//; }






                          share|improve this answer












                          share|improve this answer



                          share|improve this answer










                          answered Oct 18 '17 at 0:46









                          SammerVSammerV

                          111




                          111























                              0














                              I found that just using cat was all I needed to view the output of script in the terminal. This doesn't help when redirecting the output to another file, but does make the result readable, unlike cat -v, col -b, or a text editor.



                              To eliminate colors or save the results to a file, manually copy and paste the output from cat into a text editor, or into another cat command, i.e.:



                              cat > endResult << END
                              <paste_copied_text_here>
                              END





                              share|improve this answer





















                              • 1





                                did your script run include output with color codes attached, as in the OP's case?

                                – Jeff Schaller
                                Feb 7 at 15:34











                              • Using cat presents the original colors, which can be removed by manual copy-and-paste. The OP used cat -v and col -b, both of which present codes rather than a properly-formatted end result. I have edited my answer.

                                – Roger Dueck
                                Feb 7 at 15:53


















                              0














                              I found that just using cat was all I needed to view the output of script in the terminal. This doesn't help when redirecting the output to another file, but does make the result readable, unlike cat -v, col -b, or a text editor.



                              To eliminate colors or save the results to a file, manually copy and paste the output from cat into a text editor, or into another cat command, i.e.:



                              cat > endResult << END
                              <paste_copied_text_here>
                              END





                              share|improve this answer





















                              • 1





                                did your script run include output with color codes attached, as in the OP's case?

                                – Jeff Schaller
                                Feb 7 at 15:34











                              • Using cat presents the original colors, which can be removed by manual copy-and-paste. The OP used cat -v and col -b, both of which present codes rather than a properly-formatted end result. I have edited my answer.

                                – Roger Dueck
                                Feb 7 at 15:53
















                              0












                              0








                              0







                              I found that just using cat was all I needed to view the output of script in the terminal. This doesn't help when redirecting the output to another file, but does make the result readable, unlike cat -v, col -b, or a text editor.



                              To eliminate colors or save the results to a file, manually copy and paste the output from cat into a text editor, or into another cat command, i.e.:



                              cat > endResult << END
                              <paste_copied_text_here>
                              END





                              share|improve this answer















                              I found that just using cat was all I needed to view the output of script in the terminal. This doesn't help when redirecting the output to another file, but does make the result readable, unlike cat -v, col -b, or a text editor.



                              To eliminate colors or save the results to a file, manually copy and paste the output from cat into a text editor, or into another cat command, i.e.:



                              cat > endResult << END
                              <paste_copied_text_here>
                              END






                              share|improve this answer














                              share|improve this answer



                              share|improve this answer








                              edited Feb 7 at 15:56

























                              answered Feb 7 at 15:31









                              Roger DueckRoger Dueck

                              1033




                              1033








                              • 1





                                did your script run include output with color codes attached, as in the OP's case?

                                – Jeff Schaller
                                Feb 7 at 15:34











                              • Using cat presents the original colors, which can be removed by manual copy-and-paste. The OP used cat -v and col -b, both of which present codes rather than a properly-formatted end result. I have edited my answer.

                                – Roger Dueck
                                Feb 7 at 15:53
















                              • 1





                                did your script run include output with color codes attached, as in the OP's case?

                                – Jeff Schaller
                                Feb 7 at 15:34











                              • Using cat presents the original colors, which can be removed by manual copy-and-paste. The OP used cat -v and col -b, both of which present codes rather than a properly-formatted end result. I have edited my answer.

                                – Roger Dueck
                                Feb 7 at 15:53










                              1




                              1





                              did your script run include output with color codes attached, as in the OP's case?

                              – Jeff Schaller
                              Feb 7 at 15:34





                              did your script run include output with color codes attached, as in the OP's case?

                              – Jeff Schaller
                              Feb 7 at 15:34













                              Using cat presents the original colors, which can be removed by manual copy-and-paste. The OP used cat -v and col -b, both of which present codes rather than a properly-formatted end result. I have edited my answer.

                              – Roger Dueck
                              Feb 7 at 15:53







                              Using cat presents the original colors, which can be removed by manual copy-and-paste. The OP used cat -v and col -b, both of which present codes rather than a properly-formatted end result. I have edited my answer.

                              – Roger Dueck
                              Feb 7 at 15:53













                              -2














                              Following up on the last answer which uses tr and :cntrl: could we maybe do



                              sed "/^[[:cntrl:]]/d" output.txt



                              This seems to work for me because all lines generated by vi start with a control character. It happens to also strip out blank lines and lines that start with a tab, although that works for what I'm doing. Maybe there is a way to match any control character except for n m t.



                              Maybe we can search for the particular control character, and it looks like all junk lines generated by vi start with what looks like ^[. hexdump tells me the first character is 1b, so this seems to work too



                              sed "/^x1b/d" output.txt



                              This looks similar to an answer posted above, but it does not work properly because after running the command, some junk chars are already added to the command line as if the user had typed them.






                              share|improve this answer



















                              • 1





                                There is no "last answer" as the answers can and do change order. You should use the "share" button underneath the answer you want to reference, and include that as a link in your answer. Assuming your answer is sufficient to be more than a comment, of course. Right now I can't identify which of the several answers you're referencing.

                                – roaima
                                Jun 23 '17 at 21:22













                              • “could we maybe do …”  Yes, we could do that — but it would delete every line that begins with a control character.  On the output of, for example, ls --color (as shown in the question), your solution will delete almost every line that contains information.  Not good.  But thanks for leaving out the useless use of cat.    :-)    ⁠

                                – G-Man
                                Jun 23 '17 at 21:23













                              • Is there a way to create a character class that is :iscntrl: but not :isspace:? Maybe some syntax like ^[[:iscntrl:]-[:isspace]]

                                – snaran
                                Jun 23 '17 at 22:09


















                              -2














                              Following up on the last answer which uses tr and :cntrl: could we maybe do



                              sed "/^[[:cntrl:]]/d" output.txt



                              This seems to work for me because all lines generated by vi start with a control character. It happens to also strip out blank lines and lines that start with a tab, although that works for what I'm doing. Maybe there is a way to match any control character except for n m t.



                              Maybe we can search for the particular control character, and it looks like all junk lines generated by vi start with what looks like ^[. hexdump tells me the first character is 1b, so this seems to work too



                              sed "/^x1b/d" output.txt



                              This looks similar to an answer posted above, but it does not work properly because after running the command, some junk chars are already added to the command line as if the user had typed them.






                              share|improve this answer



















                              • 1





                                There is no "last answer" as the answers can and do change order. You should use the "share" button underneath the answer you want to reference, and include that as a link in your answer. Assuming your answer is sufficient to be more than a comment, of course. Right now I can't identify which of the several answers you're referencing.

                                – roaima
                                Jun 23 '17 at 21:22













                              • “could we maybe do …”  Yes, we could do that — but it would delete every line that begins with a control character.  On the output of, for example, ls --color (as shown in the question), your solution will delete almost every line that contains information.  Not good.  But thanks for leaving out the useless use of cat.    :-)    ⁠

                                – G-Man
                                Jun 23 '17 at 21:23













                              • Is there a way to create a character class that is :iscntrl: but not :isspace:? Maybe some syntax like ^[[:iscntrl:]-[:isspace]]

                                – snaran
                                Jun 23 '17 at 22:09
















                              -2












                              -2








                              -2







                              Following up on the last answer which uses tr and :cntrl: could we maybe do



                              sed "/^[[:cntrl:]]/d" output.txt



                              This seems to work for me because all lines generated by vi start with a control character. It happens to also strip out blank lines and lines that start with a tab, although that works for what I'm doing. Maybe there is a way to match any control character except for n m t.



                              Maybe we can search for the particular control character, and it looks like all junk lines generated by vi start with what looks like ^[. hexdump tells me the first character is 1b, so this seems to work too



                              sed "/^x1b/d" output.txt



                              This looks similar to an answer posted above, but it does not work properly because after running the command, some junk chars are already added to the command line as if the user had typed them.






                              share|improve this answer













                              Following up on the last answer which uses tr and :cntrl: could we maybe do



                              sed "/^[[:cntrl:]]/d" output.txt



                              This seems to work for me because all lines generated by vi start with a control character. It happens to also strip out blank lines and lines that start with a tab, although that works for what I'm doing. Maybe there is a way to match any control character except for n m t.



                              Maybe we can search for the particular control character, and it looks like all junk lines generated by vi start with what looks like ^[. hexdump tells me the first character is 1b, so this seems to work too



                              sed "/^x1b/d" output.txt



                              This looks similar to an answer posted above, but it does not work properly because after running the command, some junk chars are already added to the command line as if the user had typed them.







                              share|improve this answer












                              share|improve this answer



                              share|improve this answer










                              answered Jun 23 '17 at 21:01









                              snaransnaran

                              1




                              1








                              • 1





                                There is no "last answer" as the answers can and do change order. You should use the "share" button underneath the answer you want to reference, and include that as a link in your answer. Assuming your answer is sufficient to be more than a comment, of course. Right now I can't identify which of the several answers you're referencing.

                                – roaima
                                Jun 23 '17 at 21:22













                              • “could we maybe do …”  Yes, we could do that — but it would delete every line that begins with a control character.  On the output of, for example, ls --color (as shown in the question), your solution will delete almost every line that contains information.  Not good.  But thanks for leaving out the useless use of cat.    :-)    ⁠

                                – G-Man
                                Jun 23 '17 at 21:23













                              • Is there a way to create a character class that is :iscntrl: but not :isspace:? Maybe some syntax like ^[[:iscntrl:]-[:isspace]]

                                – snaran
                                Jun 23 '17 at 22:09
















                              • 1





                                There is no "last answer" as the answers can and do change order. You should use the "share" button underneath the answer you want to reference, and include that as a link in your answer. Assuming your answer is sufficient to be more than a comment, of course. Right now I can't identify which of the several answers you're referencing.

                                – roaima
                                Jun 23 '17 at 21:22













                              • “could we maybe do …”  Yes, we could do that — but it would delete every line that begins with a control character.  On the output of, for example, ls --color (as shown in the question), your solution will delete almost every line that contains information.  Not good.  But thanks for leaving out the useless use of cat.    :-)    ⁠

                                – G-Man
                                Jun 23 '17 at 21:23













                              • Is there a way to create a character class that is :iscntrl: but not :isspace:? Maybe some syntax like ^[[:iscntrl:]-[:isspace]]

                                – snaran
                                Jun 23 '17 at 22:09










                              1




                              1





                              There is no "last answer" as the answers can and do change order. You should use the "share" button underneath the answer you want to reference, and include that as a link in your answer. Assuming your answer is sufficient to be more than a comment, of course. Right now I can't identify which of the several answers you're referencing.

                              – roaima
                              Jun 23 '17 at 21:22







                              There is no "last answer" as the answers can and do change order. You should use the "share" button underneath the answer you want to reference, and include that as a link in your answer. Assuming your answer is sufficient to be more than a comment, of course. Right now I can't identify which of the several answers you're referencing.

                              – roaima
                              Jun 23 '17 at 21:22















                              “could we maybe do …”  Yes, we could do that — but it would delete every line that begins with a control character.  On the output of, for example, ls --color (as shown in the question), your solution will delete almost every line that contains information.  Not good.  But thanks for leaving out the useless use of cat.    :-)    ⁠

                              – G-Man
                              Jun 23 '17 at 21:23







                              “could we maybe do …”  Yes, we could do that — but it would delete every line that begins with a control character.  On the output of, for example, ls --color (as shown in the question), your solution will delete almost every line that contains information.  Not good.  But thanks for leaving out the useless use of cat.    :-)    ⁠

                              – G-Man
                              Jun 23 '17 at 21:23















                              Is there a way to create a character class that is :iscntrl: but not :isspace:? Maybe some syntax like ^[[:iscntrl:]-[:isspace]]

                              – snaran
                              Jun 23 '17 at 22:09







                              Is there a way to create a character class that is :iscntrl: but not :isspace:? Maybe some syntax like ^[[:iscntrl:]-[:isspace]]

                              – snaran
                              Jun 23 '17 at 22:09













                              -4














                              tr - translate or delete characters



                              cat typescript | tr -d [[:cntrl:]]





                              share|improve this answer


























                              • Welcome to Unix Stackexchange! When giving an answer it is preferable to give some explanation as to WHY your answer is the one.

                                – Stephen Rauch
                                Feb 19 '17 at 16:47











                              • Brevity is acceptable, but fuller explanations are better., also doesn't quite solve the issue here.

                                – Kusalananda
                                Feb 19 '17 at 17:27








                              • 3





                                This actually will not work correctly, as it will not remove a 01;34m for example, and will remove the end of line newline (n).

                                – sorontar
                                Feb 19 '17 at 17:41
















                              -4














                              tr - translate or delete characters



                              cat typescript | tr -d [[:cntrl:]]





                              share|improve this answer


























                              • Welcome to Unix Stackexchange! When giving an answer it is preferable to give some explanation as to WHY your answer is the one.

                                – Stephen Rauch
                                Feb 19 '17 at 16:47











                              • Brevity is acceptable, but fuller explanations are better., also doesn't quite solve the issue here.

                                – Kusalananda
                                Feb 19 '17 at 17:27








                              • 3





                                This actually will not work correctly, as it will not remove a 01;34m for example, and will remove the end of line newline (n).

                                – sorontar
                                Feb 19 '17 at 17:41














                              -4












                              -4








                              -4







                              tr - translate or delete characters



                              cat typescript | tr -d [[:cntrl:]]





                              share|improve this answer















                              tr - translate or delete characters



                              cat typescript | tr -d [[:cntrl:]]






                              share|improve this answer














                              share|improve this answer



                              share|improve this answer








                              edited Feb 19 '17 at 17:18









                              sorontar

                              4,458929




                              4,458929










                              answered Feb 19 '17 at 16:43









                              ChunkChunk

                              1




                              1













                              • Welcome to Unix Stackexchange! When giving an answer it is preferable to give some explanation as to WHY your answer is the one.

                                – Stephen Rauch
                                Feb 19 '17 at 16:47











                              • Brevity is acceptable, but fuller explanations are better., also doesn't quite solve the issue here.

                                – Kusalananda
                                Feb 19 '17 at 17:27








                              • 3





                                This actually will not work correctly, as it will not remove a 01;34m for example, and will remove the end of line newline (n).

                                – sorontar
                                Feb 19 '17 at 17:41



















                              • Welcome to Unix Stackexchange! When giving an answer it is preferable to give some explanation as to WHY your answer is the one.

                                – Stephen Rauch
                                Feb 19 '17 at 16:47











                              • Brevity is acceptable, but fuller explanations are better., also doesn't quite solve the issue here.

                                – Kusalananda
                                Feb 19 '17 at 17:27








                              • 3





                                This actually will not work correctly, as it will not remove a 01;34m for example, and will remove the end of line newline (n).

                                – sorontar
                                Feb 19 '17 at 17:41

















                              Welcome to Unix Stackexchange! When giving an answer it is preferable to give some explanation as to WHY your answer is the one.

                              – Stephen Rauch
                              Feb 19 '17 at 16:47





                              Welcome to Unix Stackexchange! When giving an answer it is preferable to give some explanation as to WHY your answer is the one.

                              – Stephen Rauch
                              Feb 19 '17 at 16:47













                              Brevity is acceptable, but fuller explanations are better., also doesn't quite solve the issue here.

                              – Kusalananda
                              Feb 19 '17 at 17:27







                              Brevity is acceptable, but fuller explanations are better., also doesn't quite solve the issue here.

                              – Kusalananda
                              Feb 19 '17 at 17:27






                              3




                              3





                              This actually will not work correctly, as it will not remove a 01;34m for example, and will remove the end of line newline (n).

                              – sorontar
                              Feb 19 '17 at 17:41





                              This actually will not work correctly, as it will not remove a 01;34m for example, and will remove the end of line newline (n).

                              – sorontar
                              Feb 19 '17 at 17:41


















                              draft saved

                              draft discarded




















































                              Thanks for contributing an answer to Unix & Linux Stack Exchange!


                              • Please be sure to answer the question. Provide details and share your research!

                              But avoid



                              • Asking for help, clarification, or responding to other answers.

                              • Making statements based on opinion; back them up with references or personal experience.


                              To learn more, see our tips on writing great answers.




                              draft saved


                              draft discarded














                              StackExchange.ready(
                              function () {
                              StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f14684%2fremoving-control-chars-including-console-codes-colours-from-script-output%23new-answer', 'question_page');
                              }
                              );

                              Post as a guest















                              Required, but never shown





















































                              Required, but never shown














                              Required, but never shown












                              Required, but never shown







                              Required, but never shown

































                              Required, but never shown














                              Required, but never shown












                              Required, but never shown







                              Required, but never shown







                              Popular posts from this blog

                              How to reconfigure Docker Trusted Registry 2.x.x to use CEPH FS mount instead of NFS and other traditional...

                              is 'sed' thread safe

                              How to make a Squid Proxy server?