I'd like to have a simple script that lets me pass it any text, and it pulls out a value for a key from the string.

I'd like this to be flexible, and accept XML or JSON input, and even poorly formed input, like text from a log.

e.g. given any of the following inputs, it should be able to pull out the value of the test key.

e.g.

$ echo "test:5 hi there" | extract_key_value test

should result in

5

Note that I don't care what it's written in, so node, ruby, etc is fine by me, but portability (Linux/osx) is nice ;-)

input1

this is test:5 i saw a value

input2

this is test:'another value' i saw a value

input3

this is test=5 i saw a value

input4

test='a string value here'

input5

my data

on line 2 test='a string value here'

more data

My quick crack at this is the following, which I feel can be improved greatly and feels like it should be solved somewhere!

extract_key_value

#!/usr/bin/env bash



function show_help()

{

  IT=$(cat <<EOF



  Helps you extract a key value from a string, typically a log msg



  usage: key {keyBeginDelim} {keyEndDelim}



  e.g. given "asd f asdf asdf test=easy asdf me=you" as input



  extract_key_value test        

  => returns easy

EOF

)

  echo "$IT"

  exit

}



if [ "$1" == "help" ]

then

  show_help

fi

if [ -z "$1" ]

then

  show_help

fi



INPUT=$(cat -)

KEY="$1"



function getVal()

{

  DELIM1="$1"

  DELIM2="$2"

  echo "$INPUT" | awk -F "$DELIM1" '{print $2}' | awk -F "$DELIM2" '{print $1}'

}



# Try whatever the user passed in or defaults for delims

if [ -n "$2" ]

then

  IT=$(getVal "$2" "$3")

fi



# Try other use cases

if [ -z "$IT" ]

then

  IT=$(getVal "$KEY:'" "'")

fi

if [ -z "$IT" ]

then

  IT=$(getVal "$KEY='" "'")

fi

if [ -z "$IT" ]

then

  IT=$(getVal "$KEY="" """)

fi

if [ -z "$IT" ]

then

  IT=$(getVal "$KEY:"" """)

fi

if [ -z "$IT" ]

then

  IT=$(getVal "$KEY:" " ")

fi

if [ -z "$IT" ]

then

  IT=$(getVal "$KEY=" " ")

fi

if [ -z "$IT" ]

then

  IT=$(getVal "$KEY=" ";")

fi

if [ -z "$IT" ]

then

  IT=$(getVal "$KEY:" ";")

fi



echo "$IT"

With pcregrep:

extract_key_value() {

  pcregrep -Mo1 "(?sx)

   (?:

       Q$1E      # key literally

     | "Q$1E"  # same in double quotes

     | 'Q$1E'    # same in single quotes

   )

   [=:]

   (?|  # branch reset

       '(.*?)'

     | "(.*?)"

     | ([^"'s]+)

   )"

}

• 
  -M: multiline match (to allow test:'foonbar'...)


• 
  -o1: output the text matched by the first capture group (see below about branch reset).


• 
  (?sx): enable the s flag (make . match newline characters as well) and x flag (allow that multiline with comment format)


• 
  Q$1E the content of $1 (the first argument to the function) is to be taken literally. That assumes it doesn't contain E itself. In ksh93-like shells like bash, you can replace $1 with ${1//\E/\E\\E\Q} to work around that.


• 
  (?|.(.).|.(.).) branch reset. The numbering of the capture group starts from 1 after each |, so -o1 will return the first capture group that matches in any of the alternation.


• 
  '.*?'. .*? is the non-greedy variant of .*, so '.*' will match from ' to the first ' after that.


• 
  s: any whitespace character.

That doesn't try to address corner cases like the x encodings in json, the embedding of quotes within quotes (which is done differently depending on the language). It doesn't allow whitespace on either side of the : or =. All those can be addressed if need be. That will depend on the type of exact input you're trying to process.

A example with grep:

function extract_key_value() {

    egrep -o "$1[:=]['"[:alnum:]]+" | egrep -o "['"[:alnum:]]+$" | egrep -o "[[:alnum:]]+"

}

echo -e "on line 1ntest:123 asasasnon line 3ntest='abc'non line 5" | extract_key_value test