Use Sed Regex Capture Group in Replace Section Method
I have a list of timestamped filenames in this following format:
cat files
- ..201807010112.log
- ..201807020112.log
- ..201807022359.log
- ..201807030112.log
- ..201807010412.log
I need to get a certain range and in order to do this, I'd like to use SED.
sed -n '/201807010112/,/201807030112/p'
Was my first attempt but sed treats the hour and minute strangely, and to make a long story short it's unable to understand this format correctly.
I may be getting in the weeds but in order to solve this, I've decided to convert the format to something SED understands.
cat files | sed -e "s/([0-9]{12}}/$(date -f '%Y%m%d%H%M' 1)/g"
My problem is that I cannot use the result of the match 1 in the date conversion block.
Is there a way to do this or better yet a better way to get the range of dates?
linux awk sed grep regular-expression
|
show 1 more comment
I have a list of timestamped filenames in this following format:
cat files
- ..201807010112.log
- ..201807020112.log
- ..201807022359.log
- ..201807030112.log
- ..201807010412.log
I need to get a certain range and in order to do this, I'd like to use SED.
sed -n '/201807010112/,/201807030112/p'
Was my first attempt but sed treats the hour and minute strangely, and to make a long story short it's unable to understand this format correctly.
I may be getting in the weeds but in order to solve this, I've decided to convert the format to something SED understands.
cat files | sed -e "s/([0-9]{12}}/$(date -f '%Y%m%d%H%M' 1)/g"
My problem is that I cannot use the result of the match 1 in the date conversion block.
Is there a way to do this or better yet a better way to get the range of dates?
linux awk sed grep regular-expression
3
This is a very good example of a XY question...
– don_crissti
Aug 6 '18 at 22:23
cat 2018070{[1-2],301}
– user1133275
Aug 6 '18 at 22:49
1
like... sycamore?
– mikeserv
Aug 7 '18 at 0:52
1
Toxic comments, even I mentioned there could be a better way. User1133275, thanks for the suggestion but I need more flexibility when specifying the range. I'll answer my own question for the benefit of the community.
– Proximo
Aug 7 '18 at 1:47
Where is yourfiles
contents coming from? There might be a better way to get the filenames than from that file, e.g. usingfind
against the actual files in the filesystem (and their timestamps). That is what @don_crissti meant by XY problem. You are may be focusing on an issue which is down a path that you don't need to take while solving another problem.
– Kusalananda
Aug 7 '18 at 6:14
|
show 1 more comment
I have a list of timestamped filenames in this following format:
cat files
- ..201807010112.log
- ..201807020112.log
- ..201807022359.log
- ..201807030112.log
- ..201807010412.log
I need to get a certain range and in order to do this, I'd like to use SED.
sed -n '/201807010112/,/201807030112/p'
Was my first attempt but sed treats the hour and minute strangely, and to make a long story short it's unable to understand this format correctly.
I may be getting in the weeds but in order to solve this, I've decided to convert the format to something SED understands.
cat files | sed -e "s/([0-9]{12}}/$(date -f '%Y%m%d%H%M' 1)/g"
My problem is that I cannot use the result of the match 1 in the date conversion block.
Is there a way to do this or better yet a better way to get the range of dates?
linux awk sed grep regular-expression
I have a list of timestamped filenames in this following format:
cat files
- ..201807010112.log
- ..201807020112.log
- ..201807022359.log
- ..201807030112.log
- ..201807010412.log
I need to get a certain range and in order to do this, I'd like to use SED.
sed -n '/201807010112/,/201807030112/p'
Was my first attempt but sed treats the hour and minute strangely, and to make a long story short it's unable to understand this format correctly.
I may be getting in the weeds but in order to solve this, I've decided to convert the format to something SED understands.
cat files | sed -e "s/([0-9]{12}}/$(date -f '%Y%m%d%H%M' 1)/g"
My problem is that I cannot use the result of the match 1 in the date conversion block.
Is there a way to do this or better yet a better way to get the range of dates?
linux awk sed grep regular-expression
linux awk sed grep regular-expression
edited Aug 7 '18 at 1:23
Proximo
asked Aug 6 '18 at 21:55
ProximoProximo
1305
1305
3
This is a very good example of a XY question...
– don_crissti
Aug 6 '18 at 22:23
cat 2018070{[1-2],301}
– user1133275
Aug 6 '18 at 22:49
1
like... sycamore?
– mikeserv
Aug 7 '18 at 0:52
1
Toxic comments, even I mentioned there could be a better way. User1133275, thanks for the suggestion but I need more flexibility when specifying the range. I'll answer my own question for the benefit of the community.
– Proximo
Aug 7 '18 at 1:47
Where is yourfiles
contents coming from? There might be a better way to get the filenames than from that file, e.g. usingfind
against the actual files in the filesystem (and their timestamps). That is what @don_crissti meant by XY problem. You are may be focusing on an issue which is down a path that you don't need to take while solving another problem.
– Kusalananda
Aug 7 '18 at 6:14
|
show 1 more comment
3
This is a very good example of a XY question...
– don_crissti
Aug 6 '18 at 22:23
cat 2018070{[1-2],301}
– user1133275
Aug 6 '18 at 22:49
1
like... sycamore?
– mikeserv
Aug 7 '18 at 0:52
1
Toxic comments, even I mentioned there could be a better way. User1133275, thanks for the suggestion but I need more flexibility when specifying the range. I'll answer my own question for the benefit of the community.
– Proximo
Aug 7 '18 at 1:47
Where is yourfiles
contents coming from? There might be a better way to get the filenames than from that file, e.g. usingfind
against the actual files in the filesystem (and their timestamps). That is what @don_crissti meant by XY problem. You are may be focusing on an issue which is down a path that you don't need to take while solving another problem.
– Kusalananda
Aug 7 '18 at 6:14
3
3
This is a very good example of a XY question...
– don_crissti
Aug 6 '18 at 22:23
This is a very good example of a XY question...
– don_crissti
Aug 6 '18 at 22:23
cat 2018070{[1-2],301}
– user1133275
Aug 6 '18 at 22:49
cat 2018070{[1-2],301}
– user1133275
Aug 6 '18 at 22:49
1
1
like... sycamore?
– mikeserv
Aug 7 '18 at 0:52
like... sycamore?
– mikeserv
Aug 7 '18 at 0:52
1
1
Toxic comments, even I mentioned there could be a better way. User1133275, thanks for the suggestion but I need more flexibility when specifying the range. I'll answer my own question for the benefit of the community.
– Proximo
Aug 7 '18 at 1:47
Toxic comments, even I mentioned there could be a better way. User1133275, thanks for the suggestion but I need more flexibility when specifying the range. I'll answer my own question for the benefit of the community.
– Proximo
Aug 7 '18 at 1:47
Where is your
files
contents coming from? There might be a better way to get the filenames than from that file, e.g. using find
against the actual files in the filesystem (and their timestamps). That is what @don_crissti meant by XY problem. You are may be focusing on an issue which is down a path that you don't need to take while solving another problem.– Kusalananda
Aug 7 '18 at 6:14
Where is your
files
contents coming from? There might be a better way to get the filenames than from that file, e.g. using find
against the actual files in the filesystem (and their timestamps). That is what @don_crissti meant by XY problem. You are may be focusing on an issue which is down a path that you don't need to take while solving another problem.– Kusalananda
Aug 7 '18 at 6:14
|
show 1 more comment
2 Answers
2
active
oldest
votes
Your question states that you need to get the names of some log files in a certain date range.
Let's disregard the fact that you have the filenames in a text file and instead assume that you have direct access to the files in some directory $logdir
.
The format of the filenames are *_YYYYMMDDhhmmss.log
where the end bit is a standard timestamp string.
To get the files between *_201807010112.log
and *_201807030112.log
in a loop (to do something to each of them), use (in bash
),
process_flag=0
for pathname in "$logdir"/*_??????????????.log
do
if [ "$process_flag" -eq 0 ]; then
if [[ "$pathname" == *_201807010112.log ]]; then
process_flag=1
else
continue
fi
fi
# Do some sort of processing of
# the logfile in "$pathname" here.
# When done...
if [[ "$pathname" == *_201807030112.log ]]; then
break
fi
done
This loop loops over all logfiles that have a similar filename format. The loop will iterate over the pathnames in lexicographical order. It is assumed that all files have the same filename prefix (you say nothing about this).
The first part of the loop detects the first file in the range and sets process_flag
to 1
when that file is found. Setting process_flag
to 1
enables the loop to get into the middle bit where you actually use "$pathname"
for whatever processing of that file that you need doing.
Before continuing with the next iteration, the last if
statement checks to see whether the current $pathname
matches the last file that we wanted to process. If it does, the loop ends by a break
statement.
add a comment |
So I decided not to try to use date function to transform the form and instead use more SED!
cat files
..._201807010112.log
..._201807010132.log
..._201807010152.log
..._201807010202.log
The date is in the format YYYYmmddHHSS and I transformed it to the following format:
YYYY-mm-dd HH:SS using this regex expression with SED.
cat files | sed -e 's/(_[0-9]{4})([0-9]{2})
([0-9]{2})([0-9]{2})([0-9]{2}) /1-2-3 4:5/g'
Next, from there it was a piece of cake to specify the range based on date.
| sed -n '/2018-07-01 01:20/,/2018-07-02 02:01/p'
The biggest take away for me was knowing the proper date-time format to utilize SED's range pattern option.
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f460912%2fuse-sed-regex-capture-group-in-replace-section-method%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
Your question states that you need to get the names of some log files in a certain date range.
Let's disregard the fact that you have the filenames in a text file and instead assume that you have direct access to the files in some directory $logdir
.
The format of the filenames are *_YYYYMMDDhhmmss.log
where the end bit is a standard timestamp string.
To get the files between *_201807010112.log
and *_201807030112.log
in a loop (to do something to each of them), use (in bash
),
process_flag=0
for pathname in "$logdir"/*_??????????????.log
do
if [ "$process_flag" -eq 0 ]; then
if [[ "$pathname" == *_201807010112.log ]]; then
process_flag=1
else
continue
fi
fi
# Do some sort of processing of
# the logfile in "$pathname" here.
# When done...
if [[ "$pathname" == *_201807030112.log ]]; then
break
fi
done
This loop loops over all logfiles that have a similar filename format. The loop will iterate over the pathnames in lexicographical order. It is assumed that all files have the same filename prefix (you say nothing about this).
The first part of the loop detects the first file in the range and sets process_flag
to 1
when that file is found. Setting process_flag
to 1
enables the loop to get into the middle bit where you actually use "$pathname"
for whatever processing of that file that you need doing.
Before continuing with the next iteration, the last if
statement checks to see whether the current $pathname
matches the last file that we wanted to process. If it does, the loop ends by a break
statement.
add a comment |
Your question states that you need to get the names of some log files in a certain date range.
Let's disregard the fact that you have the filenames in a text file and instead assume that you have direct access to the files in some directory $logdir
.
The format of the filenames are *_YYYYMMDDhhmmss.log
where the end bit is a standard timestamp string.
To get the files between *_201807010112.log
and *_201807030112.log
in a loop (to do something to each of them), use (in bash
),
process_flag=0
for pathname in "$logdir"/*_??????????????.log
do
if [ "$process_flag" -eq 0 ]; then
if [[ "$pathname" == *_201807010112.log ]]; then
process_flag=1
else
continue
fi
fi
# Do some sort of processing of
# the logfile in "$pathname" here.
# When done...
if [[ "$pathname" == *_201807030112.log ]]; then
break
fi
done
This loop loops over all logfiles that have a similar filename format. The loop will iterate over the pathnames in lexicographical order. It is assumed that all files have the same filename prefix (you say nothing about this).
The first part of the loop detects the first file in the range and sets process_flag
to 1
when that file is found. Setting process_flag
to 1
enables the loop to get into the middle bit where you actually use "$pathname"
for whatever processing of that file that you need doing.
Before continuing with the next iteration, the last if
statement checks to see whether the current $pathname
matches the last file that we wanted to process. If it does, the loop ends by a break
statement.
add a comment |
Your question states that you need to get the names of some log files in a certain date range.
Let's disregard the fact that you have the filenames in a text file and instead assume that you have direct access to the files in some directory $logdir
.
The format of the filenames are *_YYYYMMDDhhmmss.log
where the end bit is a standard timestamp string.
To get the files between *_201807010112.log
and *_201807030112.log
in a loop (to do something to each of them), use (in bash
),
process_flag=0
for pathname in "$logdir"/*_??????????????.log
do
if [ "$process_flag" -eq 0 ]; then
if [[ "$pathname" == *_201807010112.log ]]; then
process_flag=1
else
continue
fi
fi
# Do some sort of processing of
# the logfile in "$pathname" here.
# When done...
if [[ "$pathname" == *_201807030112.log ]]; then
break
fi
done
This loop loops over all logfiles that have a similar filename format. The loop will iterate over the pathnames in lexicographical order. It is assumed that all files have the same filename prefix (you say nothing about this).
The first part of the loop detects the first file in the range and sets process_flag
to 1
when that file is found. Setting process_flag
to 1
enables the loop to get into the middle bit where you actually use "$pathname"
for whatever processing of that file that you need doing.
Before continuing with the next iteration, the last if
statement checks to see whether the current $pathname
matches the last file that we wanted to process. If it does, the loop ends by a break
statement.
Your question states that you need to get the names of some log files in a certain date range.
Let's disregard the fact that you have the filenames in a text file and instead assume that you have direct access to the files in some directory $logdir
.
The format of the filenames are *_YYYYMMDDhhmmss.log
where the end bit is a standard timestamp string.
To get the files between *_201807010112.log
and *_201807030112.log
in a loop (to do something to each of them), use (in bash
),
process_flag=0
for pathname in "$logdir"/*_??????????????.log
do
if [ "$process_flag" -eq 0 ]; then
if [[ "$pathname" == *_201807010112.log ]]; then
process_flag=1
else
continue
fi
fi
# Do some sort of processing of
# the logfile in "$pathname" here.
# When done...
if [[ "$pathname" == *_201807030112.log ]]; then
break
fi
done
This loop loops over all logfiles that have a similar filename format. The loop will iterate over the pathnames in lexicographical order. It is assumed that all files have the same filename prefix (you say nothing about this).
The first part of the loop detects the first file in the range and sets process_flag
to 1
when that file is found. Setting process_flag
to 1
enables the loop to get into the middle bit where you actually use "$pathname"
for whatever processing of that file that you need doing.
Before continuing with the next iteration, the last if
statement checks to see whether the current $pathname
matches the last file that we wanted to process. If it does, the loop ends by a break
statement.
answered Feb 15 at 20:23
KusalanandaKusalananda
133k17254417
133k17254417
add a comment |
add a comment |
So I decided not to try to use date function to transform the form and instead use more SED!
cat files
..._201807010112.log
..._201807010132.log
..._201807010152.log
..._201807010202.log
The date is in the format YYYYmmddHHSS and I transformed it to the following format:
YYYY-mm-dd HH:SS using this regex expression with SED.
cat files | sed -e 's/(_[0-9]{4})([0-9]{2})
([0-9]{2})([0-9]{2})([0-9]{2}) /1-2-3 4:5/g'
Next, from there it was a piece of cake to specify the range based on date.
| sed -n '/2018-07-01 01:20/,/2018-07-02 02:01/p'
The biggest take away for me was knowing the proper date-time format to utilize SED's range pattern option.
add a comment |
So I decided not to try to use date function to transform the form and instead use more SED!
cat files
..._201807010112.log
..._201807010132.log
..._201807010152.log
..._201807010202.log
The date is in the format YYYYmmddHHSS and I transformed it to the following format:
YYYY-mm-dd HH:SS using this regex expression with SED.
cat files | sed -e 's/(_[0-9]{4})([0-9]{2})
([0-9]{2})([0-9]{2})([0-9]{2}) /1-2-3 4:5/g'
Next, from there it was a piece of cake to specify the range based on date.
| sed -n '/2018-07-01 01:20/,/2018-07-02 02:01/p'
The biggest take away for me was knowing the proper date-time format to utilize SED's range pattern option.
add a comment |
So I decided not to try to use date function to transform the form and instead use more SED!
cat files
..._201807010112.log
..._201807010132.log
..._201807010152.log
..._201807010202.log
The date is in the format YYYYmmddHHSS and I transformed it to the following format:
YYYY-mm-dd HH:SS using this regex expression with SED.
cat files | sed -e 's/(_[0-9]{4})([0-9]{2})
([0-9]{2})([0-9]{2})([0-9]{2}) /1-2-3 4:5/g'
Next, from there it was a piece of cake to specify the range based on date.
| sed -n '/2018-07-01 01:20/,/2018-07-02 02:01/p'
The biggest take away for me was knowing the proper date-time format to utilize SED's range pattern option.
So I decided not to try to use date function to transform the form and instead use more SED!
cat files
..._201807010112.log
..._201807010132.log
..._201807010152.log
..._201807010202.log
The date is in the format YYYYmmddHHSS and I transformed it to the following format:
YYYY-mm-dd HH:SS using this regex expression with SED.
cat files | sed -e 's/(_[0-9]{4})([0-9]{2})
([0-9]{2})([0-9]{2})([0-9]{2}) /1-2-3 4:5/g'
Next, from there it was a piece of cake to specify the range based on date.
| sed -n '/2018-07-01 01:20/,/2018-07-02 02:01/p'
The biggest take away for me was knowing the proper date-time format to utilize SED's range pattern option.
answered Aug 7 '18 at 1:59
ProximoProximo
1305
1305
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f460912%2fuse-sed-regex-capture-group-in-replace-section-method%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
3
This is a very good example of a XY question...
– don_crissti
Aug 6 '18 at 22:23
cat 2018070{[1-2],301}
– user1133275
Aug 6 '18 at 22:49
1
like... sycamore?
– mikeserv
Aug 7 '18 at 0:52
1
Toxic comments, even I mentioned there could be a better way. User1133275, thanks for the suggestion but I need more flexibility when specifying the range. I'll answer my own question for the benefit of the community.
– Proximo
Aug 7 '18 at 1:47
Where is your
files
contents coming from? There might be a better way to get the filenames than from that file, e.g. usingfind
against the actual files in the filesystem (and their timestamps). That is what @don_crissti meant by XY problem. You are may be focusing on an issue which is down a path that you don't need to take while solving another problem.– Kusalananda
Aug 7 '18 at 6:14