Copy line pattern from 50000 files and copy in a file using Linux
0
I want to copy one line per file from 50k files, i.e. copy single lines into a new file. The resulting file will have 50k lines.
The command I tried is:
awk '/linetocopy/'file*/*.csv>new.csv # there are 50k .csv files
It returns new.csv file too large.
What's the right way to copy lines from that many files?
command-line unix
edited Jan 11 at 12:04
Kamil Maciorowski
25.8k155678
asked Jan 11 at 10:50
Sudipta DuttaSudipta Dutta
11
add a comment |
0
I want to copy one line per file from 50k files, i.e. copy single lines into a new file. The resulting file will have 50k lines.
The command I tried is:
awk '/linetocopy/'file*/*.csv>new.csv # there are 50k .csv files
It returns new.csv file too large.
What's the right way to copy lines from that many files?
command-line unix
edited Jan 11 at 12:04
Kamil Maciorowski
25.8k155678
asked Jan 11 at 10:50
Sudipta DuttaSudipta Dutta
11
2
Please post the exact command(s) you used. Which line is the one if the file consists of many lines? or is each file a one line already? Must you process the input files in certain order? or the order doesn't matter? Please do not respond in comments, edit the question instead.
– Kamil Maciorowski
Jan 11 at 11:01
Please address my questions (which lines? in what order?). Your command doesn't make much sense. Is there any specific reason why you thinkawkis the right tool? Probablygreporcat(possibly in a loop) will suffice, but we need to know these details I'm asking.
– Kamil Maciorowski
Jan 11 at 11:53
I formatted and slightly rephrased the question. Please review if it's still what you meant. The title should be improved as well. After you specify the details, the title may be improved and adjusted to them.
– Kamil Maciorowski
Jan 11 at 12:16
add a comment |
0
0
0
I want to copy one line per file from 50k files, i.e. copy single lines into a new file. The resulting file will have 50k lines.
The command I tried is:
awk '/linetocopy/'file*/*.csv>new.csv # there are 50k .csv files
It returns new.csv file too large.
What's the right way to copy lines from that many files?
command-line unix
edited Jan 11 at 12:04
Kamil Maciorowski
25.8k155678
asked Jan 11 at 10:50
Sudipta DuttaSudipta Dutta
11
I want to copy one line per file from 50k files, i.e. copy single lines into a new file. The resulting file will have 50k lines.
The command I tried is:
awk '/linetocopy/'file*/*.csv>new.csv # there are 50k .csv files
It returns new.csv file too large.
What's the right way to copy lines from that many files?
command-line unix
command-line unix
edited Jan 11 at 12:04
Kamil Maciorowski
25.8k155678
asked Jan 11 at 10:50
Sudipta DuttaSudipta Dutta
11
edited Jan 11 at 12:04
Kamil Maciorowski
25.8k155678
asked Jan 11 at 10:50
Sudipta DuttaSudipta Dutta
11
edited Jan 11 at 12:04
Kamil Maciorowski
25.8k155678
edited Jan 11 at 12:04
Kamil Maciorowski
25.8k155678
edited Jan 11 at 12:04
Kamil Maciorowski
25.8k155678
25.8k155678
asked Jan 11 at 10:50
Sudipta DuttaSudipta Dutta
11
asked Jan 11 at 10:50
Sudipta DuttaSudipta Dutta
11
asked Jan 11 at 10:50
Sudipta DuttaSudipta Dutta
11
11
2
Please post the exact command(s) you used. Which line is the one if the file consists of many lines? or is each file a one line already? Must you process the input files in certain order? or the order doesn't matter? Please do not respond in comments, edit the question instead.
– Kamil Maciorowski
Jan 11 at 11:01
Please address my questions (which lines? in what order?). Your command doesn't make much sense. Is there any specific reason why you thinkawkis the right tool? Probablygreporcat(possibly in a loop) will suffice, but we need to know these details I'm asking.
– Kamil Maciorowski
Jan 11 at 11:53
I formatted and slightly rephrased the question. Please review if it's still what you meant. The title should be improved as well. After you specify the details, the title may be improved and adjusted to them.
– Kamil Maciorowski
Jan 11 at 12:16
add a comment |
2
Please post the exact command(s) you used. Which line is the one if the file consists of many lines? or is each file a one line already? Must you process the input files in certain order? or the order doesn't matter? Please do not respond in comments, edit the question instead.
– Kamil Maciorowski
Jan 11 at 11:01
Please address my questions (which lines? in what order?). Your command doesn't make much sense. Is there any specific reason why you thinkawkis the right tool? Probablygreporcat(possibly in a loop) will suffice, but we need to know these details I'm asking.
– Kamil Maciorowski
Jan 11 at 11:53
I formatted and slightly rephrased the question. Please review if it's still what you meant. The title should be improved as well. After you specify the details, the title may be improved and adjusted to them.
– Kamil Maciorowski
Jan 11 at 12:16
2
2
Please post the exact command(s) you used. Which line is the one if the file consists of many lines? or is each file a one line already? Must you process the input files in certain order? or the order doesn't matter? Please do not respond in comments, edit the question instead.
– Kamil Maciorowski
Jan 11 at 11:01
Please post the exact command(s) you used. Which line is the one if the file consists of many lines? or is each file a one line already? Must you process the input files in certain order? or the order doesn't matter? Please do not respond in comments, edit the question instead.
– Kamil Maciorowski
Jan 11 at 11:01
Please address my questions (which lines? in what order?). Your command doesn't make much sense. Is there any specific reason why you think
awk is the right tool? Probably grep or cat (possibly in a loop) will suffice, but we need to know these details I'm asking.– Kamil Maciorowski
Jan 11 at 11:53
Please address my questions (which lines? in what order?). Your command doesn't make much sense. Is there any specific reason why you think
awk is the right tool? Probably grep or cat (possibly in a loop) will suffice, but we need to know these details I'm asking.– Kamil Maciorowski
Jan 11 at 11:53
I formatted and slightly rephrased the question. Please review if it's still what you meant. The title should be improved as well. After you specify the details, the title may be improved and adjusted to them.
– Kamil Maciorowski
Jan 11 at 12:16
I formatted and slightly rephrased the question. Please review if it's still what you meant. The title should be improved as well. After you specify the details, the title may be improved and adjusted to them.
– Kamil Maciorowski
Jan 11 at 12:16
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1393115%2fcopy-line-pattern-from-50000-files-and-copy-in-a-file-using-linux%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Super User!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1393115%2fcopy-line-pattern-from-50000-files-and-copy-in-a-file-using-linux%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
Please post the exact command(s) you used. Which line is the one if the file consists of many lines? or is each file a one line already? Must you process the input files in certain order? or the order doesn't matter? Please do not respond in comments, edit the question instead.
– Kamil Maciorowski
Jan 11 at 11:01
Please address my questions (which lines? in what order?). Your command doesn't make much sense. Is there any specific reason why you think
awkis the right tool? Probablygreporcat(possibly in a loop) will suffice, but we need to know these details I'm asking.– Kamil Maciorowski
Jan 11 at 11:53
I formatted and slightly rephrased the question. Please review if it's still what you meant. The title should be improved as well. After you specify the details, the title may be improved and adjusted to them.
– Kamil Maciorowski
Jan 11 at 12:16