rsync N newest files in a directory
What would be the easiest way to go about rsyncing the n newest files in a directory to a remote server?
linux bash ssh rsync xargs
asked Nov 8 '10 at 17:20
LedaLeda
241511
add a comment |
What would be the easiest way to go about rsyncing the n newest files in a directory to a remote server?
linux bash ssh rsync xargs
asked Nov 8 '10 at 17:20
LedaLeda
241511
If you want to update, then rsync has an --update switch. You could also usefindwithheadif you want the N newest files, and pipe this into rsync. You may have to use xargs but I don't know for sure.
– QuickishFM
Jan 31 at 12:26
add a comment |
What would be the easiest way to go about rsyncing the n newest files in a directory to a remote server?
linux bash ssh rsync xargs
asked Nov 8 '10 at 17:20
LedaLeda
241511
What would be the easiest way to go about rsyncing the n newest files in a directory to a remote server?
linux bash ssh rsync xargs
linux bash ssh rsync xargs
asked Nov 8 '10 at 17:20
LedaLeda
241511
asked Nov 8 '10 at 17:20
LedaLeda
241511
asked Nov 8 '10 at 17:20
LedaLeda
241511
asked Nov 8 '10 at 17:20
LedaLeda
241511
asked Nov 8 '10 at 17:20
LedaLeda
241511
241511
If you want to update, then rsync has an --update switch. You could also usefindwithheadif you want the N newest files, and pipe this into rsync. You may have to use xargs but I don't know for sure.
– QuickishFM
Jan 31 at 12:26
add a comment |
If you want to update, then rsync has an --update switch. You could also usefindwithheadif you want the N newest files, and pipe this into rsync. You may have to use xargs but I don't know for sure.
– QuickishFM
Jan 31 at 12:26
If you want to update, then rsync has an --update switch. You could also use
find with head if you want the N newest files, and pipe this into rsync. You may have to use xargs but I don't know for sure.– QuickishFM
Jan 31 at 12:26
If you want to update, then rsync has an --update switch. You could also use
find with head if you want the N newest files, and pipe this into rsync. You may have to use xargs but I don't know for sure.– QuickishFM
Jan 31 at 12:26
add a comment |
2 Answers
2
active
oldest
votes
The easiest way is to run zsh instead of bash.
rsync -a -- /path/to/directory/*(om[1,42]) remote-server:
In the parentheses, om orders files by reverse modification time (i.e. by increasing age), and [1,42] selects the first 42 matches.
If you want only regular files and not directories, add a . after the opening parenthesis. For more possibilities, look under “glob qualifiers” in the zshexpn man page.
If you want to match files in the directory tree rooted at /path/to/directory, rather than just inside that directory, use /path/to/directory/**/*(.om[1,42]). This will send all the files to the same directory on the target, though.
answered Nov 9 '10 at 0:02
GillesGilles
52.9k15114161
FYI, I had to add single quote around '*(om[1,42])' to make it work. I was copying files from remote server to local though.
– c 2
Nov 19 '17 at 18:57
@c2 The quotes cause the wildcards to be expanded on the remote machine. They are needed on a remote path (and will work only if your login shell on the remote machine in zsh), and would not work on a local path.
– Gilles
Nov 19 '17 at 20:28
add a comment |
Assuming you want to send files from the current working directory:
rsync `ls -tp | grep -v / | head -n <n>` <destination> <options>
will do the trick. For example:
rsync `ls -tp | grep -v / | head -n 10` user@host:/dest/dir/ --progress --compress
This will give an error if there are no files to be found in the current working directory, or if any of the top files contain spaces or other special characters.
The ` characters around ls -tp | grep -v / | head -n <n> tell bash to run the commands and replaced them with the resulting file list as a space separated list. The -t option tells ls to sort by timestamp, the -p tells it to add a / after directory names and the grep part screens out lines ending / so you don't end up sending directories over. Add -c to the ls options if you want the newest files to be judged by creation time instead of modification time (though note that some programs will remove and replace files instead of updating them so ctime and mtime can be the same even though a file seems to have been around longer).
I'll not claim it is without doubt the easiest way but it would be the way I'd first think of.
(mwm's sub-question-in-a-comment)
what about the other way around? server to local?
That would be a little more complex as I don't think you can have multiple remote file sources on the rsync command line like you can local ones. You could run rsync on the server via over ssh, opening a tunnel back to your local sshd (with the same ssh invocation you run rsync through) to receive the connection, essentially used the same command from the other side.
answered Nov 8 '10 at 18:29
David SpillettDavid Spillett
21.9k4062
Do not use this command unless you know that the file names cannot contain any “special” character (whitespace,[?*, control characters or non-ASCII characters).
– Gilles
Nov 9 '10 at 0:06
what about the other way around? server to local?
– mwm
Jan 30 at 21:53
@mwm - more faf, but possible. See note added to answer.
– David Spillett
Jan 31 at 12:11
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f208591%2frsync-n-newest-files-in-a-directory%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
The easiest way is to run zsh instead of bash.
rsync -a -- /path/to/directory/*(om[1,42]) remote-server:
In the parentheses, om orders files by reverse modification time (i.e. by increasing age), and [1,42] selects the first 42 matches.
If you want only regular files and not directories, add a . after the opening parenthesis. For more possibilities, look under “glob qualifiers” in the zshexpn man page.
If you want to match files in the directory tree rooted at /path/to/directory, rather than just inside that directory, use /path/to/directory/**/*(.om[1,42]). This will send all the files to the same directory on the target, though.
answered Nov 9 '10 at 0:02
GillesGilles
52.9k15114161
FYI, I had to add single quote around '*(om[1,42])' to make it work. I was copying files from remote server to local though.
– c 2
Nov 19 '17 at 18:57
@c2 The quotes cause the wildcards to be expanded on the remote machine. They are needed on a remote path (and will work only if your login shell on the remote machine in zsh), and would not work on a local path.
– Gilles
Nov 19 '17 at 20:28
add a comment |
The easiest way is to run zsh instead of bash.
rsync -a -- /path/to/directory/*(om[1,42]) remote-server:
In the parentheses, om orders files by reverse modification time (i.e. by increasing age), and [1,42] selects the first 42 matches.
If you want only regular files and not directories, add a . after the opening parenthesis. For more possibilities, look under “glob qualifiers” in the zshexpn man page.
If you want to match files in the directory tree rooted at /path/to/directory, rather than just inside that directory, use /path/to/directory/**/*(.om[1,42]). This will send all the files to the same directory on the target, though.
answered Nov 9 '10 at 0:02
GillesGilles
52.9k15114161
FYI, I had to add single quote around '*(om[1,42])' to make it work. I was copying files from remote server to local though.
– c 2
Nov 19 '17 at 18:57
@c2 The quotes cause the wildcards to be expanded on the remote machine. They are needed on a remote path (and will work only if your login shell on the remote machine in zsh), and would not work on a local path.
– Gilles
Nov 19 '17 at 20:28
add a comment |
The easiest way is to run zsh instead of bash.
rsync -a -- /path/to/directory/*(om[1,42]) remote-server:
In the parentheses, om orders files by reverse modification time (i.e. by increasing age), and [1,42] selects the first 42 matches.
If you want only regular files and not directories, add a . after the opening parenthesis. For more possibilities, look under “glob qualifiers” in the zshexpn man page.
If you want to match files in the directory tree rooted at /path/to/directory, rather than just inside that directory, use /path/to/directory/**/*(.om[1,42]). This will send all the files to the same directory on the target, though.
answered Nov 9 '10 at 0:02
GillesGilles
52.9k15114161
The easiest way is to run zsh instead of bash.
rsync -a -- /path/to/directory/*(om[1,42]) remote-server:
In the parentheses, om orders files by reverse modification time (i.e. by increasing age), and [1,42] selects the first 42 matches.
If you want only regular files and not directories, add a . after the opening parenthesis. For more possibilities, look under “glob qualifiers” in the zshexpn man page.
If you want to match files in the directory tree rooted at /path/to/directory, rather than just inside that directory, use /path/to/directory/**/*(.om[1,42]). This will send all the files to the same directory on the target, though.
answered Nov 9 '10 at 0:02
GillesGilles
52.9k15114161
answered Nov 9 '10 at 0:02
GillesGilles
52.9k15114161
answered Nov 9 '10 at 0:02
GillesGilles
52.9k15114161
answered Nov 9 '10 at 0:02
GillesGilles
52.9k15114161
52.9k15114161
FYI, I had to add single quote around '*(om[1,42])' to make it work. I was copying files from remote server to local though.
– c 2
Nov 19 '17 at 18:57
@c2 The quotes cause the wildcards to be expanded on the remote machine. They are needed on a remote path (and will work only if your login shell on the remote machine in zsh), and would not work on a local path.
– Gilles
Nov 19 '17 at 20:28
add a comment |
FYI, I had to add single quote around '*(om[1,42])' to make it work. I was copying files from remote server to local though.
– c 2
Nov 19 '17 at 18:57
@c2 The quotes cause the wildcards to be expanded on the remote machine. They are needed on a remote path (and will work only if your login shell on the remote machine in zsh), and would not work on a local path.
– Gilles
Nov 19 '17 at 20:28
FYI, I had to add single quote around '*(om[1,42])' to make it work. I was copying files from remote server to local though.
– c 2
Nov 19 '17 at 18:57
FYI, I had to add single quote around '*(om[1,42])' to make it work. I was copying files from remote server to local though.
– c 2
Nov 19 '17 at 18:57
@c2 The quotes cause the wildcards to be expanded on the remote machine. They are needed on a remote path (and will work only if your login shell on the remote machine in zsh), and would not work on a local path.
– Gilles
Nov 19 '17 at 20:28
@c2 The quotes cause the wildcards to be expanded on the remote machine. They are needed on a remote path (and will work only if your login shell on the remote machine in zsh), and would not work on a local path.
– Gilles
Nov 19 '17 at 20:28
add a comment |
Assuming you want to send files from the current working directory:
rsync `ls -tp | grep -v / | head -n <n>` <destination> <options>
will do the trick. For example:
rsync `ls -tp | grep -v / | head -n 10` user@host:/dest/dir/ --progress --compress
This will give an error if there are no files to be found in the current working directory, or if any of the top files contain spaces or other special characters.
The ` characters around ls -tp | grep -v / | head -n <n> tell bash to run the commands and replaced them with the resulting file list as a space separated list. The -t option tells ls to sort by timestamp, the -p tells it to add a / after directory names and the grep part screens out lines ending / so you don't end up sending directories over. Add -c to the ls options if you want the newest files to be judged by creation time instead of modification time (though note that some programs will remove and replace files instead of updating them so ctime and mtime can be the same even though a file seems to have been around longer).
I'll not claim it is without doubt the easiest way but it would be the way I'd first think of.
(mwm's sub-question-in-a-comment)
what about the other way around? server to local?
That would be a little more complex as I don't think you can have multiple remote file sources on the rsync command line like you can local ones. You could run rsync on the server via over ssh, opening a tunnel back to your local sshd (with the same ssh invocation you run rsync through) to receive the connection, essentially used the same command from the other side.
answered Nov 8 '10 at 18:29
David SpillettDavid Spillett
21.9k4062
Do not use this command unless you know that the file names cannot contain any “special” character (whitespace,[?*, control characters or non-ASCII characters).
– Gilles
Nov 9 '10 at 0:06
what about the other way around? server to local?
– mwm
Jan 30 at 21:53
@mwm - more faf, but possible. See note added to answer.
– David Spillett
Jan 31 at 12:11
add a comment |
Assuming you want to send files from the current working directory:
rsync `ls -tp | grep -v / | head -n <n>` <destination> <options>
will do the trick. For example:
rsync `ls -tp | grep -v / | head -n 10` user@host:/dest/dir/ --progress --compress
This will give an error if there are no files to be found in the current working directory, or if any of the top files contain spaces or other special characters.
The ` characters around ls -tp | grep -v / | head -n <n> tell bash to run the commands and replaced them with the resulting file list as a space separated list. The -t option tells ls to sort by timestamp, the -p tells it to add a / after directory names and the grep part screens out lines ending / so you don't end up sending directories over. Add -c to the ls options if you want the newest files to be judged by creation time instead of modification time (though note that some programs will remove and replace files instead of updating them so ctime and mtime can be the same even though a file seems to have been around longer).
I'll not claim it is without doubt the easiest way but it would be the way I'd first think of.
(mwm's sub-question-in-a-comment)
what about the other way around? server to local?
That would be a little more complex as I don't think you can have multiple remote file sources on the rsync command line like you can local ones. You could run rsync on the server via over ssh, opening a tunnel back to your local sshd (with the same ssh invocation you run rsync through) to receive the connection, essentially used the same command from the other side.
answered Nov 8 '10 at 18:29
David SpillettDavid Spillett
21.9k4062
Do not use this command unless you know that the file names cannot contain any “special” character (whitespace,[?*, control characters or non-ASCII characters).
– Gilles
Nov 9 '10 at 0:06
what about the other way around? server to local?
– mwm
Jan 30 at 21:53
@mwm - more faf, but possible. See note added to answer.
– David Spillett
Jan 31 at 12:11
add a comment |
Assuming you want to send files from the current working directory:
rsync `ls -tp | grep -v / | head -n <n>` <destination> <options>
will do the trick. For example:
rsync `ls -tp | grep -v / | head -n 10` user@host:/dest/dir/ --progress --compress
This will give an error if there are no files to be found in the current working directory, or if any of the top files contain spaces or other special characters.
The ` characters around ls -tp | grep -v / | head -n <n> tell bash to run the commands and replaced them with the resulting file list as a space separated list. The -t option tells ls to sort by timestamp, the -p tells it to add a / after directory names and the grep part screens out lines ending / so you don't end up sending directories over. Add -c to the ls options if you want the newest files to be judged by creation time instead of modification time (though note that some programs will remove and replace files instead of updating them so ctime and mtime can be the same even though a file seems to have been around longer).
I'll not claim it is without doubt the easiest way but it would be the way I'd first think of.
(mwm's sub-question-in-a-comment)
what about the other way around? server to local?
That would be a little more complex as I don't think you can have multiple remote file sources on the rsync command line like you can local ones. You could run rsync on the server via over ssh, opening a tunnel back to your local sshd (with the same ssh invocation you run rsync through) to receive the connection, essentially used the same command from the other side.
answered Nov 8 '10 at 18:29
David SpillettDavid Spillett
21.9k4062
Assuming you want to send files from the current working directory:
rsync `ls -tp | grep -v / | head -n <n>` <destination> <options>
will do the trick. For example:
rsync `ls -tp | grep -v / | head -n 10` user@host:/dest/dir/ --progress --compress
This will give an error if there are no files to be found in the current working directory, or if any of the top files contain spaces or other special characters.
The ` characters around ls -tp | grep -v / | head -n <n> tell bash to run the commands and replaced them with the resulting file list as a space separated list. The -t option tells ls to sort by timestamp, the -p tells it to add a / after directory names and the grep part screens out lines ending / so you don't end up sending directories over. Add -c to the ls options if you want the newest files to be judged by creation time instead of modification time (though note that some programs will remove and replace files instead of updating them so ctime and mtime can be the same even though a file seems to have been around longer).
I'll not claim it is without doubt the easiest way but it would be the way I'd first think of.
(mwm's sub-question-in-a-comment)
what about the other way around? server to local?
That would be a little more complex as I don't think you can have multiple remote file sources on the rsync command line like you can local ones. You could run rsync on the server via over ssh, opening a tunnel back to your local sshd (with the same ssh invocation you run rsync through) to receive the connection, essentially used the same command from the other side.
answered Nov 8 '10 at 18:29
David SpillettDavid Spillett
21.9k4062
edited Jan 31 at 12:10
answered Nov 8 '10 at 18:29
David SpillettDavid Spillett
21.9k4062
answered Nov 8 '10 at 18:29
David SpillettDavid Spillett
21.9k4062
answered Nov 8 '10 at 18:29
David SpillettDavid Spillett
21.9k4062
21.9k4062
Do not use this command unless you know that the file names cannot contain any “special” character (whitespace,[?*, control characters or non-ASCII characters).
– Gilles
Nov 9 '10 at 0:06
what about the other way around? server to local?
– mwm
Jan 30 at 21:53
@mwm - more faf, but possible. See note added to answer.
– David Spillett
Jan 31 at 12:11
add a comment |
Do not use this command unless you know that the file names cannot contain any “special” character (whitespace,[?*, control characters or non-ASCII characters).
– Gilles
Nov 9 '10 at 0:06
what about the other way around? server to local?
– mwm
Jan 30 at 21:53
@mwm - more faf, but possible. See note added to answer.
– David Spillett
Jan 31 at 12:11
Do not use this command unless you know that the file names cannot contain any “special” character (whitespace,
[?*, control characters or non-ASCII characters).– Gilles
Nov 9 '10 at 0:06
Do not use this command unless you know that the file names cannot contain any “special” character (whitespace,
[?*, control characters or non-ASCII characters).– Gilles
Nov 9 '10 at 0:06
what about the other way around? server to local?
– mwm
Jan 30 at 21:53
what about the other way around? server to local?
– mwm
Jan 30 at 21:53
@mwm - more faf, but possible. See note added to answer.
– David Spillett
Jan 31 at 12:11
@mwm - more faf, but possible. See note added to answer.
– David Spillett
Jan 31 at 12:11
add a comment |
Thanks for contributing an answer to Super User!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f208591%2frsync-n-newest-files-in-a-directory%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
If you want to update, then rsync has an --update switch. You could also use
findwithheadif you want the N newest files, and pipe this into rsync. You may have to use xargs but I don't know for sure.– QuickishFM
Jan 31 at 12:26