smbclient alternative for large files
I'm using smbclient to transfer a set of large files (80GB) nightly from a Linux system to a Windows share. Lately, for whatever reason, I've been getting I/O timeouts:
cli_push returned NT_STATUS_IO_TIMEOUT
which causes the active file transfer to be aborted and deleted from the Windows share.
This may be due to unresolved Samba bug 8498 (or maybe not). The Windows system is not under my control, so I can't install an ssh server (to use scp or sftp), and do not want to depend on Microsoft's implementation of NFS.
Is there another simple, standard alternative that would let me move 80GB of data reliably from Linux to Windows over the network on a regular basis (the network is GB ethernet, so bandwidth isn't a problem)?
windows samba file-sharing
edited Feb 7 at 11:54
Matthias Braun
2,04921424
asked Feb 17 '12 at 3:34
Ex UmbrisEx Umbris
15218
add a comment |
I'm using smbclient to transfer a set of large files (80GB) nightly from a Linux system to a Windows share. Lately, for whatever reason, I've been getting I/O timeouts:
cli_push returned NT_STATUS_IO_TIMEOUT
which causes the active file transfer to be aborted and deleted from the Windows share.
This may be due to unresolved Samba bug 8498 (or maybe not). The Windows system is not under my control, so I can't install an ssh server (to use scp or sftp), and do not want to depend on Microsoft's implementation of NFS.
Is there another simple, standard alternative that would let me move 80GB of data reliably from Linux to Windows over the network on a regular basis (the network is GB ethernet, so bandwidth isn't a problem)?
windows samba file-sharing
edited Feb 7 at 11:54
Matthias Braun
2,04921424
asked Feb 17 '12 at 3:34
Ex UmbrisEx Umbris
15218
consider using tools such as rsync with partial mode enabled. Even WinScp should also help. Or provide a common NAS storage with NFS on Unix and CIFS on Windows, so no need to transferr at all incase if it is the same network. Best is to setup a torrent, incase the other network. ;-)
– Nikhil Mulley
Feb 17 '12 at 12:55
just stumbled across "123go file transfer program" search on google
– Nikhil Mulley
Feb 17 '12 at 12:57
add a comment |
I'm using smbclient to transfer a set of large files (80GB) nightly from a Linux system to a Windows share. Lately, for whatever reason, I've been getting I/O timeouts:
cli_push returned NT_STATUS_IO_TIMEOUT
which causes the active file transfer to be aborted and deleted from the Windows share.
This may be due to unresolved Samba bug 8498 (or maybe not). The Windows system is not under my control, so I can't install an ssh server (to use scp or sftp), and do not want to depend on Microsoft's implementation of NFS.
Is there another simple, standard alternative that would let me move 80GB of data reliably from Linux to Windows over the network on a regular basis (the network is GB ethernet, so bandwidth isn't a problem)?
windows samba file-sharing
edited Feb 7 at 11:54
Matthias Braun
2,04921424
asked Feb 17 '12 at 3:34
Ex UmbrisEx Umbris
15218
I'm using smbclient to transfer a set of large files (80GB) nightly from a Linux system to a Windows share. Lately, for whatever reason, I've been getting I/O timeouts:
cli_push returned NT_STATUS_IO_TIMEOUT
which causes the active file transfer to be aborted and deleted from the Windows share.
This may be due to unresolved Samba bug 8498 (or maybe not). The Windows system is not under my control, so I can't install an ssh server (to use scp or sftp), and do not want to depend on Microsoft's implementation of NFS.
Is there another simple, standard alternative that would let me move 80GB of data reliably from Linux to Windows over the network on a regular basis (the network is GB ethernet, so bandwidth isn't a problem)?
windows samba file-sharing
windows samba file-sharing
edited Feb 7 at 11:54
Matthias Braun
2,04921424
asked Feb 17 '12 at 3:34
Ex UmbrisEx Umbris
15218
edited Feb 7 at 11:54
Matthias Braun
2,04921424
asked Feb 17 '12 at 3:34
Ex UmbrisEx Umbris
15218
edited Feb 7 at 11:54
Matthias Braun
2,04921424
edited Feb 7 at 11:54
Matthias Braun
2,04921424
edited Feb 7 at 11:54
Matthias Braun
2,04921424
2,04921424
asked Feb 17 '12 at 3:34
Ex UmbrisEx Umbris
15218
asked Feb 17 '12 at 3:34
Ex UmbrisEx Umbris
15218
asked Feb 17 '12 at 3:34
Ex UmbrisEx Umbris
15218
15218
consider using tools such as rsync with partial mode enabled. Even WinScp should also help. Or provide a common NAS storage with NFS on Unix and CIFS on Windows, so no need to transferr at all incase if it is the same network. Best is to setup a torrent, incase the other network. ;-)
– Nikhil Mulley
Feb 17 '12 at 12:55
just stumbled across "123go file transfer program" search on google
– Nikhil Mulley
Feb 17 '12 at 12:57
add a comment |
consider using tools such as rsync with partial mode enabled. Even WinScp should also help. Or provide a common NAS storage with NFS on Unix and CIFS on Windows, so no need to transferr at all incase if it is the same network. Best is to setup a torrent, incase the other network. ;-)
– Nikhil Mulley
Feb 17 '12 at 12:55
just stumbled across "123go file transfer program" search on google
– Nikhil Mulley
Feb 17 '12 at 12:57
consider using tools such as rsync with partial mode enabled. Even WinScp should also help. Or provide a common NAS storage with NFS on Unix and CIFS on Windows, so no need to transferr at all incase if it is the same network. Best is to setup a torrent, incase the other network. ;-)
– Nikhil Mulley
Feb 17 '12 at 12:55
consider using tools such as rsync with partial mode enabled. Even WinScp should also help. Or provide a common NAS storage with NFS on Unix and CIFS on Windows, so no need to transferr at all incase if it is the same network. Best is to setup a torrent, incase the other network. ;-)
– Nikhil Mulley
Feb 17 '12 at 12:55
just stumbled across "123go file transfer program" search on google
– Nikhil Mulley
Feb 17 '12 at 12:57
just stumbled across "123go file transfer program" search on google
– Nikhil Mulley
Feb 17 '12 at 12:57
add a comment |
3 Answers
3
active
oldest
votes
Try using these socket options on smbclient
smbclient --socket-options='TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE SO_RCVBUF=131072 SO_SNDBUF=131072'
I regularly copy 40+GB files from Windows to Linux media server without error, typical transfer rate is 85MB/s with machines connected via gigabit switch.
answered Feb 17 '12 at 14:03
bsdbsd
7,90942132
1
Thanks for this - this got rid of the error for me; and correctly copied a 2G file from a Ubunutu to a Windows Share.
– monojohnny
Jan 24 '13 at 13:31
I've tried this and other variations of adjusting the values for SO_RCVBUF and SO_SNDBUF without luck. The file I'm trying to upload is about 8gig over a local network with zero packet loss.
– mhvelplund
Feb 29 '16 at 8:16
add a comment |
Maybe you can install a ftp server on your linux server, and ask Windows admin to send it the file nightly ?
FTP has a some useful functions for transferring big files and a pause/resume mechanism. For file this big, you should take care to not have a network hardware shutting down inactive connections too early. It can close your control connection before transfert ends.
answered Feb 17 '12 at 11:32
CorenCoren
3,75511943
Files are going the other way, from Linux to Windows
– Ex Umbris
Feb 17 '12 at 20:08
add a comment |
Using curl
I'm running smbclient version 4.9.4 trying to transfer a 97 MiB file from Arch Linux to Windows and calling smbclient with --socket-options='TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE SO_RCVBUF=131072 SO_SNDBUF=131072' as user bsd recommended still failed with cli_push returned NT_STATUS_IO_TIMEOUT.
Since version 7.40, curl supports the smb protocol.
Thus, I used this to upload the moderately_sized_file from Linux to the service OurRemoteDirectory on the Windows machine at 172.16.17.52:
curl --upload-file /home/me/moderately_sized_file --user "OurWindowsDomain/MyUserName:MyPassword" smb://172.16.17.52/OurRemoteDirectory/Path/To/Dir/
For me, curl has uploaded the file reliably each time and also displays upload progress, which is nice.
Note that curl doesn't yet support creating directories on the remote host.
Consequently, you might need to create /Path/To/Dir/ using the following command (but smbclient mkdir has worked without an issue so far):
smbclient //172.16.17.52/OurRemoteDirectory/ -U MyUserName%MyPassword -W OurWindowsDomain -c 'mkdir Path/To/Dir/'
answered Feb 7 at 11:39
Matthias BraunMatthias Braun
2,04921424
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f31900%2fsmbclient-alternative-for-large-files%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
Try using these socket options on smbclient
smbclient --socket-options='TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE SO_RCVBUF=131072 SO_SNDBUF=131072'
I regularly copy 40+GB files from Windows to Linux media server without error, typical transfer rate is 85MB/s with machines connected via gigabit switch.
answered Feb 17 '12 at 14:03
bsdbsd
7,90942132
1
Thanks for this - this got rid of the error for me; and correctly copied a 2G file from a Ubunutu to a Windows Share.
– monojohnny
Jan 24 '13 at 13:31
I've tried this and other variations of adjusting the values for SO_RCVBUF and SO_SNDBUF without luck. The file I'm trying to upload is about 8gig over a local network with zero packet loss.
– mhvelplund
Feb 29 '16 at 8:16
add a comment |
Try using these socket options on smbclient
smbclient --socket-options='TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE SO_RCVBUF=131072 SO_SNDBUF=131072'
I regularly copy 40+GB files from Windows to Linux media server without error, typical transfer rate is 85MB/s with machines connected via gigabit switch.
answered Feb 17 '12 at 14:03
bsdbsd
7,90942132
1
Thanks for this - this got rid of the error for me; and correctly copied a 2G file from a Ubunutu to a Windows Share.
– monojohnny
Jan 24 '13 at 13:31
I've tried this and other variations of adjusting the values for SO_RCVBUF and SO_SNDBUF without luck. The file I'm trying to upload is about 8gig over a local network with zero packet loss.
– mhvelplund
Feb 29 '16 at 8:16
add a comment |
Try using these socket options on smbclient
smbclient --socket-options='TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE SO_RCVBUF=131072 SO_SNDBUF=131072'
I regularly copy 40+GB files from Windows to Linux media server without error, typical transfer rate is 85MB/s with machines connected via gigabit switch.
answered Feb 17 '12 at 14:03
bsdbsd
7,90942132
Try using these socket options on smbclient
smbclient --socket-options='TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE SO_RCVBUF=131072 SO_SNDBUF=131072'
I regularly copy 40+GB files from Windows to Linux media server without error, typical transfer rate is 85MB/s with machines connected via gigabit switch.
answered Feb 17 '12 at 14:03
bsdbsd
7,90942132
answered Feb 17 '12 at 14:03
bsdbsd
7,90942132
answered Feb 17 '12 at 14:03
bsdbsd
7,90942132
answered Feb 17 '12 at 14:03
bsdbsd
7,90942132
7,90942132
1
Thanks for this - this got rid of the error for me; and correctly copied a 2G file from a Ubunutu to a Windows Share.
– monojohnny
Jan 24 '13 at 13:31
I've tried this and other variations of adjusting the values for SO_RCVBUF and SO_SNDBUF without luck. The file I'm trying to upload is about 8gig over a local network with zero packet loss.
– mhvelplund
Feb 29 '16 at 8:16
add a comment |
1
Thanks for this - this got rid of the error for me; and correctly copied a 2G file from a Ubunutu to a Windows Share.
– monojohnny
Jan 24 '13 at 13:31
I've tried this and other variations of adjusting the values for SO_RCVBUF and SO_SNDBUF without luck. The file I'm trying to upload is about 8gig over a local network with zero packet loss.
– mhvelplund
Feb 29 '16 at 8:16
1
1
Thanks for this - this got rid of the error for me; and correctly copied a 2G file from a Ubunutu to a Windows Share.
– monojohnny
Jan 24 '13 at 13:31
Thanks for this - this got rid of the error for me; and correctly copied a 2G file from a Ubunutu to a Windows Share.
– monojohnny
Jan 24 '13 at 13:31
I've tried this and other variations of adjusting the values for SO_RCVBUF and SO_SNDBUF without luck. The file I'm trying to upload is about 8gig over a local network with zero packet loss.
– mhvelplund
Feb 29 '16 at 8:16
I've tried this and other variations of adjusting the values for SO_RCVBUF and SO_SNDBUF without luck. The file I'm trying to upload is about 8gig over a local network with zero packet loss.
– mhvelplund
Feb 29 '16 at 8:16
add a comment |
Maybe you can install a ftp server on your linux server, and ask Windows admin to send it the file nightly ?
FTP has a some useful functions for transferring big files and a pause/resume mechanism. For file this big, you should take care to not have a network hardware shutting down inactive connections too early. It can close your control connection before transfert ends.
answered Feb 17 '12 at 11:32
CorenCoren
3,75511943
Files are going the other way, from Linux to Windows
– Ex Umbris
Feb 17 '12 at 20:08
add a comment |
Maybe you can install a ftp server on your linux server, and ask Windows admin to send it the file nightly ?
FTP has a some useful functions for transferring big files and a pause/resume mechanism. For file this big, you should take care to not have a network hardware shutting down inactive connections too early. It can close your control connection before transfert ends.
answered Feb 17 '12 at 11:32
CorenCoren
3,75511943
Files are going the other way, from Linux to Windows
– Ex Umbris
Feb 17 '12 at 20:08
add a comment |
Maybe you can install a ftp server on your linux server, and ask Windows admin to send it the file nightly ?
FTP has a some useful functions for transferring big files and a pause/resume mechanism. For file this big, you should take care to not have a network hardware shutting down inactive connections too early. It can close your control connection before transfert ends.
answered Feb 17 '12 at 11:32
CorenCoren
3,75511943
Maybe you can install a ftp server on your linux server, and ask Windows admin to send it the file nightly ?
FTP has a some useful functions for transferring big files and a pause/resume mechanism. For file this big, you should take care to not have a network hardware shutting down inactive connections too early. It can close your control connection before transfert ends.
answered Feb 17 '12 at 11:32
CorenCoren
3,75511943
answered Feb 17 '12 at 11:32
CorenCoren
3,75511943
answered Feb 17 '12 at 11:32
CorenCoren
3,75511943
answered Feb 17 '12 at 11:32
CorenCoren
3,75511943
3,75511943
Files are going the other way, from Linux to Windows
– Ex Umbris
Feb 17 '12 at 20:08
add a comment |
Files are going the other way, from Linux to Windows
– Ex Umbris
Feb 17 '12 at 20:08
Files are going the other way, from Linux to Windows
– Ex Umbris
Feb 17 '12 at 20:08
Files are going the other way, from Linux to Windows
– Ex Umbris
Feb 17 '12 at 20:08
add a comment |
Using curl
I'm running smbclient version 4.9.4 trying to transfer a 97 MiB file from Arch Linux to Windows and calling smbclient with --socket-options='TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE SO_RCVBUF=131072 SO_SNDBUF=131072' as user bsd recommended still failed with cli_push returned NT_STATUS_IO_TIMEOUT.
Since version 7.40, curl supports the smb protocol.
Thus, I used this to upload the moderately_sized_file from Linux to the service OurRemoteDirectory on the Windows machine at 172.16.17.52:
curl --upload-file /home/me/moderately_sized_file --user "OurWindowsDomain/MyUserName:MyPassword" smb://172.16.17.52/OurRemoteDirectory/Path/To/Dir/
For me, curl has uploaded the file reliably each time and also displays upload progress, which is nice.
Note that curl doesn't yet support creating directories on the remote host.
Consequently, you might need to create /Path/To/Dir/ using the following command (but smbclient mkdir has worked without an issue so far):
smbclient //172.16.17.52/OurRemoteDirectory/ -U MyUserName%MyPassword -W OurWindowsDomain -c 'mkdir Path/To/Dir/'
answered Feb 7 at 11:39
Matthias BraunMatthias Braun
2,04921424
add a comment |
Using curl
I'm running smbclient version 4.9.4 trying to transfer a 97 MiB file from Arch Linux to Windows and calling smbclient with --socket-options='TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE SO_RCVBUF=131072 SO_SNDBUF=131072' as user bsd recommended still failed with cli_push returned NT_STATUS_IO_TIMEOUT.
Since version 7.40, curl supports the smb protocol.
Thus, I used this to upload the moderately_sized_file from Linux to the service OurRemoteDirectory on the Windows machine at 172.16.17.52:
curl --upload-file /home/me/moderately_sized_file --user "OurWindowsDomain/MyUserName:MyPassword" smb://172.16.17.52/OurRemoteDirectory/Path/To/Dir/
For me, curl has uploaded the file reliably each time and also displays upload progress, which is nice.
Note that curl doesn't yet support creating directories on the remote host.
Consequently, you might need to create /Path/To/Dir/ using the following command (but smbclient mkdir has worked without an issue so far):
smbclient //172.16.17.52/OurRemoteDirectory/ -U MyUserName%MyPassword -W OurWindowsDomain -c 'mkdir Path/To/Dir/'
answered Feb 7 at 11:39
Matthias BraunMatthias Braun
2,04921424
add a comment |
Using curl
I'm running smbclient version 4.9.4 trying to transfer a 97 MiB file from Arch Linux to Windows and calling smbclient with --socket-options='TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE SO_RCVBUF=131072 SO_SNDBUF=131072' as user bsd recommended still failed with cli_push returned NT_STATUS_IO_TIMEOUT.
Since version 7.40, curl supports the smb protocol.
Thus, I used this to upload the moderately_sized_file from Linux to the service OurRemoteDirectory on the Windows machine at 172.16.17.52:
curl --upload-file /home/me/moderately_sized_file --user "OurWindowsDomain/MyUserName:MyPassword" smb://172.16.17.52/OurRemoteDirectory/Path/To/Dir/
For me, curl has uploaded the file reliably each time and also displays upload progress, which is nice.
Note that curl doesn't yet support creating directories on the remote host.
Consequently, you might need to create /Path/To/Dir/ using the following command (but smbclient mkdir has worked without an issue so far):
smbclient //172.16.17.52/OurRemoteDirectory/ -U MyUserName%MyPassword -W OurWindowsDomain -c 'mkdir Path/To/Dir/'
answered Feb 7 at 11:39
Matthias BraunMatthias Braun
2,04921424
Using curl
I'm running smbclient version 4.9.4 trying to transfer a 97 MiB file from Arch Linux to Windows and calling smbclient with --socket-options='TCP_NODELAY IPTOS_LOWDELAY SO_KEEPALIVE SO_RCVBUF=131072 SO_SNDBUF=131072' as user bsd recommended still failed with cli_push returned NT_STATUS_IO_TIMEOUT.
Since version 7.40, curl supports the smb protocol.
Thus, I used this to upload the moderately_sized_file from Linux to the service OurRemoteDirectory on the Windows machine at 172.16.17.52:
curl --upload-file /home/me/moderately_sized_file --user "OurWindowsDomain/MyUserName:MyPassword" smb://172.16.17.52/OurRemoteDirectory/Path/To/Dir/
For me, curl has uploaded the file reliably each time and also displays upload progress, which is nice.
Note that curl doesn't yet support creating directories on the remote host.
Consequently, you might need to create /Path/To/Dir/ using the following command (but smbclient mkdir has worked without an issue so far):
smbclient //172.16.17.52/OurRemoteDirectory/ -U MyUserName%MyPassword -W OurWindowsDomain -c 'mkdir Path/To/Dir/'
answered Feb 7 at 11:39
Matthias BraunMatthias Braun
2,04921424
edited Feb 8 at 12:25
answered Feb 7 at 11:39
Matthias BraunMatthias Braun
2,04921424
answered Feb 7 at 11:39
Matthias BraunMatthias Braun
2,04921424
answered Feb 7 at 11:39
Matthias BraunMatthias Braun
2,04921424
2,04921424
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f31900%2fsmbclient-alternative-for-large-files%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
consider using tools such as rsync with partial mode enabled. Even WinScp should also help. Or provide a common NAS storage with NFS on Unix and CIFS on Windows, so no need to transferr at all incase if it is the same network. Best is to setup a torrent, incase the other network. ;-)
– Nikhil Mulley
Feb 17 '12 at 12:55
just stumbled across "123go file transfer program" search on google
– Nikhil Mulley
Feb 17 '12 at 12:57