Writing a script to startup through multiple server layers [closed]
Each day, I need to begin work on a set of jupyter notebooks. The jupyter notebooks must be started on a docker, which is located on a remote server, which I have ssh connection to via my laptop. Thus each time I need to begin work it takes about 5 mins to run each command: tunneling in, starting it, getting out, establish a 2nd ssh to the notebook, etc. I would like to write a script that does this all. My issue is that I don't know how to write the script such that it executes on my laptop, creates an ssh, and then on the server's bash can then execute a script to start up a docker, and then on the docker's command line navigate to the notebook directory and run a jupyter notebook. There are only 2 or 3 commands needed on each stage but because they are on different platforms I'm having difficulty.
linux bash ssh jupyter-notebook
closed as too broad by music2myear, Mike Fitzpatrick, Debra, bertieb, DrMoishe Pippik Feb 17 at 4:07
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
add a comment |
Each day, I need to begin work on a set of jupyter notebooks. The jupyter notebooks must be started on a docker, which is located on a remote server, which I have ssh connection to via my laptop. Thus each time I need to begin work it takes about 5 mins to run each command: tunneling in, starting it, getting out, establish a 2nd ssh to the notebook, etc. I would like to write a script that does this all. My issue is that I don't know how to write the script such that it executes on my laptop, creates an ssh, and then on the server's bash can then execute a script to start up a docker, and then on the docker's command line navigate to the notebook directory and run a jupyter notebook. There are only 2 or 3 commands needed on each stage but because they are on different platforms I'm having difficulty.
linux bash ssh jupyter-notebook
closed as too broad by music2myear, Mike Fitzpatrick, Debra, bertieb, DrMoishe Pippik Feb 17 at 4:07
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
add a comment |
Each day, I need to begin work on a set of jupyter notebooks. The jupyter notebooks must be started on a docker, which is located on a remote server, which I have ssh connection to via my laptop. Thus each time I need to begin work it takes about 5 mins to run each command: tunneling in, starting it, getting out, establish a 2nd ssh to the notebook, etc. I would like to write a script that does this all. My issue is that I don't know how to write the script such that it executes on my laptop, creates an ssh, and then on the server's bash can then execute a script to start up a docker, and then on the docker's command line navigate to the notebook directory and run a jupyter notebook. There are only 2 or 3 commands needed on each stage but because they are on different platforms I'm having difficulty.
linux bash ssh jupyter-notebook
Each day, I need to begin work on a set of jupyter notebooks. The jupyter notebooks must be started on a docker, which is located on a remote server, which I have ssh connection to via my laptop. Thus each time I need to begin work it takes about 5 mins to run each command: tunneling in, starting it, getting out, establish a 2nd ssh to the notebook, etc. I would like to write a script that does this all. My issue is that I don't know how to write the script such that it executes on my laptop, creates an ssh, and then on the server's bash can then execute a script to start up a docker, and then on the docker's command line navigate to the notebook directory and run a jupyter notebook. There are only 2 or 3 commands needed on each stage but because they are on different platforms I'm having difficulty.
linux bash ssh jupyter-notebook
linux bash ssh jupyter-notebook
asked Feb 12 at 11:59
gregor114gregor114
1
1
closed as too broad by music2myear, Mike Fitzpatrick, Debra, bertieb, DrMoishe Pippik Feb 17 at 4:07
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
closed as too broad by music2myear, Mike Fitzpatrick, Debra, bertieb, DrMoishe Pippik Feb 17 at 4:07
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
script basically (very basic) are a list of command, potentially if you run ssh connection -> cd to the folder -> command execution, you could create a file contain some think like this:
#!/bin/bash
ssh user@server -x "cd /path/to && mycommand"
and so on.
add a comment |
Just to expand on AtomiX84's answer, your final script only really needs to have two lines in it.
This answer assumes you're using bash. If you haven't done any bash scripting before, have a quick look over this guide.
As demonstrated above, the SSH command on its own can supply multiple commands to a server then terminate the connection automatically when it's finished:
ssh host command; command /full/path/to/file; command
Note that with this example and with scripting in general, it's not common practice to change directories to reach a file or executable within a script. Instead, you'll specify an absolute path for the file/exectuable/image you want to make use of.
You should use docker run instead of using the docker CLI since things can get complicated when trying to automate a subshell. You should opt for a single-use command that references the absolute path of whatever jupyter image file you need. You should make sure your docker run
command is run in the background with the -d
option to run it in detached mode. You may also have to use the --rm
option with this so the daemon doesn't stop when your SSH connection terminates. That way you'll retain shell control and your SSH command can safely terminate and your script can go onto the next step of connecting to the notebook.
I haven't done much work directly with Jupyter, so I pulled the docker run command from here. Hopefully, this describes your setup. Your final SSH command will look something like:
#!/bin/bash
ssh server-host "docker run -p -d --rm 8888:8888 jupyter/scipy-notebook:2c80cf3537ca"
ssh notebook-host
The first ssh command should automatically terminate when it spins up the server, then the second ssh command will connect you to your notebook. At the end of your work day, you can ssh back into the docker machine and run docker attach
to return the server to the foreground.
add a comment |
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
script basically (very basic) are a list of command, potentially if you run ssh connection -> cd to the folder -> command execution, you could create a file contain some think like this:
#!/bin/bash
ssh user@server -x "cd /path/to && mycommand"
and so on.
add a comment |
script basically (very basic) are a list of command, potentially if you run ssh connection -> cd to the folder -> command execution, you could create a file contain some think like this:
#!/bin/bash
ssh user@server -x "cd /path/to && mycommand"
and so on.
add a comment |
script basically (very basic) are a list of command, potentially if you run ssh connection -> cd to the folder -> command execution, you could create a file contain some think like this:
#!/bin/bash
ssh user@server -x "cd /path/to && mycommand"
and so on.
script basically (very basic) are a list of command, potentially if you run ssh connection -> cd to the folder -> command execution, you could create a file contain some think like this:
#!/bin/bash
ssh user@server -x "cd /path/to && mycommand"
and so on.
answered Feb 12 at 12:04
AtomiX84AtomiX84
4779
4779
add a comment |
add a comment |
Just to expand on AtomiX84's answer, your final script only really needs to have two lines in it.
This answer assumes you're using bash. If you haven't done any bash scripting before, have a quick look over this guide.
As demonstrated above, the SSH command on its own can supply multiple commands to a server then terminate the connection automatically when it's finished:
ssh host command; command /full/path/to/file; command
Note that with this example and with scripting in general, it's not common practice to change directories to reach a file or executable within a script. Instead, you'll specify an absolute path for the file/exectuable/image you want to make use of.
You should use docker run instead of using the docker CLI since things can get complicated when trying to automate a subshell. You should opt for a single-use command that references the absolute path of whatever jupyter image file you need. You should make sure your docker run
command is run in the background with the -d
option to run it in detached mode. You may also have to use the --rm
option with this so the daemon doesn't stop when your SSH connection terminates. That way you'll retain shell control and your SSH command can safely terminate and your script can go onto the next step of connecting to the notebook.
I haven't done much work directly with Jupyter, so I pulled the docker run command from here. Hopefully, this describes your setup. Your final SSH command will look something like:
#!/bin/bash
ssh server-host "docker run -p -d --rm 8888:8888 jupyter/scipy-notebook:2c80cf3537ca"
ssh notebook-host
The first ssh command should automatically terminate when it spins up the server, then the second ssh command will connect you to your notebook. At the end of your work day, you can ssh back into the docker machine and run docker attach
to return the server to the foreground.
add a comment |
Just to expand on AtomiX84's answer, your final script only really needs to have two lines in it.
This answer assumes you're using bash. If you haven't done any bash scripting before, have a quick look over this guide.
As demonstrated above, the SSH command on its own can supply multiple commands to a server then terminate the connection automatically when it's finished:
ssh host command; command /full/path/to/file; command
Note that with this example and with scripting in general, it's not common practice to change directories to reach a file or executable within a script. Instead, you'll specify an absolute path for the file/exectuable/image you want to make use of.
You should use docker run instead of using the docker CLI since things can get complicated when trying to automate a subshell. You should opt for a single-use command that references the absolute path of whatever jupyter image file you need. You should make sure your docker run
command is run in the background with the -d
option to run it in detached mode. You may also have to use the --rm
option with this so the daemon doesn't stop when your SSH connection terminates. That way you'll retain shell control and your SSH command can safely terminate and your script can go onto the next step of connecting to the notebook.
I haven't done much work directly with Jupyter, so I pulled the docker run command from here. Hopefully, this describes your setup. Your final SSH command will look something like:
#!/bin/bash
ssh server-host "docker run -p -d --rm 8888:8888 jupyter/scipy-notebook:2c80cf3537ca"
ssh notebook-host
The first ssh command should automatically terminate when it spins up the server, then the second ssh command will connect you to your notebook. At the end of your work day, you can ssh back into the docker machine and run docker attach
to return the server to the foreground.
add a comment |
Just to expand on AtomiX84's answer, your final script only really needs to have two lines in it.
This answer assumes you're using bash. If you haven't done any bash scripting before, have a quick look over this guide.
As demonstrated above, the SSH command on its own can supply multiple commands to a server then terminate the connection automatically when it's finished:
ssh host command; command /full/path/to/file; command
Note that with this example and with scripting in general, it's not common practice to change directories to reach a file or executable within a script. Instead, you'll specify an absolute path for the file/exectuable/image you want to make use of.
You should use docker run instead of using the docker CLI since things can get complicated when trying to automate a subshell. You should opt for a single-use command that references the absolute path of whatever jupyter image file you need. You should make sure your docker run
command is run in the background with the -d
option to run it in detached mode. You may also have to use the --rm
option with this so the daemon doesn't stop when your SSH connection terminates. That way you'll retain shell control and your SSH command can safely terminate and your script can go onto the next step of connecting to the notebook.
I haven't done much work directly with Jupyter, so I pulled the docker run command from here. Hopefully, this describes your setup. Your final SSH command will look something like:
#!/bin/bash
ssh server-host "docker run -p -d --rm 8888:8888 jupyter/scipy-notebook:2c80cf3537ca"
ssh notebook-host
The first ssh command should automatically terminate when it spins up the server, then the second ssh command will connect you to your notebook. At the end of your work day, you can ssh back into the docker machine and run docker attach
to return the server to the foreground.
Just to expand on AtomiX84's answer, your final script only really needs to have two lines in it.
This answer assumes you're using bash. If you haven't done any bash scripting before, have a quick look over this guide.
As demonstrated above, the SSH command on its own can supply multiple commands to a server then terminate the connection automatically when it's finished:
ssh host command; command /full/path/to/file; command
Note that with this example and with scripting in general, it's not common practice to change directories to reach a file or executable within a script. Instead, you'll specify an absolute path for the file/exectuable/image you want to make use of.
You should use docker run instead of using the docker CLI since things can get complicated when trying to automate a subshell. You should opt for a single-use command that references the absolute path of whatever jupyter image file you need. You should make sure your docker run
command is run in the background with the -d
option to run it in detached mode. You may also have to use the --rm
option with this so the daemon doesn't stop when your SSH connection terminates. That way you'll retain shell control and your SSH command can safely terminate and your script can go onto the next step of connecting to the notebook.
I haven't done much work directly with Jupyter, so I pulled the docker run command from here. Hopefully, this describes your setup. Your final SSH command will look something like:
#!/bin/bash
ssh server-host "docker run -p -d --rm 8888:8888 jupyter/scipy-notebook:2c80cf3537ca"
ssh notebook-host
The first ssh command should automatically terminate when it spins up the server, then the second ssh command will connect you to your notebook. At the end of your work day, you can ssh back into the docker machine and run docker attach
to return the server to the foreground.
answered Feb 12 at 21:05
baelxbaelx
1,919817
1,919817
add a comment |
add a comment |