wget for links inside html pages
$begingroup$
I am trying to download a file from the following repository: https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR7276474
As you can see, there are several layers to the webpage. For example, clicking on the download tab doesn't change the URL and, the link is not a 'download link' per ce - where simply clicking the link automates a download. I have tried some of the answers on the forum, where they have advised using quotes and the operation:
wget "url/?target=link". This however does not work in the following instance.
Basically, I want to download the file labeled 'P1TLH.bam', in the download tab of the link provided:
ncbi
edited Feb 4 at 15:40
terdon♦
4,4001730
asked Feb 4 at 15:08
h3ab74h3ab74
1067
$endgroup$
migrated from askubuntu.com Feb 4 at 15:33
This question came from our site for Ubuntu users and developers.
add a comment |
$begingroup$
I am trying to download a file from the following repository: https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR7276474
As you can see, there are several layers to the webpage. For example, clicking on the download tab doesn't change the URL and, the link is not a 'download link' per ce - where simply clicking the link automates a download. I have tried some of the answers on the forum, where they have advised using quotes and the operation:
wget "url/?target=link". This however does not work in the following instance.
Basically, I want to download the file labeled 'P1TLH.bam', in the download tab of the link provided:
ncbi
edited Feb 4 at 15:40
terdon♦
4,4001730
asked Feb 4 at 15:08
h3ab74h3ab74
1067
$endgroup$
migrated from askubuntu.com Feb 4 at 15:33
This question came from our site for Ubuntu users and developers.
$begingroup$
Thanks, found it. But doesn'twget https://sra-download.ncbi.nlm.nih.gov/traces/sra64/SRZ/007276/SRR7276474/P1TLH.bamwork?
$endgroup$
– terdon♦
Feb 4 at 15:40
$begingroup$
It does! Thank you so much. What is the logic behind it though? Don't quite understand why the original command was not working..
$endgroup$
– h3ab74
Feb 4 at 15:45
$begingroup$
Because you weren't using the link to that file, presumably. Wget will download what you tell it to, so you need to tell it to get the target of the link you want.
$endgroup$
– terdon♦
Feb 4 at 15:48
add a comment |
$begingroup$
I am trying to download a file from the following repository: https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR7276474
As you can see, there are several layers to the webpage. For example, clicking on the download tab doesn't change the URL and, the link is not a 'download link' per ce - where simply clicking the link automates a download. I have tried some of the answers on the forum, where they have advised using quotes and the operation:
wget "url/?target=link". This however does not work in the following instance.
Basically, I want to download the file labeled 'P1TLH.bam', in the download tab of the link provided:
ncbi
edited Feb 4 at 15:40
terdon♦
4,4001730
asked Feb 4 at 15:08
h3ab74h3ab74
1067
$endgroup$
I am trying to download a file from the following repository: https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR7276474
As you can see, there are several layers to the webpage. For example, clicking on the download tab doesn't change the URL and, the link is not a 'download link' per ce - where simply clicking the link automates a download. I have tried some of the answers on the forum, where they have advised using quotes and the operation:
wget "url/?target=link". This however does not work in the following instance.
Basically, I want to download the file labeled 'P1TLH.bam', in the download tab of the link provided:
ncbi
ncbi
edited Feb 4 at 15:40
terdon♦
4,4001730
asked Feb 4 at 15:08
h3ab74h3ab74
1067
edited Feb 4 at 15:40
terdon♦
4,4001730
asked Feb 4 at 15:08
h3ab74h3ab74
1067
edited Feb 4 at 15:40
terdon♦
4,4001730
edited Feb 4 at 15:40
terdon♦
4,4001730
edited Feb 4 at 15:40
terdon♦
4,4001730
4,4001730
asked Feb 4 at 15:08
h3ab74h3ab74
1067
asked Feb 4 at 15:08
h3ab74h3ab74
1067
asked Feb 4 at 15:08
h3ab74h3ab74
1067
1067
migrated from askubuntu.com Feb 4 at 15:33
This question came from our site for Ubuntu users and developers.
migrated from askubuntu.com Feb 4 at 15:33
This question came from our site for Ubuntu users and developers.
$begingroup$
Thanks, found it. But doesn'twget https://sra-download.ncbi.nlm.nih.gov/traces/sra64/SRZ/007276/SRR7276474/P1TLH.bamwork?
$endgroup$
– terdon♦
Feb 4 at 15:40
$begingroup$
It does! Thank you so much. What is the logic behind it though? Don't quite understand why the original command was not working..
$endgroup$
– h3ab74
Feb 4 at 15:45
$begingroup$
Because you weren't using the link to that file, presumably. Wget will download what you tell it to, so you need to tell it to get the target of the link you want.
$endgroup$
– terdon♦
Feb 4 at 15:48
add a comment |
$begingroup$
Thanks, found it. But doesn'twget https://sra-download.ncbi.nlm.nih.gov/traces/sra64/SRZ/007276/SRR7276474/P1TLH.bamwork?
$endgroup$
– terdon♦
Feb 4 at 15:40
$begingroup$
It does! Thank you so much. What is the logic behind it though? Don't quite understand why the original command was not working..
$endgroup$
– h3ab74
Feb 4 at 15:45
$begingroup$
Because you weren't using the link to that file, presumably. Wget will download what you tell it to, so you need to tell it to get the target of the link you want.
$endgroup$
– terdon♦
Feb 4 at 15:48
$begingroup$
Thanks, found it. But doesn't
wget https://sra-download.ncbi.nlm.nih.gov/traces/sra64/SRZ/007276/SRR7276474/P1TLH.bam work?$endgroup$
– terdon♦
Feb 4 at 15:40
$begingroup$
Thanks, found it. But doesn't
wget https://sra-download.ncbi.nlm.nih.gov/traces/sra64/SRZ/007276/SRR7276474/P1TLH.bam work?$endgroup$
– terdon♦
Feb 4 at 15:40
$begingroup$
It does! Thank you so much. What is the logic behind it though? Don't quite understand why the original command was not working..
$endgroup$
– h3ab74
Feb 4 at 15:45
$begingroup$
It does! Thank you so much. What is the logic behind it though? Don't quite understand why the original command was not working..
$endgroup$
– h3ab74
Feb 4 at 15:45
$begingroup$
Because you weren't using the link to that file, presumably. Wget will download what you tell it to, so you need to tell it to get the target of the link you want.
$endgroup$
– terdon♦
Feb 4 at 15:48
$begingroup$
Because you weren't using the link to that file, presumably. Wget will download what you tell it to, so you need to tell it to get the target of the link you want.
$endgroup$
– terdon♦
Feb 4 at 15:48
add a comment |
2 Answers
2
active
oldest
votes
$begingroup$
If you want to download a file, you need to use the link to that file. Your original attempt, wget https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR7276474 wouldn't work since that's a link to the trace page of the relevant run. If you want to download something else, just right click on the link (the one in the screenshot in your question), copy the URL and use that:
wget https://sra-download.ncbi.nlm.nih.gov/traces/sra64/SRZ/007276/SRR7276474/P1TLH.bam
answered Feb 4 at 15:48
terdon♦terdon
4,4001730
$endgroup$
$begingroup$
I see, thanks again!
$endgroup$
– h3ab74
Feb 4 at 15:52
add a comment |
$begingroup$
NCBI has its own digger for this purpose. No, my bad the traditional Perl approach of LWP and HTTP::Request::Common to interrogate NCBI, isn't going to work no how much data munging or spider crawling is done, because its just an NGS data dump. ... well its diggable, but it would be easier to dump the data and interrogate it locally.
answered Feb 4 at 16:33
Michael G.Michael G.
6821216
$endgroup$
2
$begingroup$
Please post the link and an explanation of how to use it, yes. Without it, this isn't really an answer. With it, it would become a very useful answer, much more so than my own.
$endgroup$
– terdon♦
Feb 4 at 20:16
$begingroup$
Well it obviously isn't! :0) Its in Perl, so no-one understands that now
$endgroup$
– Michael G.
Feb 5 at 17:12
$begingroup$
Oy! Nothing wrong with Perl! Just because these young whippersnappers don't get its beauty, doesn't mean it isn't a great language! Joking apart, please edit the answer so it can stand on its own or just delete and post a comment instead. As you see, it's already attracted a downvote (not mine).
$endgroup$
– terdon♦
Feb 5 at 17:22
$begingroup$
No, I just didn't get around to doing it and when I get around to it I'll update the post
$endgroup$
– Michael G.
Feb 5 at 17:42
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "676"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fbioinformatics.stackexchange.com%2fquestions%2f6949%2fwget-for-links-inside-html-pages%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
If you want to download a file, you need to use the link to that file. Your original attempt, wget https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR7276474 wouldn't work since that's a link to the trace page of the relevant run. If you want to download something else, just right click on the link (the one in the screenshot in your question), copy the URL and use that:
wget https://sra-download.ncbi.nlm.nih.gov/traces/sra64/SRZ/007276/SRR7276474/P1TLH.bam
answered Feb 4 at 15:48
terdon♦terdon
4,4001730
$endgroup$
$begingroup$
I see, thanks again!
$endgroup$
– h3ab74
Feb 4 at 15:52
add a comment |
$begingroup$
If you want to download a file, you need to use the link to that file. Your original attempt, wget https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR7276474 wouldn't work since that's a link to the trace page of the relevant run. If you want to download something else, just right click on the link (the one in the screenshot in your question), copy the URL and use that:
wget https://sra-download.ncbi.nlm.nih.gov/traces/sra64/SRZ/007276/SRR7276474/P1TLH.bam
answered Feb 4 at 15:48
terdon♦terdon
4,4001730
$endgroup$
$begingroup$
I see, thanks again!
$endgroup$
– h3ab74
Feb 4 at 15:52
add a comment |
$begingroup$
If you want to download a file, you need to use the link to that file. Your original attempt, wget https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR7276474 wouldn't work since that's a link to the trace page of the relevant run. If you want to download something else, just right click on the link (the one in the screenshot in your question), copy the URL and use that:
wget https://sra-download.ncbi.nlm.nih.gov/traces/sra64/SRZ/007276/SRR7276474/P1TLH.bam
answered Feb 4 at 15:48
terdon♦terdon
4,4001730
$endgroup$
If you want to download a file, you need to use the link to that file. Your original attempt, wget https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR7276474 wouldn't work since that's a link to the trace page of the relevant run. If you want to download something else, just right click on the link (the one in the screenshot in your question), copy the URL and use that:
wget https://sra-download.ncbi.nlm.nih.gov/traces/sra64/SRZ/007276/SRR7276474/P1TLH.bam
answered Feb 4 at 15:48
terdon♦terdon
4,4001730
answered Feb 4 at 15:48
terdon♦terdon
4,4001730
answered Feb 4 at 15:48
terdon♦terdon
4,4001730
answered Feb 4 at 15:48
terdon♦terdon
4,4001730
4,4001730
$begingroup$
I see, thanks again!
$endgroup$
– h3ab74
Feb 4 at 15:52
add a comment |
$begingroup$
I see, thanks again!
$endgroup$
– h3ab74
Feb 4 at 15:52
$begingroup$
I see, thanks again!
$endgroup$
– h3ab74
Feb 4 at 15:52
$begingroup$
I see, thanks again!
$endgroup$
– h3ab74
Feb 4 at 15:52
add a comment |
$begingroup$
NCBI has its own digger for this purpose. No, my bad the traditional Perl approach of LWP and HTTP::Request::Common to interrogate NCBI, isn't going to work no how much data munging or spider crawling is done, because its just an NGS data dump. ... well its diggable, but it would be easier to dump the data and interrogate it locally.
answered Feb 4 at 16:33
Michael G.Michael G.
6821216
$endgroup$
2
$begingroup$
Please post the link and an explanation of how to use it, yes. Without it, this isn't really an answer. With it, it would become a very useful answer, much more so than my own.
$endgroup$
– terdon♦
Feb 4 at 20:16
$begingroup$
Well it obviously isn't! :0) Its in Perl, so no-one understands that now
$endgroup$
– Michael G.
Feb 5 at 17:12
$begingroup$
Oy! Nothing wrong with Perl! Just because these young whippersnappers don't get its beauty, doesn't mean it isn't a great language! Joking apart, please edit the answer so it can stand on its own or just delete and post a comment instead. As you see, it's already attracted a downvote (not mine).
$endgroup$
– terdon♦
Feb 5 at 17:22
$begingroup$
No, I just didn't get around to doing it and when I get around to it I'll update the post
$endgroup$
– Michael G.
Feb 5 at 17:42
add a comment |
$begingroup$
NCBI has its own digger for this purpose. No, my bad the traditional Perl approach of LWP and HTTP::Request::Common to interrogate NCBI, isn't going to work no how much data munging or spider crawling is done, because its just an NGS data dump. ... well its diggable, but it would be easier to dump the data and interrogate it locally.
answered Feb 4 at 16:33
Michael G.Michael G.
6821216
$endgroup$
2
$begingroup$
Please post the link and an explanation of how to use it, yes. Without it, this isn't really an answer. With it, it would become a very useful answer, much more so than my own.
$endgroup$
– terdon♦
Feb 4 at 20:16
$begingroup$
Well it obviously isn't! :0) Its in Perl, so no-one understands that now
$endgroup$
– Michael G.
Feb 5 at 17:12
$begingroup$
Oy! Nothing wrong with Perl! Just because these young whippersnappers don't get its beauty, doesn't mean it isn't a great language! Joking apart, please edit the answer so it can stand on its own or just delete and post a comment instead. As you see, it's already attracted a downvote (not mine).
$endgroup$
– terdon♦
Feb 5 at 17:22
$begingroup$
No, I just didn't get around to doing it and when I get around to it I'll update the post
$endgroup$
– Michael G.
Feb 5 at 17:42
add a comment |
$begingroup$
NCBI has its own digger for this purpose. No, my bad the traditional Perl approach of LWP and HTTP::Request::Common to interrogate NCBI, isn't going to work no how much data munging or spider crawling is done, because its just an NGS data dump. ... well its diggable, but it would be easier to dump the data and interrogate it locally.
answered Feb 4 at 16:33
Michael G.Michael G.
6821216
$endgroup$
NCBI has its own digger for this purpose. No, my bad the traditional Perl approach of LWP and HTTP::Request::Common to interrogate NCBI, isn't going to work no how much data munging or spider crawling is done, because its just an NGS data dump. ... well its diggable, but it would be easier to dump the data and interrogate it locally.
answered Feb 4 at 16:33
Michael G.Michael G.
6821216
edited Feb 6 at 16:05
answered Feb 4 at 16:33
Michael G.Michael G.
6821216
answered Feb 4 at 16:33
Michael G.Michael G.
6821216
answered Feb 4 at 16:33
Michael G.Michael G.
6821216
6821216
2
$begingroup$
Please post the link and an explanation of how to use it, yes. Without it, this isn't really an answer. With it, it would become a very useful answer, much more so than my own.
$endgroup$
– terdon♦
Feb 4 at 20:16
$begingroup$
Well it obviously isn't! :0) Its in Perl, so no-one understands that now
$endgroup$
– Michael G.
Feb 5 at 17:12
$begingroup$
Oy! Nothing wrong with Perl! Just because these young whippersnappers don't get its beauty, doesn't mean it isn't a great language! Joking apart, please edit the answer so it can stand on its own or just delete and post a comment instead. As you see, it's already attracted a downvote (not mine).
$endgroup$
– terdon♦
Feb 5 at 17:22
$begingroup$
No, I just didn't get around to doing it and when I get around to it I'll update the post
$endgroup$
– Michael G.
Feb 5 at 17:42
add a comment |
2
$begingroup$
Please post the link and an explanation of how to use it, yes. Without it, this isn't really an answer. With it, it would become a very useful answer, much more so than my own.
$endgroup$
– terdon♦
Feb 4 at 20:16
$begingroup$
Well it obviously isn't! :0) Its in Perl, so no-one understands that now
$endgroup$
– Michael G.
Feb 5 at 17:12
$begingroup$
Oy! Nothing wrong with Perl! Just because these young whippersnappers don't get its beauty, doesn't mean it isn't a great language! Joking apart, please edit the answer so it can stand on its own or just delete and post a comment instead. As you see, it's already attracted a downvote (not mine).
$endgroup$
– terdon♦
Feb 5 at 17:22
$begingroup$
No, I just didn't get around to doing it and when I get around to it I'll update the post
$endgroup$
– Michael G.
Feb 5 at 17:42
2
2
$begingroup$
Please post the link and an explanation of how to use it, yes. Without it, this isn't really an answer. With it, it would become a very useful answer, much more so than my own.
$endgroup$
– terdon♦
Feb 4 at 20:16
$begingroup$
Please post the link and an explanation of how to use it, yes. Without it, this isn't really an answer. With it, it would become a very useful answer, much more so than my own.
$endgroup$
– terdon♦
Feb 4 at 20:16
$begingroup$
Well it obviously isn't! :0) Its in Perl, so no-one understands that now
$endgroup$
– Michael G.
Feb 5 at 17:12
$begingroup$
Well it obviously isn't! :0) Its in Perl, so no-one understands that now
$endgroup$
– Michael G.
Feb 5 at 17:12
$begingroup$
Oy! Nothing wrong with Perl! Just because these young whippersnappers don't get its beauty, doesn't mean it isn't a great language! Joking apart, please edit the answer so it can stand on its own or just delete and post a comment instead. As you see, it's already attracted a downvote (not mine).
$endgroup$
– terdon♦
Feb 5 at 17:22
$begingroup$
Oy! Nothing wrong with Perl! Just because these young whippersnappers don't get its beauty, doesn't mean it isn't a great language! Joking apart, please edit the answer so it can stand on its own or just delete and post a comment instead. As you see, it's already attracted a downvote (not mine).
$endgroup$
– terdon♦
Feb 5 at 17:22
$begingroup$
No, I just didn't get around to doing it and when I get around to it I'll update the post
$endgroup$
– Michael G.
Feb 5 at 17:42
$begingroup$
No, I just didn't get around to doing it and when I get around to it I'll update the post
$endgroup$
– Michael G.
Feb 5 at 17:42
add a comment |
Thanks for contributing an answer to Bioinformatics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fbioinformatics.stackexchange.com%2fquestions%2f6949%2fwget-for-links-inside-html-pages%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
$begingroup$
Thanks, found it. But doesn't
wget https://sra-download.ncbi.nlm.nih.gov/traces/sra64/SRZ/007276/SRR7276474/P1TLH.bamwork?$endgroup$
– terdon♦
Feb 4 at 15:40
$begingroup$
It does! Thank you so much. What is the logic behind it though? Don't quite understand why the original command was not working..
$endgroup$
– h3ab74
Feb 4 at 15:45
$begingroup$
Because you weren't using the link to that file, presumably. Wget will download what you tell it to, so you need to tell it to get the target of the link you want.
$endgroup$
– terdon♦
Feb 4 at 15:48