CURL & WGET and how download the Internet

Curl

“curl is used in command lines or scripts to transfer data. curl is also used in cars, television sets, routers, printers, audio equipment, mobile phones, tablets, media players and is the Internet transfer engine for thousands of software applications in over ten billion installations.”

curl [OPTIONS] https://myawesomewebsite.com/datafile1.txt

Options:

• -b : Go to background immediately after startup
• -q : Turn off the output
• -c : resume broken download

Curl: download all the file in a text file

Example with my file called url_list.txt

> cat url_list.txt
https://myawesomewebsite.com/datafile1.txt
https://myawesomewebsite.com/datafile2.txt
https://myawesomewebsite.com/datafile2.txt
> wget -i url_list.txt   # download all the files

Troubleshooting curl

Curl has two prticulary useful option flag in case of timeouts during downloads