CURL Cheat-Sheet
Curl is a command line tool for doing all sorts of URL manipulations and transfers. The client, curl, sends an HTTP request. The request contains a method (like GET, POST, HEAD, etc), a number of request headers and sometimes a request body. The HTTP server responds with a status line (indicating if things went well), response headers and most often also a response body. The “body” part is the plain data you requested, like the actual HTML or the image, etc.
curl — verbose http://www.example.com
[Displays request header, response header, response]
OR
curl -v http://www.example.com [Shorthand]
OR
curl http://www.example.com
[Returns back the webpage in response to the GET request]
OR
curl — include http://www.example.com
[Complete header information as well]
OR
curl -I http://www.example.com [Shorthand]
curl — trace-ascii hello.txt http://www.example.com
[To display and save all the request response on to the file]
curl — trace-ascii hello.txt — trace-time http://www.example.com
[Display all information along with timestamp information]
curl http://www.example.com -o hello.txt
[Displays only the response and no header information and saves it into the hello.txt file]
curl http://www.example.com — resolve www.example.com:80:127.0.0.1
[ — resolve [DOMAIN]:[PORT]:[IP], that routes all web requests performed during the execution of a cURL command that match a given [DOMAIN] and [PORT] to a specified [IP] address. ]
curl http://www.example.com:8099/
[To make a request to a particular port number]
curl — proxy http://proxy.example.com:4321 http://remote.example.com
[The port number you specify in the URL is the number that the server uses to offer its services. Sometimes you may use a local proxy, and then you may need to specify that proxy’s port number separately for what curl needs to connect to locally. Like when using an HTTP proxy on port 4321]
curl http://user:password@example.org
[To insert the user and password in the URL]
OR
curl -u user:password http://example.org
[To provide the user and password separately]
OR
curl — user name:password http://example.com
curl — proxy-user proxyusername:proxypassword http://example.com
[Proxy Login]
Note: If you use any one of these user+password options but leave out the password part, curl will prompt for the password interactively.
curl http://www.example.com — head
[To make a HEAD request similar to GET request, the server returns back the headers but without a body]
curl http://url1.example.com http://url2.example.com
[Request to multiple URLs]
curl — data name=rahul http://url1.example.com http://url2.example.com
[Supplies the same parameter to both URLs while making the POST call]
curl -I http://example.com — next https://google.com
[First a HEAD and then a GET]
curl -d [ — data] score=10 http://example.com/post — next http://google.com
[POST score on URL1 then GET]
curl http://example.com?data1=1905&age=24
[To post data via GET request]
curl — data “data1=1905$age=%2024%20” http://example.com
[To post data via POST request] [Post hidden fields also in the same way]
This kind of POST will use the Content-Type application/x-www-form-urlencoded and is the most widely used POST kind
curl — data-urlencode “name=I am Rahul Sharma” http://example.com
[To encode the POST data]
Note: If you repeat — data several times on the command line, curl will concatenate all the given data pieces — and put a ‘&’ symbol between each data segment.
curl — form upload=@filename — form press=OK http://example.com
[To upload file via form post à Content-type:multipart/form-data]
curl — upload-file filename http://example.com
[File upload using PUT request]
curl — referer http://example1.com http://example.com
[A HTTP request may include a ‘referer’ field (yes it is misspelled), which can be used to tell from which URL the client got to this particular resource. Some programs/scripts check the referer field of requests to verify that this wasn’t arriving from an external site or an unknown page. While this is a stupid way to check something so easily forged, many scripts still do it. Using curl, you can put anything you want in the referer-field and thus more easily be able to fool the server into serving your request.]
curl — user-agent “Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)” http://example.com
[Fool server by looking like Mozilla]
curl — location http://example.com
[Allow redirections]
Note: If you use curl to POST to a site that immediately redirects you to another page, you can safely use — location (-L) and — data/ — form together. Curl will only use POST in the first request, and then revert to GET in the following operations.
Cookie Basics
The way the web browsers do “client side state control” is by using cookies. Cookies are just names with associated contents. The cookies are sent to the client by the server. The server tells the client for what path and hostname it wants the cookie sent back, and it also sends an expiration date and a few more properties.
When a client communicates with a server with a name and path as previously specified in a received cookie, the client sends back the cookies and their contents to the server, unless of course they are expired.
Many applications and servers use this method to connect a series of requests into a single logical session. To be able to use curl in such occasions, we must be able to record and send back cookies the way the web application expects them. The same way browsers deal with them.
curl — cookie “name=Daniel” http://www.example.com
[Send cookie to the server]
curl — dump-header headers_and_cookies http://www.example.com
[To dump (store) cookies]
curl — cookie stored_cookies_in_file http://www.example.com
[Curl has a full-blown cookie parsing engine built-in that comes useful if you want to reconnect to a server and use cookies that were stored from a previous connection (or hand-crafted manually to fool the server into believing you had a previous connection)]
curl — cookie cookies.txt — cookie-jar newcookies.txt http://www.example.com
[Curl has the ability to read and write cookie files that use the same file format that Netscape and Mozilla once used. It is a convenient way to share cookies between scripts or invokes. The — cookie (-b) switch automatically detects if a given file is such a cookie file and parses it, and by using the — cookie-jar (-c) option you’ll make curl write a new cookie file at the end of an operation]
curl — data “<xml>” — header “Content-Type: text/xml” — request PROPFIND url.com
[For example, you can change the POST request to a PROPFIND and send the data as “Content-Type: text/xml” (instead of the default Content-Type)]
curl — header “Host:” http://www.example.com
[delete a default header by providing one without content]
curl — header “Destination:https://google.com" http://www.example.com
[Custom headers]
curl -X POST http://example.org/
[curl will still think and act as if it sent a GET so it won’t send any request body etc].
Thanks.
