joshtronic
Making network requests from the command-line comes in a variety of shapes and sizes. Some folks swear by curl
others love wget
. For me, I love the lwp-request
package which provides familiarly named GET
, POST
and HEAD
commands. The third of the list, HEAD
, is something I use quite regularly to troubleshoot the headers returned by servers.
The commands we’re going to use are provided by a package called lwp-request
.
The package in question is written in Perl and tends to come standard with quite a few Linux distributions as well as other Unix-like operating systems like macOS.
The lwp-request
command could be invoked directly with a series of arguments, but we’ll be going over the helper scripts that it comes with for the sake of simplicity.
If you are lacking GET
, POST
and HEAD
on your system, you most likely will be able to install the lwp-request
package from your favorite package manager.
On Ubuntu and Debian derived systems, you can install it using the following commands:
- sudo apt update
- sudo apt install libwww-perl
On CentOS, Rocky Linux, Fedora, and other RedHat derived systems, you can install it with support for HTTPS URLs using the following command:
- sudo dnf install perl-libwww-perl.noarch perl-LWP-Protocol-https.noarch
GET
Let’s say you loved the Alligator.io logo so much, you wanted to download it locally. It’s a pretty amazing logo, who wouldn’t want their own personal copy?
To GET
the file, you can simply run:
- GET https://alligator.io/images/logo-fancy.svg
Not so fast! All that does is show a bunch of SVG markup.
In true Unix Philosophy fashion, the GET
command does one thing really well and that’s GET
ting a file.
This is great when you want to check a URL to see what’s being returned by the web server, but if you want to truly download that lovely logo, you’ll want to send the output to a file:
- GET https://alligator.io/images/logo-fancy.svg > logo-fancy.svg
Now we have that fantastic Alligator.io logo downloaded to a local file!
POST
The GET
command allows us to consume files from remote servers while POST
allows us to send data to a server for processing as well as return it’s output.
At it’s very least, the syntax for POST
is the same as GET
:
- POST https://httpbin.org/post
This will then prompt you for the content you’d like to POST
. The string expected should be in a query string format that looks something like this:
reptile=alligator&color=#008f68
When you’re done entering in your content, simply hit CTRL-D
and the content will be POST
ed. The service we’re posting to will mirror back the request:
{
"args": {},
"data": "",
"files": {},
"form": {
"color": "#008f68\n",
"reptile": "alligator"
},
"headers": {
"Content-Length": "32",
"Content-Type": "application/x-www-form-urlencoded",
"Host": "httpbin.org",
"User-Agent": "lwp-request/6.39 libwww-perl/6.39"
},
"json": null,
"origin": "203.0.113.5",
"url": "https://httpbin.org/post"
}
HEAD
As mentioned, HEAD
is not only extremely useful for debugging and troubleshooting, but I’m pretty sure it’s in my top 5 of all-time favorite command-line utilities.
Similar to GET
and POST
, the syntax for HEAD
is quite minimal:
- HEAD http://alligator.io/
This will return a 200 OK
and information about the headers returned by the web service.
Unfortunately, this isn’t quite right, because we serve up Alligator.io over HTTPS like the good security-minded reptilian webizens that we are.
The HEAD
command by default only gives you information about the last stop in the request chain. To see all of the requests, including the automatic 301 Moved Permanently
, pass in the -S
argument:
- HEAD -S http://alligator.io/
Which gives us a bit more insight:
OutputHEAD http://alligator.io/
301 Moved Permanently
HEAD https://alligator.io/
200 OK
Cache-Control: public, max-age=0, must-revalidate
Connection: close
Date: Sat, 29 Jun 2019 00:49:18 GMT
Age: 1
ETag: "8b85849c835909679fc1ba80b307d144-ssl"
Server: Netlify
Content-Length: 0
Content-Type: text/html; charset=UTF-8
Client-Date: Sat, 29 Jun 2019 00:49:18 GMT
Client-Peer: 203.0.113.1:443
Client-Response-Num: 1
Client-SSL-Cert-Issuer: /C=US/O=Let's Encrypt/CN=Let's Encrypt Authority X3
Client-SSL-Cert-Subject: /CN=alligator.io
Client-SSL-Cipher: TLS_AES_256_GCM_SHA384
Client-SSL-Socket-Class: IO::Socket::SSL
Strict-Transport-Security: max-age=31536000
X-NF-Request-ID: 60babe56-c0ea-4658-aa5a-3e185f1e851f-10342
Monochromatic output got you down? If so, you could alias HTTPie’s http
command to GET
, POST
and HEAD
.
HTTPie can do everything the lwp-request
library does, with similar syntax, with the added bonus of colorful output.
On Ubuntu and Debian derived systems you can install HTTPie with the following commands:
- sudo apt update
- sudo apt install httpie
On Centos, Rocky Linux, Fedora, and RedHat derived distributions you can install HTTPie using the following commands, as long as you have the EPEL :
- sudo dnf install epel-release
- sudo dnf install httpie
My local aliases look like this:
- # HTTPie aliases
- alias GET='http'
- alias POST='http POST'
- alias HEAD='http HEAD'
Next time you need to make a network request to an API or are troubleshooting the headers returned by a server, you can leave Postman and similar tools at the door.
You can omit your browser entirely, too!
Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.
While we believe that this content benefits our community, we have not yet thoroughly reviewed it. If you have any suggestions for improvements, please let us know by clicking the “report an issue“ button at the bottom of the tutorial.
This textbox defaults to using Markdown to format your answer.
You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!