There are several ways to control and communicate with data in the Linux terminal, and curl is the best way to do that. curl is a command-line tool that you can use for transferring data from or to a server. You can use the curl command for downloading and uploading data using any supported protocols, including FTP, HTTP,  SFTP, HTTPS, and SCP. Curl has many features, including the ability to restrict bandwidth, resume transfers, user authentication, support for proxy servers, and much more.

 Below is the list of protocols currently supported by the curl:

• FTP/FTPS
• Gopher
• HTTP
• HTTP/2
• SMTP/SMTPS
• IMAP/IMAPS
• SMB
• POP3/POP3S
• RTMP
• SCP
• SFTP
• RTSP
• LDAP/LDAPS
• Telnet and TFTP
• RTSP

Additional features include:

• User and password authentication
  • Basic
  • Digest Plain
  • NTLM
  • CRAM-MD5
  • Kerberos
  • Negotiate
• Cookies
• Proxy tunneling
• Resume file transfer operation
• SSL certificates
• HTTP and HTTPS forms upload

wget and curl are often compared because their functionality overlaps to some degree. Both tools can retrieve content from the Internet, but wget has more features like web scraping, and recursive downloads as well it is more user-friendly. wget is considered a better option if you only want to download files in the terminal.

Learning curl command basics will help you upload and download files with advanced HTTP authentication procedures. Furthermore, wget only supports FTP and HTTP(S) while the curl supports way more protocols.

That’s all about curl command. Now let’s jump straight to the terminal.

Installing ‘curl’ on Linux

If you do not have curl on your Linux system, Install it by utilizing the following command. Otherwise, skip the installation steps and move toward the examples.

sudo apt-get update
sudo apt-get install curl -y

Now, verify that the curl is available on your system by checking its version:

curl --version

Examples of using curl

We can do a lot of cool things using curl. Let’s take a look at some of them.

Getting external server IP

There’s an amazing resource on the internet that allows you to get your internet IP address – https://ifconfig.me (named in the glory of the famous Linux network configuration utility – ifconfig).

If you sent an HTTP request to that site using curl, it would return you external IP address in the terminal in the form of the simple string:

curl https://ifconfig.me

So, you can easily put this result to the bash variable:

MY_EXTERNAL_IP=$(curl -s https://ifconfig.me)
echo $MY_EXTERNAL_IP

Here’s the result:

Here, -s argument allows to avoid curl download progress output:

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                  Dload  Upload   Total   Spent    Left  Speed
 100    12  100    12    0     0    181      0 --:--:-- --:--:-- --:--:--   181

Weather forecast

Do you want to feel like a hacker and display the weather information in the terminal? You can do it using curl!

curl http://wttr.in/LOCATION

Saving ‘curl’ output to a text file

You can save the output of the curl command to the specified file.

Here’s an example of saving JSON API output (StarWars demo API) to the file:

curl https://swapi.dev/api/planets/1/ -o Tatooine.json

curl is smart enough to detect binary file download. Here’s an example of downloading one of the most popular open-source infrastructure-as-code management software Terraform (learn more about Terraform):

curl https://releases.hashicorp.com/terraform/0.15.0/terraform_0.15.0_linux_amd64.zip -o terraform_0.15.0_linux_amd64.zip

Downloading multiple files

You can use curl to download multiple files at a time. Just add the -o argument as many times as you need.

curl -o https://example.com/files/file-1 -o file-2 https://example.com/files/file-2 -o file-3 https://example.com/files/file-3

Limiting the download speed

Another useful feature of the curl is the restriction of the file download speed. You can do it by using the --limit-rate argument and specifying the speed rate:

curl --limit-rate 1M -O https://releases.hashicorp.com/terraform/0.15.0/terraform_0.15.0_linux_amd64.zip

The given speed is measured in bytes/second unless a suffix is appended. Appending ‘k’ or ‘K’ will count the number as kilobytes, ‘m’ or M’ makes it megabytes, while ‘g’ or ‘G’ makes it gigabytes.

Downloading URLs list

In this example, we’ll download all files listed in the text file. To do that, you need to use a combination of xargs with curl commands:

xargs -n 1 curl -O < urllists.txt

Here’s output:

Basic authentication

You can use the -u argument for providing username and password for basic HTTP authentication (basic authentication):

curl -u username:password -O https://example.com/files/README

Getting the URL headers

HTTP headers are key-value pairs separated by colons that contain information like the requested resource content type, user agent, encoding, etc. The request or response transfers headers between the client and the server. To get the headers information of any website, use -I argument:

curl -I https://hands-on.cloud

Using cookies

You may need to use the cookies in subsequent requests to the same website.

To save the cookies received from the web server, use the following command:

curl -s -o /dev/null -c google_cookies.txt 'https://www.google.com'

Here arguments are:

• -s – the silent mode, curl does not print download progress information
• -o /dev/null – do not print web page output to the terminal
• -c google_cookies.txt – save cookies information to the file

To use cookies received from the previous request, use the following command:

curl -b google_cookies.txt 'https://www.google.com'

Checking HTTP/2 support

Use the -I, --http2 and -s options together to verify if the specified site supports HTTP/2 headers:

curl -I --http2 -s https://hands-on.cloud/ | grep HTTP

Note: For more information on the grep command, check out the “How to use the “grep” command in Linux” article.

Summary

This article covered the most commonly used examples of using the curl command. We hope this article will allow you to get started with this powerful utility. If you like the article, please, help us to spread it to the world!

Related articles

• Top 10 SSH Features You MUST Know To Be More Productive
• The Best Linux Distribution for your next cloud server
• How To Remove Files And Directories In Linux
• How to install Minecraft client on Ubuntu
• AWS CloudFormation. Managing VPC