Perhaps, you've noticed a little lock icon next to your browser's URL bar when visiting certain websites. That's HTTPS in action. But what exactly is HTTPS? Well, HTTPS, or Hypertext Transfer Protocol Secure, is an extension of the HTTP. It is used for secure communication over a computer network, such as the internet, and is broadly used on the World Wide Web. HTTPS is a communication protocol that is built on top of the TCP (Transmission Control Protocol) protocol.
The HTTPS handshake process involves an initial TCP handshake to establish the connection. This is followed by the TLS, or Transport Layer Security handshake, which is used to establish a secure, encrypted channel. Finally, the transmission of encrypted data occurs over this secure channel.
Now, let's proceed to break down and explore each component in further detail.
TCP Handshake
The internet connection involves a router and a device, the router acts as the intermediary between the device and the internet, routing data packets between them- using an automatic log-in system to send data, made possible by the Internet protocol suite including the Transmission Control Protocol (TCP).
TCP, a data transfer method between devices, has been standardized since RFC 793 (1981) with several updates, the latest being RFC 7323 (2014).
TCP enables two-way data transfer, like phone calls, with software handling connections and transfers, managed by network apps via certain interfaces.
The three-way handshake: Establishing a TCP connection
To establish a TCP connection, both endpoints must have a unique IP address and an assigned, enabled port. Establishing a TCP connection involves a process where:
The client sends the server a SYN packet with a unique, random number to synchronize transmission.
The server accepts the connection by returning a SYN-ACK packet with the client's sequence number, plus 1, and its own sequence number.
The client acknowledges the SYN-ACK segment by sending an ACK packet with the server's sequence number, plus 1, and begins transferring data.
Let's start by initiating a TCP connection with a trusted website using your terminal.
First, we need to determine the network interface. To do this on macOS, enter the following command:
networksetup -listallhardwareports | grep -A 1 "Wi-Fi" | awk '/Device/ {print $2}'In my case, the output is:
en0Once you have identified the network interface name, you can use it in the tcpdump command to capture packets related to your website's connection. Replace en0 with your network interface , and 'host noratech.blog' by the website with the desired website. In my case.
sudo tcpdump -i en0 'host noratech.blog' -l | awk '{print $3,$5,$9,$11,$13,$15,$17}'Great! Now it will listen until you establish a TCP connection.
Next, open another terminal window and install Telnet. Once installed, enter the following command:
telnet noratech.blog 443In the first terminal window, you will see the three-way handshake. Here's an example output:
192.168.10.211.49520 unalocated.63.wixsite.com.https: 472838408, 65535, [mss 6,nop,nop,TS 3477074087
unalocated.63.wixsite.com.https 192.168.10.211.49520: 1714745662, 472838409, 14100, [mss 2,sackOK,TS
192.168.10.211.49520 unalocated.63.wixsite.com.https: 1, 2058, [nop,nop,TS 3477074182 922479727]Let's analyze the provided output and identify the type of each packet:
Armed with a robust understanding of TCP communication, you're now well-equipped to advance to the next exciting segment, TLS Handshake.
HTTPS and TLS handshake
When you enter a website using HTTPS, browser or client will initiate a TCP connection with the web server. Within the TCP connection, the TLS handshake take place where the two systems agree on the encryption algorithm and cryptographic keys to use when transmitting data.
The illustration below demystifies the exchange between the client and server:
Let's break down this virtual conversation into four fundamental steps:
TCP Handshake: A detailed examination provided in the preceding section.
Certificate Check: Upon establishment of the TCP connection, the digital certificate of the server becomes presented to the client. This certificate comprises the public key of the server and other related information. The responsibility of the client, at this point, involves authenticating the validity of the given certificate and assessing its trust level with the certificate issuer. To further elucidate this process, consider these sequential steps: a. Client sends Client Hello. b. Server responds with Server Hello. c. Server sends digital certificate. d. Client verifies server's certificate.
Key Exchange: After validating the certificate, the client and server engage in a key exchange process. They negotiate and agree on an encryption algorithm and generate a shared secret key (session key). The client encrypts the session key using the server's public key and sends it to the server. The server then decrypts the session key using its private key. This shared key is used to encrypt and decrypt the data transmitted between the client and server, ensuring the privacy of their communication.
Data Transmission: With the shared secret key in place, the client and server can securely transmit data over the established TCP connection. The client encrypts the data using the shared key, and the server decrypts it upon receiving. This ensures that any sensitive information exchanged between the client and server remains confidential.
Before delving into the hands-on section, it is beneficial to have an in-depth discussion about the Server Name Indicator (SNI)
Server Name Indicator (SNI)
SNI is an extension to the TLS protocol that enables a client to specify the hostname of the server it wants to connect to during the initial handshake. This is especially valuable when a server hosts multiple websites or services with distinct domain names. For instance, suppose there are three apps hosted on a server with different domains, and the server's IP address is 100.200.30.30. If someone enters "https://100.200.30.30" into their browser, which app are they trying to access?
So here where the SNI comes, the server will rely on SNI to determine which app they are trying to access. The SNI extension allows the client to include the intended hostname in the TLS handshake within the Client Hello message. When the client establishes a connection to the server using the IP address, the server examines the SNI field in the TLS handshake to identify the requested app based on the specified domain name.
Below is an illustrative example of where SNI positions within the communication framework.
Inspecting HTTPS Interactions via Command Line
Now, let see how HTTPS works behind the scenes? You can use the command line to observe HTTPS interactions in action. Tools like CURL and OpenSSL make it possible to delve into the details.
Open your terminal and run the following curl command:
curl -v https://noratech.blogCurl will initiate an HTTPS request to "noratech.blog" and display the verbose output in your terminal. The `-v` option enables verbose mode to display detailed output.
Ok , this is the output I got :
* Trying 185.230.63.107:443...
* Connected to noratech.blog (185.230.63.107) port 443 (#0)
* ALPN: offers h2
* ALPN: offers http/1.1
* CAfile: /etc/ssl/cert.pem
* CApath: none
* [CONN-0-0][CF-SSL] (304) (OUT), TLS handshake, Client hello (1):
* [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, Server hello (2):
* [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, Unknown (8):
* [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, Certificate (11):
* [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, CERT verify (15):
* [CONN-0-0][CF-SSL] (304) (IN), TLS handshake, Finished (20):
* [CONN-0-0][CF-SSL] (304) (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / AEAD-CHACHA20-POLY1305-SHA256
* ALPN: server accepted h2
* Server certificate:
* subject: CN=noratech.blog
* start date: Oct 1 00:00:00 2023 GMT
* expire date: Dec 30 23:59:59 2023 GMT
* subjectAltName: host "noratech.blog" matched cert's "noratech.blog"
* issuer: C=GB; ST=Greater Manchester; L=Salford; O=Sectigo Limited; CN=Sectigo RSA Domain Validation Secure Server CA
* SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* h2h3 [:method: GET]
* h2h3 [:path: /]
* h2h3 [:scheme: https]
* h2h3 [:authority: noratech.blog]
* h2h3 [user-agent: curl/7.87.0]
* h2h3 [accept: */*]
* Using Stream ID: 1 (easy handle 0x13d00a800)
> GET / HTTP/2
> Host: noratech.blog
> user-agent: curl/7.87.0
> accept: */*
>
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 301
< date: Mon, 16 Oct 2023 06:22:02 GMT
< content-length: 0
< location: https://www.noratech.blog/
< strict-transport-security: max-age=3600
< x-wix-request-id: 1697437322.1601248696283125374
< age: 0
< server-timing: cache;desc=miss, varnish;desc=miss, dc;desc=84
< x-seen-by: wmgbEcS9zOENaefw7bU4YUB0kmPC/05AKUBuQHHHBmo=,W1c2/pqHBqplxcWufHCkILxkNjrXdwdgtu6E0yACibU=,m0j2EEknGIVUW/liY8BLLqv1Icz/+auWkycB0m9nz7sFJmEKNgQ96+wiTVoMq713,2d58ifebGbosy5xc+FRalmNCgLhHYZgyTRrW0VW/UX1/hzvVzus3vvE7uK+V8QtU3OIO0EWIn7wGpwavbAm9GQ==,2UNV7KOq4oGjA5+PKsX47PQEXbXFZaYW1Dg5frPZCM1YgeUJqUXtid+86vZww+nL,BBFUBQ3/n2DDz6iDQALVB53DYjpL8g/j68fC98i8OfA=,3Nrtc4p7ME8mtM+FeG7FGBmFGgVLN0WwWbwFfdM2KVs=,wjXkXN74v+Dcwxj+Ualvvm4mJ7FLIN31CmCmIre4NHSUyrVmGQPcCQm3OgV2hEqrX9e1ZFb+gMaVCbfb2CRb/g==
< cache-control: no-cache
< x-content-type-options: nosniff
< server: Pepyaka/1.19.10
<
* Connection #0 to host noratech.blog left intactBy observing the coloration displayed above, you can discern the initiation of each step. However we must highlights:
The CAfile path is the location where the client looks for trusted CA (Certificate Authority) certificates during the SSL/TLS handshake process.
* CAfile: /etc/ssl/cert.pem
* CApath: noneWhen the client and server negotiate the SSL/TLS encryption settings, it will include the version of the protocol (TLSv1.3) and the cipher suite (AEAD-CHACHA20-POLY1305-SHA256) to be used for secure communication.
The transmission of data starts with the line: HTTP/2 GET This line represents an HTTP/2 request being sent by the client to the server.
What if, as a client, you want to bypass the certificate check? In some cases, developers may want to test the HTTPS connection to their application before deploying it or without having a trusted certificate. In such situations, they can use curl with the --insecure or -k option. Since I trust noratech.blog , I will do the command on it to see the difference
curl -v --insecure https://noratech.blogThe --insecure or -k option tells curl to ignore any SSL/TLS certificate verification errors. When this option is used, curl will establish the connection with the server without validating the server's certificate against a trusted certificate authority.
This allows developers to proceed with the HTTPS connection, even if the server's certificate cannot be verified!
Now, back to our command with “-k”, you will notice that the output will be similar to the previous command, but with some slight differences.
The client and server still exchange greetings, and the server sends its certificate to the client. However, the client does not check the CAfile (certificate authority file) or the subjectAltName of the server.
In this article, we explored the TCP handshake and TLS handshake to gain a comprehensive understanding of the HTTPS handshake process. We also delved into the concept of Server Name Indicator (SNI) and its role in identifying the requested app during the handshake. Moreover, we conducted practical experiments using curl to examine the details of the handshake. By diving into these subjects, we obtained valuable insights into the intricacies of HTTPS and its fundamental components. Stay tuned for our upcoming article, where we will delve deeper into the intricacies of securing your apps with HTTPS. We will provide you with valuable insights and practical guidance to ensure that your applications remain safeguarded against potential threats.
Thank you for reading, and we look forward to sharing more knowledge with you in our next installment♡
Comments