The HTTP protocol is the standard protocol for communication between web browsers and web servers. HTTP defines how a client and server connect, how the client requests data from the server, how the server responds to that request, and how the connection is closed. TCP/IP is the data transfer protocol used by HTTP connections. Each request from client to server follows a four-step process:
1. The client opens a TCP connection to the server on port 80, by default; other ports may be specified in the URL.
2. The client sends a message to the server in which it requests the resource at the specified path. The request includes a header and, depending on the nature of the request, a blank line followed by the request's data.
3. The server responds to the client. The response starts with a response code, then a header with metadata, a blank line, and either the requested document or an error message.
4. The connection is terminated by the server.
This is the fundamental HTTP 1.0 procedure. Multiple requests and responses can be sent in series over a single TCP connection in HTTP 1.1 and later. That is, steps 2 and 3 may be repeated several times between steps 1 and 4. Furthermore, requests and responses in HTTP 1.1 can be sent in multiple chunks. This method is more scalable.
Relax, You are not required to memorize the HTTP specification. If you're interested, the HTTP protocol is an IETF standard, RFC 2616. Apache is a type of Web server that handles HTTP requests. Mozilla is an example of a Web Browser, which allows users to make HTTP requests and view the documents returned by the server.
This course only requires understanding the structure of HTTP messages.
What Is The Structure Of HTTP Messages?
The start-line and HTTP headers of an HTTP message are referred to as the request’s head, while the body is referred to as the payload.
What Is The Format Of HTTP Messages?
HTTP messages are used to transfer data between a server and a client. Messages are classified into two types: request and response.
HTTP Request Message Structure
HTTP requests and responses have a similar structure and are made up of the following elements:
1. A request line,
2. An HTTP header containing metadata,
3. a blank line,
4. A message body.
A typical client request looks something like this:
GET /index.html HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/220.127.116.11 Safari/537.36
Accept-Encoding: gzip, deflate, br
When a GET request does not contain a message body like this one, the request ends with a blank line. We’ve gone over the message structure of this GET request in great detail below.
- Request line:
GET /index.html HTTP/1.1
The first thing you’ll notice in the request line is an HTTP method name. These aren’t Java methods, but the concept is the same. The method name informs the server about the type of request being made as well as how the rest of the message will be formatted. The HTTP protocol has several methods, but the most common ones are GET and POST.
The method defines the operation that is being requested. The GET method requests that the server return a representation of a resource. Right after the method name, you’ll see the path to the resource requested from the server. The path to the resource requested from the server is /index.html. HTTP/1.1 is the protocol version that the client is familiar with.
- HTTP header containing metadata
Although only the request line is required, a client request usually includes additional information in a header. HTTP headers are name-value pairs that are included in the HTTP request or response message. They provide additional information about the request or response, such as the type of content being sent, the encoding of the content, the language of the content, and more.
Each line is written as follows:
Keywords are not case sensitive. Values sometimes are and sometimes aren’t. Both keywords and values should only contain ASCII characters. If a value is too long, you can continue it by adding a space or tab at the beginning of the next line.
A carriage-return linefeed pair is used to end lines in the header.
Some common HTTP headers include:
- Content-Type: Specifies the type of content being sent, such as text/html, application/json, or image/jpeg.
- Content-Length: Specifies the length of the content in bytes.
- Accept: Specifies the types of content that the client is willing to accept in the response.
- Accept-Encoding: Specifies the types of encoding that the client is willing to accept in the response.
- User-Agent: Specifies the client software and version that is making the request.
- Cookie: Specifies cookies that should be sent with the request or received with the response.
- Authorization: Specifies the credentials for authentication.
- Connection: Specifies the type of connection the client would like to use.
Headers can be used to control caching, provide authentication and authorization, negotiate content, and more. For example, the Expires header can be used to specify the time after which a response should be considered stale, and the Cache-Control header can be used to specify the caching rules for a response.
Except for the oldest first-generation browsers, all browsers include a Host field that specifies the server’s name, allowing web servers to differentiate between different named hosts served from the same IP address:
The third keyword in our example is User-Agent, which informs the server about the browser being used and allows it to send files that are optimized for that browser type. The following line indicates that the request is from the Mozilla Firefox browser version 5.0:
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64)
Another important keyword in this example is Accept, which informs the server about the types of data that the client can handle (though servers often ignore this). The following line, for example, states that the client can handle four MIME media types, which correspond to HTML documents, plain text, and JPEG and GIF images:
What Are ‘MIME Media Types’?
MIME types, also known as media types or content types, are a way to specify the format of a file or the type of content being sent in a HTTP request or response. They are used by web servers and browsers to determine how to handle a file or data.
MIME types consist of two parts: a type and a subtype, separated by a forward slash (/).
The type shows very generally what kind of data is contained: is it a picture, text, or movie? The data type is identified by the subtype: GIF image, JPEG image, TIFF image. The content type of HTML, for example, is text/html; the type is text, and the subtype is html. A JPEG image’s content type is image/jpeg; the type is image, and the subtype is jpeg. There are eight top-level types defined:
text/* for human-readable words image/* for pictures model/* for 3D models such as VRML files audio/* for sound video/* for moving pictures, possibly including sound application/* for binary data message/* for protocol-specific envelopes such as email messages and HTTP responses multipart/* for containers of multiple documents and resources
Each of these has numerous subtypes.
The most up-to-date list of registered MIME types can be found at http://www.iana.org/assignments/media-types/. Nonstandard custom types and subtypes can also be freely defined as long as they start with x-. Flash files, for example, are commonly assigned the type application/x-shockwave-flash.
- Blank Line
Finally, the request is terminated with a blank line—two carriage return/linefeed pairs, \r\n\r\n. When the server encounters that blank line, it begins sending its response to the client over the same connection. In the next section we will be talking about Response Message Structure.
Before discussing Response Message Structure let us briefly discuss another main HTTP Request : POST.
HTTP POST is one of the HTTP methods used to request that a server accept and store the data enclosed in the body of the message. It is typically used to submit data to be processed by the resource identified by the URI, for example, to submit a form data or upload a file.
When a client sends an HTTP POST request, it includes a message body that contains the data to be sent to the server. This data is usually in the form of key-value pairs and is encoded in the message body using a format such as application/x-www-form urlencoded or multipart/form-data.
The server then processes the data, typically by storing it in a database or using it to update the resource identified by the URI.
An example of an HTTP POST request would be a user submitting a login form on a website. The browser would send an HTTP POST request to the server, with the message body containing the user’s login credentials (username and password), encoded in the format specified by the Content-Type header.
POST requests are not idempotent, meaning that multiple identical requests may have different effects.
It is important to note that the information sent in a POST request is not visible in the URL and also can’t be bookmarked, which makes it less suitable for certain types of information that need to be publicly accessible. GET requests can be bookmarked, linked to, spidered, and so on because the URL contains all necessary information.
In practice, POST is overused on the Internet today. GET should be used instead of POST for any safe operation that does not commit the user to anything. POST should only be used for operations that commit the user.
When forms require a large amount of input, it is common practice to prefer POST over GET. It’s a common misconception that browsers can only handle query strings of a few hundred bytes. Although this was true in the mid-1990s, all major browsers are now capable of handling URLs of at least 2,000 characters. If you have more form data to submit than that, you may indeed need to support POST; however, for nonbrowser clients, safe operations should still prefer GET.
However, this is less common than you might think. You typically exceed those limits only when uploading data to the server to create a new resource rather than simply locating an existing resource on the server; and in these cases, POST is usually the correct answer anyway.
Aside from the two main HTTP methods, a few others are used in specific situations.
HTTP Response Message Structure
The response starts with a status line, then a header describing the response using the same “name: value” syntax as the request header, then a blank line, and finally the requested resource. A typical successful response looks something like this:
HTTP/1.1 200 OK Date: Sun, 21 Apr 2022 15:12:46 GMT Server: Apache Connection: close Content-Type: text/html; charset=ISO-8859-1 Content-length: 115 <html> <!-- Some Sample HTML --> <head> <title>A Sample HTML Page</title> </head> <body> <h1 align="center">Geek's Sample HTML Page</h1> <p align="right"> <img src="https://images.pexels.com/photos/161702/harmony-relax-rock-moqui-161702.jpeg?auto=compress&cs=tinysrgb&w=1260&h=750&dpr=1" alt="A pic of rock" width="130" height="140"/> </p> <form action="someAction" method="GET"> Name: <input type="text" name="param1"/><br><br> Password: <input type="text" name="param2"/><br><br> <center> <input type="submit"/> </center> </form> </body> </html>
HTTP/1.1 200 OK
The status line, or first line, of an HTTP response contains the following information:
- The protocol version, usually HTTP/1.1.
- A status code that indicates whether the request was successful or unsuccessful. The most common status codes are 200, 404, or 302.
- A status update. A brief, purely informative textual description of the status code to assist a human in understanding the HTTP message.
The most common response code is 200 OK, which indicates that the request was successful.
Or, 404 Not Found, which indicates that the browser was able to communicate with a specific server, but the server was unable to locate the requested resource. The error can also be used when a server refuses to reveal whether or not it has the requested information.
A response code from 100 to 199 always indicates an informational response, 200 to 299 always indicates success, 300 to 399 always indicates redirection, 400 to 499 always indicates a client error, and 500 to 599 always indicates a server error, regardless of version.
The header lines identify the date the request was made in the server’s time frame, the server software (Apache), a promise that the server will close the connection when it’s finished sending, the MIME media type, and the length of the document delivered (not counting this header)—in this case, 107 bytes.It’s not 107 bytes for the aforementioned page. We put it there to challenge you. Can you evaluate the true size of the content? In a later tutorial, we will calculate them. Stay sharp!
One more thing before we wrap up this tutorial. We’d like to discuss the Connection keyword.
For each request, HTTP 1.0 establishes a new connection. In practice, the time it takes to open and close all of the connections in a typical web session can outweigh the time it takes to transmit the data, particularly for sessions with a large number of small documents.
This is especially problematic for encrypted HTTPS connections using SSL or TLS, because the handshake to establish a secure socket requires significantly more effort than establishing a regular socket.
The server does not have to close the socket after sending the response in HTTP 1.1 and later. It can leave it open and await a new client request on the same socket. Over a single TCP connection, multiple requests and responses can be sent in succession. However, the lockstep pattern of a client request followed by a server response remains the same.
A client indicates its willingness to reuse a socket by including a Connection field with the value Keep-Alive in the HTTP request header.