HTTP From the Ground Up: Writing a Web Server In C

One of the most important lessons I have learned is that there are no free abstractions. Because there are no free abstractions, I believe every software engineer should seek to expand his or her knowledge of the underlying software abstractions they utilize. Abstraction can often be a big problem in web development. There are too many frameworks that are just optimized for the fastest “hello world” program in order to market to junior software engineers who have limited knowledge of the problem their favorite framework seeks to solve. To combat this, Software engineers should arm themselves with knowledge of the underlying web platform that we use to deliver applications.

Many of the common abstractions used in web development are taken for granted. The biggest seems to be HTTP clients. More specifically, how HTTP is actually implemented. The core technological paradigm that HTTP provides an abstraction for is the management of socket connections. Most importantly, HTTP provides a stateless interface to stateful networking. Instead of having to manage a connection to a server, we just specify a hostname or IP address, a path to the desired resource, and optionally a bit of data that defines the content of the request.

There are a few concessions I have made to keep the code concise. When you write C, good error handling is paramount for creating reliable applications. This guide mostly foregoes handling errors and assumes everything works. However, if you want to implement an actual web server in C, I highly suggest you read through the Berkeley socket man pages to get a feel for what things may fail when you try to create a socket. The second thing that is worth mentioning, is that I do not use any dynamic memory allocation (malloc, calloc, free) in this program because I find it to be simpler and shorter without it. This does prevent the web server from being able to process some requests, but having a few KiB of statically allocated memory is more than enough for a pedagogical example.

I am going to explain how you can write an HTTP server in C with just Berkeley sockets. Instead of writing a fully-featured HTTP server like NGINX or Apache, we are going to create an example server that returns the content of every request in a response. This type of server is called an "echo server" because it echos whatever you send to it.

Creating a Socket

Let us get started by creating a socket using the socket function provided in <sys/socket.h> header file.

#include <sys/socket.h>

int main(int argc, char* argv[]) {
    int socket_fd = socket(
        AF_INET,
        SOCK_STREAM,
        0
    );
}

The socket function takes three parameters: domain, type, and protocol. The domain parameter specifies what type of addressing scheme is used. We use AF_INET for the domain because we are going to be using IPv4 addressing. The type of socket we use is SOCK_STREAM. This variant ensures a reliable connection which is required of every HTTP implementation as specified in RFC 1945 Section 1.3. The protocol field specifies what protocol we are using within the specified domain because there is only one protocol in IPv4 we are going to set this value to zero because that is the only option.

Now it is time to bind this socket to an IP address and listen to traffic.

#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>

int main(int argc, char* argv[]) {
    // ...

    struct sockaddr_in address = {
        // IPv4 Address Family
        .sin_family = AF_INET,
        .sin_port = htons(4444),
        .sin_addr = inet_addr("127.0.0.1"),
    };

    int code = bind(
        socket_fd,
        (struct sockaddr*)&address,
        sizeof(address)
    );
    if (code == -1) {
        printf("ERROR!\n");
        return EXIT_FAILURE;
    }

    // ...
}

The first step to binding our socket to an IP address is creating the address. We are going to use the IP address 127.0.0.1. IP addresses in the 127.0.0.0 to 127.255.255.255 range are called loopback addresses. A loopback is an address that only supports communication within your computer system and not any local network you may be connected to. 127.0.0.1 is commonly referred to as localhost because you can only access a web server from localhost if it is running on your machine. In addition to the IP address, we also have to specify the port.

We could just assign 4444 as the port for this address, but it is not that simple. When communicating over a network it is very important to have the correct endianness, or else systems with differing byte ordering will be sending malformed data to each other. By default, all network communication uses big endian ordering for bytes. Because we do not know what system this program might be running on, we ensure the port is formatted correctly with the htons function. This function converts an unsigned 32-bit integer to an unsigned 32-bit integer with big-endian byte ordering.

Once we call bind with the specified arguments, we map the socket file descriptor to the specified address and port.

#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <errno.h>
#include <stlib.h>

int main(int argc, char* argv[]) {
    // ...

    if (listen(socket_fd, 64) == -1) {
        printf("An error has occurred...\n");
        return EXIT_FAILURE;
    }

    // ...
}

Once our socket has been bound to an address, this enables HTTP clients to send HTTP requests to that address. But their HTTP requests will fall on deaf ears if we do not listen first. The listen function is what we use to start receiving connections on our socket. It has only two parameters: socket, and backlog. The socket parameter is the file descriptor of the socket we are listening with. The backlog parameter is an integer that determines the maximum number of requests that can queue up before our server starts refusing connections. We set it to 64 because it is a power of 2 and I like powers of 2, but if you want to have a different number you can set it to whatever you like.

Main Control Loop

Now that we have started listening we can accept connections at the address and port by using the accept function.

#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <errno.h>
#include <stlib.h>

int main(int argc, char* argv[]) {
    // ...

    struct sockaddr_in peer = (struct sockaddr_in){ 0 };
    size_t peer_len = sizeof(peer);

    while (1) {
        int connection_fd = accept(
            socket_fd,
            (struct sockaddr)&peer,
            &peer_len
        );

        // ...
    }

    // ...
}

The only arguments needed to use this function are the original socket file descriptor (socket), a pointer to where the address of the client can be written to (address), and a pointer to where the length of the address can be written to. (address_len). The accept connection creates a new file descriptor for the new connection and returns it as the return value of the function. This is very handy as creating a new file descriptor for each new connection frees us up to use multiple threads to serve new connections instead of blocking traffic on only one socket. We use this file descriptor to both read requests and send responses.

Once a client has connected to our server we must read the request that they have sent to us. We can use the recv function to read bytes of the HTTP request into a 4 KiB buffer. The recv function returns how many bytes that were read into the buffer. This will be important later when we have to create the Content-Length header for our response.

#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <errno.h>
#include <stlib.h>

int main(int argc, char* argv[]) {
    // ...

    while (1) {
        // ...
        char request_buffer[0x1000] = { 0 };

        ssize_t request_length = recv(
            connection_fd,
            request_buffer,
            sizeof(request_buffer),
            MSG_PEEK
        );

        // ...
    }

    // ...
}

We use an array to statically allocate a 4 KiB buffer to store the bits in the HTTP request we are receiving. The recv function takes three parameters: socket, buffer, length, and flags. The buffer is a pointer to the buffer that will store the received data. The length parameter indicates the maximum amount of bytes that can be read into the buffer parameter. The last parameter is an int called flags. flags can be thought of as a vector of bits, each place in the vector meaning a different thing when it equals 1. The options for the flags parameter are MSG_OOB, MSG_PEEK, and MSG_WAITALL. These options are not mutually exclusive so they can be combined by using the bitwise OR operator (|). For now we will just use the MSG_PEEK option for our flags parameter.

Now that we have read in the full HTTP request (assuming it was under 4 KiB), we must retrieve the body of the request so we can relay it back to the sender. Here is an example HTTP request where we send a message in JSON to a web server located at http://johnhiggins.io:

POST / HTTP/1.1
Host: johnhiggins.io
Content-Type: application/json
Content-Length: 28

{ message: "Hello world!" }

You can tell when the body starts because of the two line breaks between the Content-Length header field and the first curly brace of our JSON payload. So let us write a parser that finds where the body starts. Well as it turns out, that is pretty easy. C has a function included in its standard library called strstr. This function returns a pointer to where a specified substring is first found in a larger string. Here is how we can find the beginning of the HTTP request body with strstr:

#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <errno.h>
#include <stlib.h>

int main(int argc, char* argv[]) {
    // ...

    while (1) {
        // ...

        char* body_start = strstr(request_buffer, "\r\n\r\n") + 4;
        size_t header_length = (size_t)(body_start - request_buffer);
        size_t body_length = request_length - header_length;

        // ...
    }

    // ...
}

First we get the beginning of the body by detecting the double line break. Line breaks in HTTP requests and responses are encoded with a carriage return character followed by a line feed. Once we find where the double line break occurs we add 4 to the address, so we can skip past the end of the header content and read in just the body. We know the length of the body by subtracting the length of the header from the length of the entire request. Knowing the length of the body is required because when sending an HTTP response, we need to provide a Content-Length header or else the response will be invalid.

// ...

const char HTTP_HEADER_F_STR[] = "HTTP/1.1 200 OK\r\n"
                                 "Content-Type: text/html\r\n"
                                 "Content-Length: %lu\r\n"
                                 "Server: C Echo Server\r\n\r\n";

int main(int argc, char* argv[]) {
    // ...

    while (1) {
        // ...

        char response_buffer[0x1000]  = { 0 };
        snprintf(header, sizeof(header), HTTP_HEADER_F_STR, body_size);

        char* response_body_start = strstr(response_buffer, "\r\n\r\n");
        memcpy(response_body_start, body_start, body_length);

        // ...
    }

    // ...
}

Now that we have our request body loaded into a buffer and our Content-Length value, we are ready to send the response. To make space for combining the new header fields and the response body, we will statically allocate a buffer with 4 KiB of space. After this we will use snprintf to write our interpolated HTTP header fields into the response_buffer. Once we have written the header to the buffer, we can write the response body to the address right after the end of the HTTP response header region. We find the end of the header region the same way we found it with the request header region, by using strstr.

// ...

int main(int argc, char* argv[]) {
    // ...

    while (1) {
        // ...

        size_t response_header_length =
                (size_t)(response_body_start - response_buffer);
        size_t response_length =
                response_header_length + body_length;

        send(connection_fd, response_buffer, response_length, 0);
        close(connection_fd);
    }

    // ...
}

Now that we have loaded our buffer with the full response, we send it using the send function. The send function takes four parameters:

  • sockfd: the socket file descriptor. This will end up being set to the current connection's socket file descriptor.
  • buf: the buffer to read bytes from. This will be set to response_buffer.
  • len: the number of bytes to send from the buf. This will be calculated based on the total length of the request. (header + body)
  • flags: a bit vector that allows us to apply special settings to our invocation. We are going to just ignore this and set it to zero.

Once we have sent our HTTP response, we can close the connection and wait for another request, beginning the cycle again.

The End Result

Here's the entire program:

#include <sys/socket.h>
#include <fcntl.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <string.h>
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>

#define HTTP_HEADER_F_STR "HTTP/1.1 200 OK\r\n"\
                          "Content-Type: application/json\r\n"\
                          "Content-Length: %lu\r\n"\
                          "Server: C Echo Server\r\n"\
                          "\r\n"

int main(int argc, char* argv[]) {
    int socket_fd = socket(AF_INET, SOCK_STREAM, 0);

    struct sockaddr_in address = {
        .sin_family = AF_INET,
        .sin_port = htons(4444),
        .sin_addr = inet_addr("127.0.0.1"),
    };

    int code = bind(
        socket_fd,
        (struct sockaddr*)&address,
        sizeof(address)
    );
    if (code == -1) {
        printf("ERROR!\n");
        return EXIT_FAILURE;
    }

    if (listen(socket_fd, 64) == -1) {
        printf("ERROR!\n");
        return EXIT_FAILURE;
    }

    struct sockaddr_in peer = (struct sockaddr_in){ 0 };
    socklen_t peer_len = sizeof(peer);

    while (1) {
        int connection_fd = accept(
            socket_fd,
            (struct sockaddr*)&peer,
            &peer_len
        );

        char request_buffer[0x1000] = { 0 };

        ssize_t request_length = recv(
            connection_fd,
            request_buffer,
            sizeof(request_buffer),
            MSG_PEEK
        );

        char* body_start = strstr(request_buffer, "\r\n\r\n") + 4;
        size_t header_length = (size_t)(body_start - request_buffer);
        size_t body_length = request_length - header_length;

        char response_buffer[0x1000]  = { 0 };
        snprintf(response_buffer, sizeof(response_buffer), HTTP_HEADER_F_STR, body_length);

        char* response_body_start = strstr(response_buffer, "\r\n\r\n") + 4;
        memcpy(response_body_start, body_start, body_length);

        size_t response_header_length = (size_t)(response_body_start - response_buffer);
        size_t response_length = response_header_length + body_length;

        response_buffer[response_length] = '\r';
        response_buffer[response_length + 1] = '\n';

        send(connection_fd, response_buffer, response_length + 2, 0);
        close(connection_fd);
    }

    return EXIT_SUCCESS;
}
Testing The Echo Server

To test this server we are going to use the cURL command. If you prefer a GUI option you could use Postman or Thunder Client instead. Really any HTTP client program will do. First compile your program and run it. It should block until it receives an HTTP request, at which point the server should send a response that contains a duplicate body to the request. Here is an example curl command you can use to test the server:

curl http://localhost:4444 \
        -d '{ "message": "Hello world!" }' \
        -H 'Content-Type: application/text'

Your output should be similar to this:

{ "message": "Hello world!" }
Further Reading