Serverless environment and beating cold start
Disclaimer: This article is based on my current knowledge and understanding. If you have any query or see any improvement, please mention in the comment section
Serverless computing has redefined application development by eliminating the need for server management. In this cloud execution model, the cloud provider dynamically manages the underlying infrastructure, allowing developers to focus solely on writing code. Billing is simple—you only pay for the actual execution time.
However, this model is not without challenges. One significant issue is the cold start problem.
What is a Cold Start?
A cold start occurs when a serverless function is invoked after a period of inactivity. The cloud provider must initialize the runtime environment, load the necessary libraries, and prepare the execution context before the function can run. This process introduces latency, which is especially undesirable in scenarios where responsiveness is critical.
A Practical Use Case
Imagine a grocery store that has chosen serverless infrastructure for its IT systems. Customers use the store's app for online payments, product browsing, and placing orders.
Why did the grocery store choose serverless?
1. No server management: They want to focus on their business, not on maintaining infrastructure.
2. Fluctuating traffic: Serverless is ideal for scaling dynamically based on demand, avoiding underutilized or over-provisioned resources.
But here’s the problem: During peak hours, when many customers simultaneously use the app, some requests might experience a cold start delay. This could lead to poor user experience, especially when speed is critical.
Understanding the Layers
To understand the overhead causing cold starts, let’s look at the layers involved in running a serverless function on a platform built on Kubernetes:
1. Physical hardware
2. Host operating system
3. Container runtime and Kubernetes
4. Application code
Each layer adds a bit of latency because they must interact to get the application code running. When a new pod is spun up to handle a request after a period of inactivity, these layers must be initialized, introducing delays.
From a cloud provider’s perspective, scaling horizontally adds even more complexity. Spinning up new nodes involves provisioning hardware, initializing the Kubernetes installation, and deploying the application code—all of which contribute to cold start latency.
Reducing Cold Starts
How do we reduce this overhead? The key lies in simplifying the architecture.
Running Applications on Bare Metal
One approach is to bypass many of the intermediary layers by running applications directly on a bare-metal server. By cutting out the container runtime, Kubernetes, and even the host operating system in some cases, we can drastically reduce the initialization time.
Leveraging Unikernels
Unikernels are another promising solution. A unikernel is a highly specialized operating system that runs a single application. It packages just the essentials required for the application to function, allowing it to run directly on hardware.
With unikernels, the layers are reduced to:
1. Physical hardware
2. Unikernel with application code
Fewer layers mean faster startup times, significantly reducing cold start latency.
Specialized Virtual Machines
Another innovation in this space is Firecracker, a lightweight virtual machine manager designed for serverless workloads. Firecracker creates microVMs that can be initialized in milliseconds, offering a near-instant runtime environment.
By leveraging tools like Firecracker, cloud providers can reduce the overhead of cold starts, ensuring faster and more reliable function execution for end-users.
Combining Serverless with Bare-Metal Efficiency
As shown in the image above, a C# program can be compiled to produce a .efi file. This file can run directly on bare-metal servers or be combined with unikernels for deployment. Such an approach reduces the layers involved in running application code, minimizing overhead and addressing cold start issues effectively.
Huge thanks to bflat library to help compile C# code to .efi file
