Graceful Shutdown in Spring

Graceful Shutdown in Spring: A Deep Dive into `spring.lifecycle.timeout-per-shutdown-phase` and Beyond

Introduction

As senior Java developers, we've all faced the critical challenge of ensuring our applications shut down gracefully. A clean shutdown is not just about stopping the process; it's about completing in-flight requests, releasing resources, and leaving the system in a consistent state. In the world of Spring Boot, achieving this is surprisingly elegant, thanks to a few key properties. This blog post will take a deep dive into spring.lifecycle.timeout-per-shutdown-phase, and explore the broader context of graceful shutdowns with real-world use cases.

The Problem with Abrupt Shutdowns

Imagine a microservice handling e-commerce orders. An abrupt shutdown, caused by a kill -9 or an unexpected server termination, can lead to:

Data Inconsistency: A payment transaction might be processed by a third-party gateway, but the local database update fails, leaving the order in a limbo state.
Failed Requests: Users receive 503 Service Unavailable or connection reset errors in the middle of a request, leading to a poor user experience.
Resource Leaks: Open database connections, file handles, and network sockets are not properly closed, potentially causing issues for other services or the operating system.

A graceful shutdown, on the other hand, allows the application to:

Stop Accepting New Requests: It signals to the load balancer or service mesh that it's no longer healthy and shouldn't receive new traffic.
Complete In-Flight Requests: It waits for all currently active requests to finish their execution.
Clean Up Resources: It performs a controlled shutdown of database connection pools, message listeners, and other critical components.

Spring Boot's Built-in Graceful Shutdown Mechanism

Spring Boot 2.3 introduced a significant improvement with its graceful shutdown capabilities. The core mechanism is controlled by the server.shutdown property.

`server.shutdown`

This property configures how the web server (Tomcat, Jetty, or Undertow) handles shutdown.

server.shutdown=immediate (Default): The server shuts down as soon as the signal is received. This is the classic, abrupt shutdown behavior.
server.shutdown=graceful: This is the magic bullet. When a shutdown signal is received (e.g., SIGTERM), the web server will:
1. Stop accepting new requests.
2. Allow a configurable timeout period for existing requests to complete.
3. Once the timeout expires or all requests are finished, the server shuts down.

The `spring.lifecycle.timeout-per-shutdown-phase` Property

While server.shutdown=graceful handles the web server, what about the rest of your application's lifecycle? What if you have long-running background tasks, message listeners, or other components that need to be cleaned up?

This is where spring.lifecycle.timeout-per-shutdown-phase comes into play. This powerful property, introduced to provide fine-grained control over the application context shutdown process, defines the maximum time an application component has to shut down cleanly.

Let's break down how this works. Spring's application context shutdown is a multi-phase process. It involves:

Phase -100 to -1 (Web Server): This is where the web server, along with its graceful shutdown process, operates.
Phase 0 (Default): This is the default phase for most components, where they are shut down in a controlled manner.
Phase 1 to 100 (Custom): You can define custom shutdown phases for your own components using the @Order annotation or implementing the SmartLifecycle interface.

spring.lifecycle.timeout-per-shutdown-phase sets a timeout for each of these phases. For example, setting spring.lifecycle.timeout-per-shutdown-phase=30s means that each shutdown phase is given up to 30 seconds to complete its tasks before being forcefully terminated.

Example Configuration:

# application.yml
server:
  shutdown: graceful

spring:
  lifecycle:
    timeout-per-shutdown-phase: 30s

---

Real-World Use Cases and Scenarios

Use Case 1: The Long-Running API Request

Scenario: An API endpoint processes a file upload that can take up to 20 seconds. An operator initiates a shutdown of the service during a rolling deployment.

Without server.shutdown=graceful: The deployment script sends a SIGTERM. The application immediately exits. The in-flight file upload request is terminated, causing a Connection Reset error for the user. The partially uploaded file might be left in a temporary directory, leading to orphaned resources.

With server.shutdown=graceful: The SIGTERM is received. The web server stops accepting new connections but allows the 20-second file upload to continue. The request completes successfully. The application then proceeds to its full shutdown. The user is unaware of the ongoing deployment.

Use Case 2: Kafka Consumers and Background Tasks

Scenario: Your microservice has a @KafkaListener that processes messages from a topic and a @Scheduled task that runs every 5 minutes to clean up old data.

Without spring.lifecycle.timeout-per-shutdown-phase: The application receives a SIGTERM. While the web server might be graceful, the Kafka listener might not have a chance to commit the last offset. The application is terminated while the scheduled task is running, leading to an incomplete cleanup process.

With spring.lifecycle.timeout-per-shutdown-phase: The web server gracefully shuts down. Then, Spring's lifecycle mechanism kicks in. The Kafka consumer, being a SmartLifecycle component, is given the configured timeout (e.g., 30s) to finish processing its current batch of messages and commit offsets. Similarly, the scheduled task has a chance to complete its execution before the application context is closed. This ensures data integrity and a clean state.

Use Case 3: Custom Shutdown Logic

Scenario: Your application maintains a connection to an external cache cluster and needs to send a final close signal to release resources on the cache server.

Solution: You can create a custom component that implements the SmartLifecycle interface.

import org.springframework.context.SmartLifecycle;
import org.springframework.stereotype.Component;

@Component
public class CacheClientShutdown implements SmartLifecycle {

    private volatile boolean running = false;

    @Override
    public void start() {
        // Initialize cache connection
        this.running = true;
    }

    @Override
    public void stop() {
        // Stop logic, send a close signal to the cache server
        System.out.println("Gracefully stopping the cache client...");
        this.running = false;
    }

    @Override
    public boolean isRunning() {
        return this.running;
    }

    @Override
    public int getPhase() {
        // A phase lower than 0 to run before default shutdown
        return 50; 
    }
}

By defining a custom getPhase() and leveraging spring.lifecycle.timeout-per-shutdown-phase, you ensure that your custom shutdown logic is executed within a controlled timeframe during the application context shutdown process.

---

Final Thoughts and Best Practices

Don't rely on kill -9: This signal bypasses the JVM and the operating system's graceful shutdown process. Use kill (which sends SIGTERM) for a controlled shutdown.
Configure your orchestrator: In Kubernetes or other container orchestration platforms, ensure your terminationGracePeriodSeconds is set appropriately to give your application enough time to shut down. This value should be greater than spring.lifecycle.timeout-per-shutdown-phase.
Log your shutdown process: Add info or debug logs to your shutdown hooks and stop() methods to track what is happening during the shutdown. This is invaluable for debugging issues.
The combination is key: The true power of graceful shutdown in Spring comes from the combination of server.shutdown=graceful for the web server and spring.lifecycle.timeout-per-shutdown-phase for the entire application context.

By mastering these simple yet powerful Spring properties, you can elevate your application's resilience and robustness, ensuring a smooth and predictable experience for your users and a consistent state for your data.

Graceful Shutdown in Spring