Spring Boot production readiness — beyond the happy path

A Spring Boot service can be easy to deploy and still be badly prepared for production.

The application may start successfully, expose an endpoint, connect to the database, and still fail in ways that are painful to debug later: Kubernetes restarts it for the wrong reason, traffic reaches it before it is ready, shutdown drops in-flight work, the DB pool becomes the bottleneck, or logs do not have enough context to trace a failed request.

This is the production checklist I care about before trusting a Spring Boot service in a Kubernetes environment.

Health checks: liveness and readiness are not the same thing

The most common mistake is making health checks too clever.

A liveness check should answer one narrow question:  Is this process broken beyond recovery?

A readiness check asks something different:   Should this instance receive traffic right now?

That distinction matters. If the database is temporarily slow, I may want the pod to become not ready so it stops receiving new traffic. I usually do not want Kubernetes to restart every application instance just because the database had a short incident.

A basic Spring Boot Actuator setup:

management:
  endpoints:
    web:
      exposure:
        include: health,info,metrics,prometheus
  endpoint:
    health:
      probes:
        enabled: true
      show-details: never

Then Kubernetes can use the dedicated probe endpoints:

livenessProbe:
  httpGet:
    path: /actuator/health/liveness
    port: 8080
  initialDelaySeconds: 30
  periodSeconds: 10

readinessProbe:
  httpGet:
    path: /actuator/health/readiness
    port: 8080
  initialDelaySeconds: 10
  periodSeconds: 5

For external dependencies, I prefer being explicit. A database, broker, or downstream API check usually belongs in readiness, not liveness.

management:
  endpoint:
    health:
      group:
        liveness:
          include: livenessState
        readiness:
          include: readinessState,db

The readiness check should also be cheap. A health endpoint that runs a heavy query or calls multiple downstream systems can become part of the outage.

Graceful shutdown is a contract, not just a property

Graceful shutdown is not only a Spring Boot setting. It is a contract between the application, Kubernetes, the load balancer, and the workload itself.

For Spring Boot, I make shutdown behavior explicit:

server:
  shutdown: graceful

spring:
  lifecycle:
    timeout-per-shutdown-phase: 25s

And I align that with Kubernetes:

terminationGracePeriodSeconds: 35

The important part is the ordering.

When Kubernetes wants to stop a pod, it sends SIGTERM. The app should stop accepting new work, finish what it can, release resources, and exit before Kubernetes sends SIGKILL.

For a normal HTTP API, that mostly means letting in-flight requests finish.

For background workers, Kafka consumers, schedulers, or reconciliation-style jobs, graceful shutdown is more subtle. The service needs to stop pulling new work before it exits.

Conceptually:

@Component
public class WorkerLifecycle implements SmartLifecycle {

    private volatile boolean running = false;

    @Override
    public void start() {
        running = true;
        // start consuming or scheduling work
    }

    @Override
    public void stop(Runnable callback) {
        running = false;

        try {
            // stop accepting new work
            // finish or safely checkpoint current work
            // commit offsets / persist progress where appropriate
        } finally {
            callback.run();
        }
    }

    @Override
    public boolean isRunning() {
        return running;
    }
}

The exact implementation depends on the workload. A Kafka consumer, payment workflow, scheduled reconciliation job, and HTTP service should not all shut down the same way.

Metrics should explain production behavior

Exposing Prometheus metrics is the easy part. Choosing useful metrics is harder.

I do not want metrics only because dashboards look nice. I want metrics that answer production questions:

• Is traffic increasing?
• Are requests getting slower?
• Are failures isolated to one dependency?
• Is the DB pool saturated?
• Are retries hiding a downstream problem?
• Is a consumer falling behind?
• Are timeouts increasing before errors appear?

For Spring Boot, Actuator and Micrometer give a good baseline:

management:
  endpoints:
    web:
      exposure:
        include: health,metrics,prometheus
  metrics:
    tags:
      application: my-service

For business or workflow metrics, I prefer measuring things that help during incidents.

Example:

@Service
public class PaymentProcessor {

    private final Counter failedPayments;
    private final Timer processingLatency;

    public PaymentProcessor(MeterRegistry registry) {
        this.failedPayments = Counter.builder("payments_failed_total")
            .description("Number of failed payment processing attempts")
            .register(registry);

        this.processingLatency = Timer.builder("payment_processing_duration")
            .description("Payment processing duration")
            .publishPercentileHistogram()
            .register(registry);
    }

    public void process(PaymentCommand command) {
        processingLatency.record(() -> {
            try {
                // process payment
            } catch (RuntimeException ex) {
                failedPayments.increment();
                throw ex;
            }
        });
    }
}

The metric names are just examples. The principle matters more: metrics should map to real operational questions.

For APIs, I care about request rate, latency, and error rate.

For Kafka consumers, I care about consumer lag, processing duration, failures, retries, and dead-letter events.

For database-backed services, I care about connection pool usage, connection acquisition time, query latency, and transaction failures.

Connection pools: bigger is not automatically better

HikariCP defaults are usually decent, but they are not a substitute for understanding the workload.

A bigger connection pool is not automatically better. It can move the bottleneck from the application to the database and make overload harder to recover from.

A starting point:

spring:
  datasource:
    hikari:
      maximum-pool-size: 10
      minimum-idle: 2
      connection-timeout: 30000
      idle-timeout: 600000
      max-lifetime: 1800000

The number I care most about is not the pool size itself. It is whether threads are waiting too long to acquire a connection.

Useful signals:

• active connections
• idle connections
• pending threads
• connection acquisition time
• transaction duration
• database CPU and lock waits
• number of application replicas

One common mistake is tuning one pod in isolation. If one service has 10 replicas and each replica has a pool size of 30, that service alone can open up to 300 database connections.

That might be fine. It might also be enough to hurt the database.

So I usually think about DB pool sizing at service level, not only pod level.

Logging: correlation IDs should not live in controllers

Logs become much more useful when every request has a correlation ID. But I do not like adding MDC manually inside controllers.

This is fragile:

MDC.put("requestId", requestId);

If it is not cleared correctly, thread reuse can leak context between requests. It also spreads logging plumbing across business code.

I prefer handling request IDs at the edge of the application, usually in a filter.

@Component
public class CorrelationIdFilter extends OncePerRequestFilter {

    private static final String HEADER = "X-Request-ID";

    @Override
    protected void doFilterInternal(
            HttpServletRequest request,
            HttpServletResponse response,
            FilterChain chain
    ) throws ServletException, IOException {

        String requestId = request.getHeader(HEADER);

        if (requestId == null || requestId.isBlank()) {
            requestId = UUID.randomUUID().toString();
        }

        MDC.put("requestId", requestId);
        response.setHeader(HEADER, requestId);

        try {
            chain.doFilter(request, response);
        } finally {
            MDC.clear();
        }
    }
}

Then include it in the log pattern:

<pattern>%d{ISO8601} [%thread] %-5level [%X{requestId}] %logger{36} - %msg%n</pattern>

For distributed systems, the next step is propagation: outgoing HTTP calls, messages, and async workflows should carry the same correlation context where possible.

Otherwise, the first service has good logs and everything after that becomes guesswork.

Configuration should fail safely

I prefer services that fail fast when required configuration is missing.

A bad configuration should not create a half-working service that starts successfully and fails later under traffic.

For example, required settings should be validated at startup:

@ConfigurationProperties(prefix = "payments")
@Validated
public class PaymentProperties {

    @NotBlank
    private String providerUrl;

    @Min(100)
    private int timeoutMillis;

    // getters and setters
}

This is especially useful in Kubernetes, where configuration comes from ConfigMaps, Secrets, Helm values, or environment-specific overlays. A typo in configuration should be caught during deployment, not during the first real request.

Timeouts everywhere

Production services should not wait forever.

That applies to:

• HTTP clients
• database calls
• message processing
• external provider calls
• retries
• shutdown
• connection acquisition

A missing timeout can turn a small dependency issue into thread starvation.

For example, an HTTP client should have explicit connect and read/response timeouts. The exact values depend on the service, but the absence of a timeout is usually the real bug.

The same applies to retries. Retrying without a timeout, backoff, or maximum attempt count can amplify an outage.

Startup behavior matters

A service being alive does not mean it is ready.

During startup, a Spring Boot service may still be warming caches, connecting to dependencies, loading configuration, initializing schedulers, or preparing consumers.

That is why readiness is important. Kubernetes should not route traffic to a pod just because the JVM process exists.

For workers, I also care about whether consumers start too early. Sometimes the application should finish initialization before it begins pulling messages from Kafka or executing scheduled jobs.

Practical checklist

Before I consider a Spring Boot service production-ready, I usually want:

• Actuator health endpoints enabled.
• Separate liveness and readiness probes.
• External dependency checks kept out of liveness.
• Graceful shutdown enabled and aligned with Kubernetes termination grace period.
• Background workers and consumers able to stop safely.
• Prometheus metrics exposed.
• Metrics that explain real production behavior, not only generic JVM stats.
• DB connection pool sizing checked against total replicas and database capacity.
• Correlation IDs added at the application edge.
• MDC cleared correctly.
• Required configuration validated at startup.
• HTTP clients, DB operations, retries, and shutdown paths configured with explicit timeouts.
• Logs, metrics, and health checks tested before the first incident.

Final thought

Getting a Spring Boot service into Kubernetes is easy. Making it behave well during restarts, dependency failures, traffic spikes, slow databases, and shutdowns is the real production work.

The goal is not to add every possible framework feature. The goal is to make failure modes visible, controlled, and boring.