techlist.io
KREN
line

Implementing SLO/SLI for Improved Reliability Part 3 - Service Application Cases (opens in new tab)

observabilitysite-reliability-engineeringresource-allocationservice-level-indicatorservice-level-objectiveuser-journeycritical-user-journeydashboard-visualization

SLI/SLO adoption is not merely a matter of choosing metrics; it requires redefining how a service is understood from the user’s perspective. LINE’s SRE team applies this approach by identifying critical user journeys, measuring reliability with clear criteria, and setting realistic objectives. The resulting data helps teams balance reliability, engineering costs, feature delivery, and incident response.

The Mindset Behind SLI/SLO

Understanding the Service and Users

  • Begin by identifying the services and features users depend on.
  • Map user journeys and select critical user journeys (CUJs) based on:
    • How frequently users use a feature
    • Whether the feature is essential to the service
    • Its relationship to business objectives
  • Aligning SLOs with business goals ensures that reliability work supports organizational priorities.

Communication and Collaboration

  • SLI/SLOs should be defined and managed collaboratively rather than by a single team.
  • Product or service owners define CUJs because they understand the user experience best.
  • Infrastructure teams provide scalable systems for collecting and managing metrics.
  • SREs build the measurement tools and processes used to monitor and improve reliability.
  • Shared ownership is essential so SLOs can guide both daily operations and new feature launches.

Implementing SLI/SLOs

Analyze Critical User Journeys

  • List the services and functions provided to users.
  • Ask:
    • Which features are used most often?
    • Which features are indispensable?
  • LINE examples include:
    • Account registration
    • Sending and receiving messages
    • User authentication and encryption
    • LINE Login
    • Profile information
  • The goal is not to include every feature, but to select the most important ones from the user’s perspective.

Define Service Level Indicators

For each CUJ, determine:

  • Measurement location: Choose the point that best represents the user experience, such as a gateway, frontend, or backend.
  • Measurement API: Select a representative API to avoid unnecessarily complex calculations.
  • Success criteria: Establish clear boundaries between successful and failed requests.

Common SLI criteria include:

  • Latency: Define a percentile, such as the 99.9th percentile, and the maximum acceptable response time.
  • Success rate: Define the required percentage of successful responses during the measurement period.

For example, a messaging service might require 99.9% of requests to complete within 500 milliseconds and 99.999% of all requests to receive successful responses. If a CUJ cannot be measured reliably or its success criteria cannot be defined clearly, it may be excluded or supported with a dedicated measurement metric.

Set SLO Targets

  • Define the reliability level the service must maintain over a specific period.
  • An example target is achieving the defined latency and success-rate criteria for 99.9% of a 28-day period.
  • Targets must be realistic:
    • Excessively high targets increase operational and infrastructure costs.
    • Excessively low targets can result in poor user experiences.
  • SLOs should balance reliability requirements with available resources.

Visualize Reliability

  • Provide dashboards that allow all stakeholders to understand the current SLO status quickly.
  • Show overall SLO performance and error-budget consumption, with detailed dashboards for individual CUJs.
  • Keep dashboards simple and easy to scan rather than displaying excessive information.
  • Use visual indicators such as:
    • Green for healthy performance
    • Orange for warning conditions
    • Red for missed objectives

How SLI/SLOs Are Used

Quantifying Reliability

  • Replace vague descriptions such as “the service is slow” with measurable statements.
  • Teams can identify issues such as latency exceeding a 400-millisecond SLI threshold or success rates falling below 99.99%.
  • Dashboards also help correlate periods of poor performance with incidents or operational changes.

Guiding Resource Allocation

  • SLOs show whether reliability targets are being met.
  • Error budgets indicate how much additional failure or downtime is acceptable.
  • When performance exceeds the SLO and the error budget is healthy, teams can invest more aggressively in:
    • New features
    • Faster release cycles
    • Product experimentation
  • When little error budget remains, resources can instead focus on prevention, remediation, and reliability improvements.

Supporting On-Call Operations

  • LINE uses alerts triggered by changes in error-budget status to help on-call teams recognize and respond to service issues.
  • SLO reviews are also incorporated into regular meetings and preventive reliability work.

SLI/SLO implementation works best as a shared, user-focused operating model. By combining clear CUJs, measurable criteria, realistic targets, and actionable dashboards, teams can make informed decisions about when to prioritize innovation and when to prioritize stability.