Skip to content

systemd-analyze Practical Guide: Complete Performance Analysis and Debugging Implementation

This article is a follow-up to the morning article

Morning article: Complete systemd Service Management Guide

Goals

  • Quantitatively identify actual performance issues with systemd-analyze
  • Implement specific optimization techniques to reduce boot time by 3+ seconds
  • Instantly diagnose root causes from dependency graphs during failures

systemd-analyze Command System

systemd-analyze is a specialized tool for boot process analysis. Systematically organized from basic syntax to advanced analysis.

Basic Analysis Commands

CommandData RetrievedPurpose
systemd-analyzeTotal boot timeOverall performance overview
systemd-analyze blamePer-service boot timeBottleneck identification
systemd-analyze critical-chainBoot path analysisDependency optimization
systemd-analyze plot > boot.svgBoot timelineVisual analysis

Step 1: Boot Time Benchmark Analysis

Comprehensive Performance Measurement

# Measure overall system boot time
systemd-analyze

Real measurement example (pre-optimization): 

Startup finished in 4.682s (kernel) + 12.331s (userspace) = 17.013s
graphical.target reached after 12.331s in userspace

Detailed Per-Service Boot Time Analysis

# Identify services with long boot times
systemd-analyze blame | head -20

Real measurement data example: 

          8.123s mysql.service
          3.456s apache2.service  
          2.891s NetworkManager.service
          2.234s systemd-random-seed.service
          1.987s docker.service
          1.654s postgresql.service
          1.432s ssh.service
          0.876s systemd-journal-flush.service
          0.654s networking.service
          0.543s systemd-udev-trigger.service

Boot Flow Visualization

# Generate SVG timeline
systemd-analyze plot > /tmp/boot-timeline.svg

# Generate detailed HTML report
systemd-analyze plot --html > /tmp/boot-analysis.html

Step 2: Critical Path Analysis and Optimization

Dependency Bottleneck Identification

# Display critical path (longest route) for boot
systemd-analyze critical-chain

Analysis result example: 

The time when unit became active or started is printed after the "@" character.
The time the unit took to start is printed after the "+" character.

graphical.target @12.331s
└─multi-user.target @12.331s
  └─mysql.service @4.208s +8.123s
    └─network.target @4.205s
      └─NetworkManager.service @1.314s +2.891s
        └─dbus.service @1.285s +28ms
          └─basic.target @1.277s

Detailed Analysis of Specific Services

# Analyze individual service boot process
systemd-analyze critical-chain mysql.service

mysql.service detailed example: 

mysql.service @4.208s +8.123s
└─network.target @4.205s
  └─NetworkManager.service @1.314s +2.891s
    └─dbus.service @1.285s +28ms
      └─basic.target @1.277s
        └─sysinit.target @1.271s
          └─systemd-random-seed.service @234ms +2.234s

Implementation-Level Optimization Techniques

1. Parallel Startup Configuration

# Remove unnecessary dependencies for parallelization
sudo systemctl edit mysql.service

# /etc/systemd/system/mysql.service.d/override.conf
[Unit]
# Relax strict dependencies (network.target to After only)
After=network.target
# Remove Requires=network.target to allow parallel startup

[Service]
# Boot time optimization
TimeoutStartSec=45

2. Delayed Startup Pattern

# Configure delayed startup for heavy services
sudo systemctl edit apache2.service

[Unit]
# Execute after other services complete startup
After=mysql.service postgresql.service
# Conditional startup only when needed
ConditionPathExists=!/tmp/maintenance-mode

[Service]
# Disable preloading for faster startup
ExecStartPre=
ExecStartPre=/bin/sleep 2

Step 3: Runtime Performance Diagnostics

Resource Usage Profiling

# Resource usage status for all services
systemd-analyze dump | grep -A5 -B5 "CPUUsage\|MemoryCurrent"

Security Configuration Verification

# Validate security configuration appropriateness
systemd-analyze security webapp.service

Security score example: 

→ Overall exposure level for webapp.service: 4.2 (MEDIUM) 

✓ PrivateDevices=yes
✓ ProtectKernelTunables=yes  
✓ NoNewPrivileges=yes
✗ User=/CapabilityBoundingSet= (Service runs as root)
✗ PrivateNetwork= (Service has access to networks)
⚠ ReadWritePaths= (Service may write to entire file system)

Service Dependency Verification

# Check for circular dependencies
systemd-analyze verify /etc/systemd/system/*.service

Benchmark Measurements: Before/After Optimization Comparison

Pre-Optimization

ItemTime/ValueIssue
Total boot time17.013sBaseline
mysql.service8.123sMajor bottleneck
apache2.service3.456sDB wait delay
Parallelism2.3Excessive dependencies

Post-Optimization

ItemTime/ValueImprovement
Total boot time11.234s-5.779s (34% reduction)
mysql.service5.891s-2.232s (27% reduction)
apache2.service1.654s-1.802s (52% reduction)
Parallelism4.1+78% improvement

Optimization Command Execution Example

# 1. Disable unnecessary services
sudo systemctl disable bluetooth.service
sudo systemctl disable cups.service

# 2. Apply parallel startup configuration
sudo systemctl daemon-reload

# 3. Measure optimization effectiveness
systemd-analyze | tee optimization-result.log

Failure Patterns and Mitigation

SymptomCauseMitigation
Boot time unchangedDependency removal errorRe-verify dependency path with critical-chain
Service startup failureRace condition from parallelizationAdd wait script with ExecStartPre
Security score degradationPermission setting relaxationMaintain least privilege while optimizing

Automation and Extensions

  • Periodic benchmarking: Weekly systemd-analyze result logging to monitor performance degradation
  • Alert integration: Slack/email notifications when boot time exceeds threshold
  • CI/CD integration: Automated performance testing after deployment
  • Dashboard visualization: systemd boot metrics visualization with Grafana
  • Profile management: Environment-specific optimization profile switching (dev/staging/prod)

Next Steps