'The Server Is Slow' With Zero Budget and No APM: A Command-Line Triage

It is late, a client in Nairobi is messaging you that the system is crawling, and you open your monitoring dashboard to see what is wrong. There is no dashboard. There is no Datadog, no New Relic, no Grafana, because nobody budgeted for one and the whole thing runs on a single modest VPS. You have SSH and the tools that came with the distro. That is the entire toolkit.

Every article you find says the same two useless things: buy an APM, or check your monitoring, which assumes you already have the monitoring you do not have. Neither helps at 1am. What you actually need is an order of operations: a short, fixed sequence that isolates whether the problem is CPU, memory, disk, network, the app, or the database, using tools that are already installed.

That order exists, it is free, and it is fast. The trick is to follow it rather than jumping straight to restarting things and hoping. Restarting is not diagnosis, it is a way of destroying the evidence before you have looked at it.

A five-minute triage order

The reason to fix an order is that a slow server has too many plausible causes to check at random. Going CPU, memory, disk, network, then app narrows it fast, because each layer's tool rules the layer in or out in seconds.

Step 1: load average and CPU

uptime
# 01:12:44 up 40 days,  load average: 8.13, 6.40, 4.02

Compare load to your core count (nproc). A load of 8 on a 2-core box means processes are queued four deep waiting for CPU. Rising numbers (the first figure higher than the third) mean it is getting worse, not recovering. Then open top and press P to sort by CPU. One process at 100 percent tells a completely different story from forty processes at 5 percent each. Note whether the time is in user space (your code) or shown under wa, I/O wait, which is your cue that the real problem is disk, not CPU, and you should jump to step 3.

Step 2: memory and swap

free -h
#               total        used        free      shared  buff/cache   available
# Mem:           3.8Gi       3.4Gi       120Mi        60Mi       280Mi       180Mi
# Swap:          2.0Gi       1.6Gi       420Mi

The number that matters is available, not free; Linux uses spare memory for cache and gives it back on demand. But if Swap used is non-trivial and growing, you have found a strong suspect: every access to swapped-out memory is a disk read pretending to be RAM, and it makes everything feel slow at once. On a small VPS this is the single most common cause, and the honest fix is often a bigger instance, which is a real cost decision, not a config trick.

Step 3: disk I/O

iostat -x 2 3    # from the sysstat package
# Device   r/s   w/s   rkB/s   wkB/s  await  %util
# vda     12.0 340.0   192.0  8200.0   48.2   98.7

%util near 100 means the disk is saturated and is the bottleneck. High await (milliseconds a request waits) confirms requests are queuing at the device. To find who is doing the I/O, run iotop -o (only processes actually doing I/O). On a cheap VPS with network-backed storage this is frequently where "the server is slow" actually lives, and no amount of application tuning fixes a saturated disk.

Step 4: network and connections

ss -s              # socket summary
ss -tn state established | wc -l   # count of established TCP connections

A pile of connections stuck in TIME-WAIT or a connection count near a limit points at connection exhaustion rather than compute. This is also where you notice a flood of inbound requests, which is a different problem entirely, covered in Surviving a Bot Flood on a $20 VPS.

Step 5: the database and the application

Only now, having confirmed the machine itself is healthy or having found which resource is saturated, do you look at the app. For Postgres, the single most useful query is the live activity view:

SELECT pid, now() - query_start AS runtime, state, wait_event_type, left(query, 80)
FROM pg_stat_activity
WHERE state = 'active'
ORDER BY runtime DESC;

A query in active state with a runtime of several seconds is your culprit, in the flesh. If you see many rows blocked on the same lock, you have contention, not a slow query. Turn on the slow query log to catch these when you are not watching. For MariaDB and MySQL, the equivalent walk-through is in A MariaDB Slow Query Log Playbook.

Separating a slow database from slow application code

This is the distinction that people with no profiler get wrong most often. The test is simple: is the database busy or idle while requests are slow?

What you observe	Bottleneck	Next move
pg_stat_activity shows long active queries; slow log filling	The database	Explain the slow query, add an index, or fix the N+1
DB mostly idle, but access log $request_time is high	Application code	Profile the endpoint, check external API calls, check locks in the app
Both idle, high I/O wait in top	Disk or swap	Back to steps 2 and 3

If the database is idle and the request is still slow, the query was never the problem, and a repeating pattern of one query per row is the usual reason application time balloons, which is exactly the N+1 query problem.

Finding the slow request without tracing

You do not need distributed tracing to know which endpoint is slow. Add request timing to the nginx log format once:

log_format timed '$remote_addr $request "$status" rt=$request_time';
access_log /var/log/nginx/access.log timed;

Then, during or after the incident, sort by the timing field:

awk '{print $NF, $0}' /var/log/nginx/access.log | sort -rn | head

The slowest requests, and their exact URLs, float to the top. That is your list of endpoints to investigate, produced for free.

Capture evidence during the incident, not just fix it

The cruelest part of no monitoring is that when load returns to normal, all the evidence is gone. So while it is bad, spend thirty seconds snapshotting it to a file:

ts=$(date +%s)
{ uptime; echo; free -h; echo; top -b -n1 | head -20; echo; iostat -x 1 3; } > "/tmp/incident-$ts.txt"

For Postgres, dump the activity view to a file too. Five redirected commands give you more to work with tomorrow than any dashboard you do not own.

What people get wrong

Restarting first. Bouncing the app or the database usually makes the symptom disappear for a while, and destroys every piece of evidence about the cause. You will be back in the same place next week with nothing learned. Snapshot first, restart second, and only if you must to restore service.

Reading load average as a percentage. A load of 4 is fine on an 8-core box and a crisis on a 2-core box. Load is a count of runnable processes, not a percentage. Always compare it to nproc.

Trusting free's "free" column. A nearly empty free column alarms people into thinking they are out of memory when Linux is just using RAM for cache, which is healthy. Read available, and watch swap, not free.

Assuming an APM would have told you instantly. It would have drawn you a graph, but the graph still would have needed you to read it in this same order. The command-line path is not a poor substitute; it is the same reasoning without the subscription.

What to instrument for free before the next incident

You do not need a paid tool to stop flying blind. In priority order:

1. Turn on the database slow query log with a sane threshold, so the next slow query is recorded whether or not you are watching.
2. Add $request_time to your access log format now, so the slowest-request analysis works retroactively.
3. Install sysstat and enable its collector (sar), which records CPU, memory and I/O history to disk for free, the closest thing to a free time-series database you already have.
4. Write the five-command snapshot above into a small script named something you will remember at 1am, like whyslow, so capturing evidence is one command, not five you have to recall under pressure.

When the next message arrives that the system is crawling, you will not need a dashboard. You will have an order to follow and the tools to follow it, which is what diagnosis actually is. For a broader take on keeping production systems trustworthy, see Optimizing ERPNext Performance.