Slow responses and 500s: diagnose from the edge inward.
The dashboard is green. The customer sees a broken page. Follow the request from the edge inward, using a real thirty-day incident as the evidence.
The whole chapter, in sixty seconds.
Slow responses and 500s often share an investigation, but they are not the same symptom. Establish what failed, for whom, and at which layer before choosing a fix. The mistake is to start inside, at the server you can log into, and work outward. Start at the edge and work in.
- Start at the edge, not the server.Start with the CDN record of requests, including failures that application monitoring may miss. Correlate with the gateway and origin before assigning a cause.
- A disconnect is evidence, not a diagnosis.CloudFront status 000 means the viewer closed the connection before a response. Timing and result fields help explain why.
- Measure the experience your probe misses.In this incident, 65% of observed image-requesting clients had no successful image response. CPU and simple probes did not reveal the impact.
Diagnose from the edge inward.
Each station has one record that only it keeps. Read it, form a hypothesis, then correlate the next layer before ruling anything out.
What the visitor actually got
Status, result type and time-taken per request. Count 000 and 5xx by path and by hour. Compare unique clients that succeeded with those that did not.
Establishes scope and timing. A status alone does not identify the failing component.
Whether the origin answered at all
Load balancer and ingress logs: backend connection closed, upstream timeout, probe results, pod churn at the same minute.
Separates routing and connection symptoms from application responses. Upstream delays can still begin in PHP or the database.
What PHP was doing
PHP-FPM active and idle workers, listen queue and slow log. Correlate web-server errors with container memory, termination reason and restarts; exit 137 alone does not prove an out-of-memory kill.
Shows worker pressure and runtime errors. Slow database calls can occupy PHP workers, so correlate the next layer.
What waited on the database
Slow query log, connection count, lock waits, and whatever plugin writes to the options table on every page view.
Tests whether slow queries, lock waits or exhausted connections explain the delays above.
The order matters because each layer can only see what reached it. An application log may show work that the visitor never received. An edge log shows the response outcome but may not explain its cause. Match timestamps, paths and request identifiers across layers until the explanation fits the evidence.
Thirty days of logs, and what the monitors missed.
This case uses a JavaScript image optimizer. The diagnostic method applies to WordPress too; the runtime-specific fix does not.
A museum's event site, built with a JavaScript front end. Uptime green, ordinary probes returning 200, the only pod at twelve percent CPU. A visitor sends a screenshot of a gallery with no pictures.
Nothing in application monitoring showed a problem, because nothing in the application had failed in a way it could see. The optimizer was waiting. The visitor was waiting. The edge recorded long waits and status 000: the viewer had closed the connection before a response. That field records the disconnect, not its cause. We had to correlate it with the optimizer behavior to explain the failure.
So we went to the edge. Thirty days of CDN standard logs for the distribution, grouped by unique client, by path and by hour. The first number ended the argument about whether this was rare.
The case / Evidence layer
231,178 edge requests in thirty days65% of observed image clients had no successful response.
The p95 duration of image requests was 30.09 seconds. The result was a gallery with missing images while ordinary uptime checks still passed.
2,032 observed clients requested images between September 10 and October 9, 2026. These are log-derived client groups, not a count of distinct people. Source: thirty days of CDN standard logs.
Bug one
187clientsThe hang: a wedged cache key, thirty seconds, then nothingepisodic, three times in the month
Bug two
1,203clientsThe rejection: a width the server stopped making in Mayevery single day
These mechanism counts are separate observations and may overlap; they are not slices of the outcome total above.
Image requests had a lower observed cache hit rate than HTML. Error caching depends on the CDN policy and status code; errors are not universally uncacheable. These ratios describe this incident, not a general rule.
Mechanism one: the hang
A client abort wedged the cache key.
The front-end framework's image optimizer shared one in-flight request per image variant. When a visitor aborted during a cold fetch, the shared promise never settled, and every later request for that variant queued behind it until the CDN timed out. It was an upstream bug, and the fix shipped in the framework's next minor release while we were still reading the logs. The site was two minor versions behind. A pod restart cleared it, three times in the month, and nothing paged anyone in between.
Mechanism two: the rejection
A May fix with no cache invalidation.
After an out-of-memory incident in May, the site's generated image widths were capped. Correct, and tested. But the HTML was cached at the edge for a year with no invalidation on deploy, so for five months it kept telling browsers to request the one width the server no longer made. Each request was rejected with a 400 in milliseconds. It never showed up as slow, it never showed up as an outage, and it cost more visitors than the dramatic bug did, every single day.
The dramatic bug was the smaller one. The quiet rejection, caused by our own correct fix, affected about 6.4 times as many observed clients: 1,203 versus 187.
Resolution: images moved off the application tier to an image CDN behind a thirteen-line loader, and the HTML cache lifetime was cut so a deploy is visible the same day. Both are chapters of their own.
Six more, found at every station.
The other slow-or-broken incidents in the operating record, numbered as they appear there. Each one was found at a different station of the ladder.
01503s at scale-upApplication
ObservedBursts of 503 errors, about 1,850 a week, clustered at the minutes the cluster scaled up.
CauseThe load balancer health check watched a helper process that was ready in six seconds, while PHP behind it needed thirty-five. New pods took traffic before they could serve it, and the autoscaler flapped between two and four replicas.
Evidence & resolutionLoad balancer log: 903 of the last 1,000 errors were backend connections closed at scale-up. Fixed by probing the real PHP path and raising the replica floor.
06Image service killed for memoryMedia
ObservedAn image service pod killed for memory once a day.
CauseThe CDN honored a 60-second cache lifetime instead of the application's four hours, so the origin re-optimized 3,160 images an hour for 303 distinct variants.
Evidence & resolutionCDN hit ratio alongside the container memory graph. Fixed with an origin-honoring cache policy and a memory arena cap.
15PHP tuning never reached productionApplication
ObservedOut-of-memory kills and 502s in traffic spikes despite per-site PHP tuning.
CauseThe init script edited the PHP-FPM pool file as the wrong user, the error was swallowed, and every site silently ran the default of 15 workers.
Evidence & resolutionReading the live pool config inside the container, not the repository: 22 deployments affected. Fixed by mounting the config read-only and asserting it at boot.
17A login page that failed only in productionApplication
ObservedLogin page returning 502 on production only.
CauseThe production web server had 64k FastCGI buffers while staging had 256k, and chunked responses without a content length also made the CDN skip compression.
Evidence & resolutionA configuration diff between the two environments. Fixed in infrastructure code for both, which is where the drift started.
16A database killed before it could startData
ObservedA fresh install that never came up.
CauseMySQL takes about 100 seconds to initialize on burst-starved nodes, and the liveness probe killed it at 60 to 90 seconds, every time.
Evidence & resolutionPod events: exit 137 at the same age on every attempt. Fixed with a five-minute startup probe.
19The screenshot test was measuring the wrong thingEvidence
ObservedA site that appeared to flip between two renderings for a week, with the object cache suspected.
CauseOur own observer was wrong: one screenshot suite alternated between staging and production against a single baseline. A controlled cache flush rebuilt cleanly and ruled out poisoning.
Evidence & resolutionFour days lost before the suite was split. Verify the instrument before the system.
Put the evidence where you can reach it.
Make these six decisions before the incident. The tools can vary; the evidence needs to be available and comparable.
- Keep edge logs you can query.Retain queryable CDN logs with status, duration and result type. They show the edge perspective; combine them with application logs and real-user signals.
- Track failures and their share of traffic.Use counts to see impact and rates to compare changing traffic volumes. Separate synthetic probes from visitor traffic and inspect disconnects alongside 5xx responses.
- Probe what serves traffic.A readiness check through the web server and PHP, with no database call, on the port that takes visitors.
- Expose PHP-FPM status and the slow log.Active workers, listen queue and requests over a threshold, scraped into the same dashboard as the edge counts.
- Set limits you have measured.Measure memory under representative load. Alert on OOMKilled and unexpected restarts, then inspect termination details instead of treating every exit 137 as a memory failure.
- Fetch the exact URLs your code fetches.Test the HTML, layout JSON and image variants the front end actually requests. A simple homepage-200 check does not verify that those dependencies work.
You are reading the example.
This handbook is served by the system it describes. Here is where the evidence lives for the page you are reading, checked against the deployed manifests on October 12, 2026.
Every request, logged.
CloudFront standard logs for this site land in an S3 bucket by the hour, with status, result type and time taken per request. The thirty-day reconstruction above was built from exactly this kind of record. A WAF in front carries managed SQL injection and account takeover rule sets plus a rate limit.
An APM agent in every pod.
The container image ships an application monitoring agent and the production deployment enables it. Errors group by message, so a fatal from an abandoned add-on shows up as a count, not a rumor.
Alerts on the database, not the symptom.
Cloud SQL alert policies cover free storage, CPU and read operations, delivered to email and a Slack channel. They fire before PHP starts timing out on connections.
Synthetic checks against the public URL.
Synthetic monitors hit udx.io from outside and page a Slack channel. The pod readiness probe, by contrast, asks a process supervisor whether nginx and PHP-FPM are running; it does not send a request through PHP. Chapter 02 shows the boot that gap sits in.
The ones we are asked most.
Why does my WordPress site return 502, 503 or 504 instead of 500?
A 500 reports an unexpected server error; PHP is one possible source. A 502 indicates an invalid upstream response, a 504 an upstream timeout, and a 503 temporary unavailability such as overload or maintenance. Several layers can emit these codes. Use the response and correlated logs to identify which one generated it.
Why is my site slow when the server shows low CPU?
Low CPU can mean requests are waiting on database locks, external services, an exhausted worker pool or a shared in-flight request. It does not prove spare capacity. Compare PHP-FPM queues, dependency timings and the edge response-time distribution before adding replicas.
What does CloudFront status 000 mean?
In CloudFront standard logs, 000 means the viewer closed the connection before the server could respond. It does not establish a fixed thirty-second timeout or prove the origin never received the request. Correlate time-taken and edge result fields with gateway and origin logs.
How do I reduce initial server response time in WordPress?
Serve anonymous pages from the CDN so PHP never runs for them. Put a persistent object cache in front of the database. Move WP-Cron off visitor requests. Then measure time to first byte at the edge, not from your own office, and read the p95, because the average hides the visitors who left.
Where should I look first when WordPress is down?
At the edge logs for the last hour: what status did visitors get, on which paths, and how long did they wait. Then the gateway for backend connection errors and pod restarts at the same minute. Then PHP-FPM and the web server error log. Then the database. Working inward means you stop at the first layer that explains the symptom instead of guessing from the inside out.
Technical references: CloudFront log fields and error caching behavior.
Stuck between a green dashboard and an angry customer?
Tell us where the site runs, what visitors experience and what you have already measured. Talk with an engineer about diagnosing the incident and improving the checks around it.