You are given an Nginx access log in the default combined format, 2 GB for one day. In a bash script, using only standard Unix tools (awk, sort, uniq, grep, cut), report: total requests, requests per status class (2xx, 3xx, 4xx, 5xx), the top 10 client IPs by request count, the top 10 paths returning 5xx, the 95th percentile response size, and the busiest minute. The script must be safe to run on production logs and fast enough to finish in under a minute on a laptop.

Usage: ./access_report.sh <access.log>

Output sections, in this order:
  1. Total requests
  2. Status classes:   class  count  percent
  3. Top 10 IPs:       count  ip
  4. Top 10 5xx paths: count  path
  5. Busiest minute:   dd/Mon/yyyy:HH:MM  count
  6. p95 bytes

Exit codes: 0 ok, 2 usage, 1 file missing or empty

Nginx combined log format, one request per line:

192.168.4.21 - - [03/Nov/2026:10:41:07 +0530] "GET /api/orders?id=42 HTTP/1.1" 200 5123 "https://shop.example/cart" "Mozilla/5.0 ..."
|            | | |                           |                                 |   |    |                            |
$1 remote_addr    $4 [time (strip the [)   $6 method $7 path $8 proto     $9 status  $10 bytes   $11 referer   $12+ user agent

Fields split on whitespace with awk. The request line is quoted, so $6 begins with a quote
and $7 is the path. Status is $9 and bytes is $10 (a dash when nothing was sent).
Minute key = substr($4, 2, 17)  -> 03/Nov/2026:10:41

1. Validate input: one argument, file exists, is readable and non-empty; otherwise print usage to stderr and exit 2 or 1.
2. Total and status classes in one awk pass: take the first character of $9, count into an array, print with percentages at END.
3. Top IPs: awk '{print $1}' | sort | uniq -c | sort -rn | head -10. The sort is the expensive step; LC_ALL=C makes it several times faster because it compares bytes, not locale-aware characters.
4. Top 5xx paths: filter in awk ($9 ~ /^5/), print $7 with the query string removed (split($7, a, "?")), then the same sort and uniq pipeline.
5. Busiest minute: awk extracts substr($4, 2, 17), counts per key, and at END keeps the maximum; no sort needed.
6. p95 bytes: awk prints $10 when it is numeric, sort -n, then pick the line at index ceil(0.95 * n) with awk reading the sorted stream and the count passed in.
7. Wrap the sections in functions, give each a header, and make the script create a small sample log when run with --demo so it can be tested without a real file.

Why awk and not a loop

A while read line loop in bash runs the shell's interpreter once per line, with a fork for every cut or grep you call inside it; on 10 million lines that is hours. awk reads the file once in C, splits fields for you, and keeps counts in associative arrays, so a full pass is a few seconds. The rule interviewers want to hear: bash orchestrates, awk and sort do the work. The one place bash loops are fine is over the handful of result lines at the end.

sort is the bottleneck, so help it

sort | uniq -c | sort -rn sorts the whole column once and the counts once more. Two things change its speed by a large factor. LC_ALL=C switches to byte comparison, which is often 3 to 10 times faster than a UTF-8 locale. sort -S 50% (GNU) lets it use memory instead of temp files; leave it out for portability but mention it. For the top-N question there is also a pure-awk answer that never sorts: count into an array, then at END loop to find the top 10, which is O(n) instead of O(n log n). Show both; the sorted pipeline is what most people write, the awk-only version is what shows you understand cost.

Percentiles need all the values, or an approximation

A true p95 needs the sorted list of all 10 million sizes, which is why the script sorts numerically and indexes into the stream at position ceil(0.95 * n). Reading the count from the first pass avoids a second wc -l. For a live system you would not do this on raw logs at all: a histogram metric in Prometheus gives p95 from a few buckets with constant memory. Saying "for one-off analysis sort; for monitoring use histogram buckets" is the complete answer.

Safety on production logs

The script only reads. It never writes next to the log, never uses sed -i, and never follows a path from an untrusted variable without quoting. set -euo pipefail makes a failing stage in a pipeline fail the script, which matters because awk ... | sort | head would otherwise hide an unreadable file behind head's success. One exception to know: head closing the pipe early can make sort receive SIGPIPE, which pipefail reports as a failure; the script handles that by sorting into a variable only for the top-N sections, or by tolerating exit 141. Keep the log on the machine: copying 2 GB over the network to analyse it on a laptop is slower than running the script in place.

Correctness tests. Run with --demo, which writes a 2,000-line sample log with known counts (for example exactly 120 5xx lines, one IP that appears 300 times, and a minute with 300 requests), and assert each section matches. Check that a line with - for bytes is skipped in the percentile but counted in totals. Check a path with a query string collapses to its base path in the 5xx table.

Edge cases. Missing argument exits 2; missing file exits 1; empty file prints "0 requests" and exits 1; a log with no 5xx prints an empty table, not an error; IPv6 clients work because $1 is still the first field.

Performance. Time the script on a 1 million line file; the sort sections dominate, and LC_ALL=C should show a visible difference. State the complexity: O(n) for the awk passes, O(n log n) for the sorts.

What a strong answer adds. Field numbers from memory for the combined format; the LC_ALL=C trick; the awk-only top-N; knowing that p95 from logs is a one-off and histograms are the production answer; and --previous-style thinking: run the report on the rotated file, not the live one, when exact day boundaries matter.

• Wrong field numbers. A custom log format shifts every column and the report is silently wrong. Mitigation: print one parsed line at the start as a sanity check.
• Query strings in paths. /api/orders?id=1 and ?id=2 look like different paths. Mitigation: split on ? before counting.
• Dash in the bytes column. sort -n puts - first and the percentile is off by the number of such lines. Mitigation: filter numeric values in awk.
• SIGPIPE from head under pipefail. The script exits 141 after printing the right answer. Mitigation: let sort finish into a variable, or handle exit 141 for that pipeline.
• Reading the live file during rotation. Counts straddle two files. Mitigation: analyse the rotated file or pass both.
• Running on a laptop after scp of 2 GB. Slower than running in place. Mitigation: run on the host with nice and ionice.