Build a Bash Report Script: sort, uniq, tee, xargs

Once you’re comfortable with the basics from our last post on basic Bash commands, the next useful skill is chaining them together. A single command rarely answers a real question on its own. “Which IP hit us the most today” takes three or four commands piped into each other, not one. This post builds a small but genuinely useful report script out of sort, uniq, cut, tr, tee and xargs, then covers a mistake that trips up a surprising number of scripts: looping over ls output.

Everything below ran against a real (synthetic) access log and a folder of files with spaces in their names, because the ls problem doesn’t actually show up until a filename has a space in it.

sort, uniq and cut: turning raw lines into a summary

Start with a log file where each line begins with a client IP:

wc -l access.log
10 access.log

cut pulls out one field by a delimiter, here a space, field one:

cut -d" " -f1 access.log | head -5
203.0.113.12
198.51.100.7
203.0.113.12
203.0.113.12
198.51.100.7

That’s just a list of IPs, one per request, duplicates and all. uniq -c collapses the duplicates and counts them, but it only works on sorted input, uniq only compares neighboring lines, so sort has to come first:

cut -d" " -f1 access.log | sort | uniq -c
      2 192.0.2.44
      3 198.51.100.7
      5 203.0.113.12

Add a second sort -nr on top and you’ve got the busiest IP first, which is usually the actual question:

cut -d" " -f1 access.log | sort | uniq -c | sort -nr
      5 203.0.113.12
      3 198.51.100.7
      2 192.0.2.44

That four-command chain, cut, sort, uniq -c, sort -nr, is one of the most reused patterns in shell scripting. Swap the field and the delimiter and it’ll count anything: status codes, user agents, usernames in an auth log.

tr: fixing the text itself, not just filtering it

sort and cut rearrange lines. tr edits the characters inside them. A common real case is a file that’s picked up stray \r characters from being edited on Windows at some point, which makes every line look fine to the eye but breaks exact string comparisons:

cat -A dos.txt | head -2
line one^M$
line two^M$

That ^M is the carriage return, invisible in a normal cat, obvious once you ask cat -A to show it. tr -d deletes a character outright:

tr -d "\r" < dos.txt > unix.txt
cat -A unix.txt
line one$
line two$

Clean line endings, confirmed by the same cat -A check. tr can’t match patterns like sed can, but for straight character swaps and deletions it’s faster to type and easier to read in a script.

tee: watch it happen and keep a copy

Pipe a command into a file with > and you stop seeing the output. tee writes to a file and still passes everything through to the next step, or to your screen:

cut -d" " -f1 access.log | sort | uniq -c | sort -nr | tee top-ips.txt
      5 203.0.113.12
      3 198.51.100.7
      2 192.0.2.44
--- saved to file ---
cat top-ips.txt
      5 203.0.113.12
      3 198.51.100.7
      2 192.0.2.44

Same output twice, once live on the terminal and once from the file tee wrote. In a cron job you won’t be watching it live, but tee is still useful there to keep a copy on disk while a wrapper script captures the same output for a notification.

Why looping over ls breaks on real file names

This is the one that actually catches people. Three files sitting in a folder, two of them with a space in the name, which is extremely normal for anything a human uploaded:

ls uploads
invoice 2026.pdf
notes.txt
quarterly report.csv

Now loop over that the way a lot of tutorials still show it:

for f in $(ls uploads); do echo "would process: [$f]"; done
would process: [invoice]
would process: [2026.pdf]
would process: [notes.txt]
would process: [quarterly]
would process: [report.csv]

Three files went in, five “files” came out. Bash split $(ls uploads) on whitespace, same as it would split any unquoted variable, so invoice 2026.pdf became two separate words. Any script that renamed, uploaded or deleted based on that loop would be operating on garbage, not the files you actually have.

The fix: glob instead of parsing text

You don’t need ls at all here, Bash can expand the file names itself:

for f in uploads/*; do echo "would process: [$f]"; done
would process: [uploads/invoice 2026.pdf]
would process: [uploads/notes.txt]
would process: [uploads/quarterly report.csv]

Three files in, three files out, spaces intact, because a glob expands to the real file names as separate words, it never joins them into one string that has to be re-split. For anything recursive, or where you need to filter by type, find paired with a null-delimited xargs is the other safe pattern:

find uploads -type f -print0 | xargs -0 -I{} echo "would process: [{}]"
would process: [uploads/invoice 2026.pdf]
would process: [uploads/notes.txt]
would process: [uploads/quarterly report.csv]

-print0 separates entries with a null byte instead of a newline, and xargs -0 reads them back the same way. A null byte can’t legally appear in a filename, so this is the one separator that’s always safe, no matter what someone named their file. xargs on its own is just “take these lines and turn each one into an argument for another command,” which is genuinely useful beyond the safety angle too:

find uploads -type f -print0 | xargs -0 wc -l
0 uploads/invoice 2026.pdf
0 uploads/notes.txt
0 uploads/quarterly report.csv
0 total

One wc -l call covering every file, with the names passed through intact, spaces and all.

Putting it all together: a real report script

Here’s all of it in one script, the IP summary piped into tee so the report is both printed and saved:

#!/bin/bash
set -euo pipefail

logfile="access.log"
outfile="report-$(date +%Y%m%d).txt"

{
    echo "Request report for $(date +%Y-%m-%d)"
    echo "Total requests: $(wc -l < "$logfile")"
    echo ""
    echo "Requests per IP:"
    cut -d" " -f1 "$logfile" | sort | uniq -c | sort -nr
    echo ""
    echo "Error responses (5xx):"
    grep -c ' 5[0-9][0-9]$' "$logfile" || true
} | tee "$outfile"

Run it:

./report.sh
Request report for 2026-02-13
Total requests: 10

Requests per IP:
      5 203.0.113.12
      3 198.51.100.7
      2 192.0.2.44

Error responses (5xx):
1
--- report file written to disk ---
ls -la report-*.txt
-rw-rw-r-- 1 asif asif 154 Feb 13 13:30 report-20260213.txt

The { ... } | tee "$outfile" wrapper is the trick worth remembering, grouping several commands in braces lets you pipe their combined output through one tee instead of adding >> after every line.

Frequently asked questions

Why does uniq -c need sort first?

uniq only compares each line to the one right before it. If the duplicates aren't next to each other, nothing gets collapsed. Sorting first guarantees every matching line is adjacent.

When should I reach for tr instead of sed?

tr is for simple character-level work, deleting a character, swapping one set of characters for another. The moment you need a pattern or a multi-character match, that's sed's job, not tr's.

Is it ever safe to parse ls output?

Basically no, not for anything that feeds into another command. ls is for a human to look at. For scripts, use a glob (for f in dir/*) or find ... -print0 | xargs -0, both of which handle spaces, newlines and anything else a real file name can contain.

What does xargs actually save me from writing?

A manual loop that reads lines and calls a command once per line. xargs does that in one line, and with -0 it does it safely, which a hand-rolled while read loop can also do but takes more code to get right.