The argument over text streams vs object pipelines gets treated as a matter of taste — Unix people like bytes, Windows people like objects, everyone goes home. It is not a matter of taste. It is a specific, scoreable engineering trade: Unix bought composition between programs whose authors never spoke to each other, and paid for it with fifty years of hand-rolled parsers, newline-hostile filenames, and no typed failure channel. Both halves of that sentence are true, and an honest scorecard has to carry both.
The complication is that the doctrine everyone argues from is not the doctrine anyone wrote. The single most-quoted line about Unix design — “Write programs to handle text streams, because that is a universal interface” — does not appear in the 1978 Bell Labs text it is universally attributed to. It is a 1994 condensation by a historian. So before scoring the trade, it is worth establishing what was actually claimed, by whom, and when.
The Doctrine Nobody Actually Wrote
The canonical formulation of the Unix philosophy is the foreword by M. D. McIlroy, E. N. Pinson and B. A. Tague to The Bell System Technical Journal 57(6) Part 2 (July–August 1978), the Unix issue. It contains four numbered maxims, and the phrase “universal interface” is not among them:
- Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new features.
- Expect the output of every program to become the input to another, as yet unknown, program. Don’t clutter output with extraneous information. Avoid stringently columnar or binary input formats. Don’t insist on interactive input.
- Design and build software, even operating systems, to be tried early, ideally within weeks. Don’t hesitate to throw away the clumsy parts and rebuild them.
- Use tools in preference to unskilled help to lighten a programming task, even if you have to detour to build the tools and expect to throw some of them out after you’ve finished.
The three-line version — do one thing well, work together, handle text streams because that is a universal interface — is Peter Salus’s condensation in A Quarter Century of UNIX (1994, pp. 52–53), offered as a summary of McIlroy’s position. It is a fine summary. It is not a primary source, and the doctrine that the entire industry now quotes is therefore a paraphrase of a post-hoc summary of an observation about work that had already been done for nine years.
Read maxim 2 closely and you can see what got lost in compression. McIlroy, Pinson and Tague never said “text.” They said: don’t clutter, avoid stringently columnar formats, avoid binary formats, don’t require interactivity. That is a set of constraints on machine-readability, not an endorsement of unstructured bytes. JSON satisfies every one of those four constraints. The 1978 foreword does not rule out structured data at all; the 1994 paraphrase does, and the paraphrase is what got canonized.
| Source | Date | Form | What it says about output format |
|---|---|---|---|
| McIlroy, Pinson & Tague, BSTJ foreword | 1978 | 4 numbered maxims | Don’t clutter; avoid stringently columnar or binary formats; don’t insist on interactive input |
| Salus, A Quarter Century of UNIX | 1994 | 3 lines, attributed to McIlroy | “Text streams, because that is a universal interface” |
| Raymond, The Art of Unix Programming §1.6 | 2003 | 17 named rules | Rule of Representation, Rule of Transparency, Rule of Composition — several in direct tension |
Doctrine inflation is measurable: four points, then three lines, then seventeen rules — and the rules contradict each other (Raymond’s Rule of Parsimony, “write a big program only when it is clear by demonstration that nothing else will do,” sits badly against his Rule of Extensibility). One related correction, since it travels with this material: the 1984 Pike and Kernighan paper that mounts the sharpest internal critique of all this is “Program Design in the UNIX Environment”, AT&T Bell Laboratories Technical Journal 63(8) Part 2, October 1984 — it is not titled “cat -v considered harmful.” That slogan was attached by later readers and appears nowhere in the text.
What Text Actually Bought: Composition With Zero Prior Agreement
Strip the doctrine away and the engineering win is still enormous, and it is more specific than “text is simple.”
A pipe is a real kernel object, not a metaphor. In the Sixth Edition kernel, pipe.c allocated an inode on the root device (ialloc(rootdev)) with #define PIPSIZ 4096 — a pipe was literally a file, and flow control was sleep/wakeup against it, the reader sleeping on empty and the writer sleeping at the size limit. Modern numbers differ and are routinely conflated: per pipe(7), Linux capacity is 16 pages (65,536 bytes at 4 KiB pages) since 2.6.11, tunable via F_SETPIPE_SZ and bounded by /proc/sys/fs/pipe-max-size, while PIPE_BUF — the atomicity boundary for concurrent writers — is 4,096, and POSIX guarantees only 512. Capacity is not PIPE_BUF, and neither is portable.
That primitive gives you three properties at once, which is more than most hand-rolled stream plumbing manages: a bounded buffer, backpressure (both ends block), and cancellation — when every read end closes, the writer takes SIGPIPE, or EPIPE if the signal is ignored. That is why yes | head -3 terminates instead of filling a disk. Cancellation propagating upstream is the part in-process queues almost always forget.
But the primitive is not what made composition work. The connector did. Pike and Kernighan put it precisely in 1984: “Filename argument patterns are expanded by the shell for all programs, without prearrangement in each command. The same is true of input and output redirection.” When the Seventh Edition shell gained backquotes, “no changes were made in any other program … all programs called by the shell acquire the feature transparently and uniformly.” Centralizing the connector, rather than shrinking the programs, is the load-bearing decision — and the same lesson shows up whenever you choose between orchestration and choreography in a distributed system.
Here is the property that actually matters, stated as a design metric rather than a slogan: score an interface by how much the two ends must agree in advance. For byte streams that number is approximately zero. Two engineers who never met, at companies that did not exist when the tools were written, in languages invented decades apart, compose successfully because the contract is “bytes, mostly lines.” N tools compose without N² adapters and without any coordination between their authors.
The most famous demonstration is Bentley, Knuth and McIlroy in CACM 29(6), June 1986: Knuth wrote a roughly ten-page literate WEB/Pascal program for the common-words problem; McIlroy answered with a six-stage shell pipeline — tr -cs A-Za-z '\n' | tr A-Z a-z | sort | uniq -c | sort -rn | sed ${1}q — and reviewed Knuth’s with the line “Knuth has shown us here how to program intelligibly, but not wisely.” Six programs, written by different people over more than a decade, none of them designed for word frequency, composed into a correct solution with zero coordination cost. Nothing about object pipelines has ever matched that number.
What It Cost: No Schema, So Every Consumer Writes a Parser
The invoice arrives at the receiving end. With no schema, every consumer writes a parser, and the parsers are wrong.
The canonical demonstration is the filesystem itself. A Unix filename may contain any byte except / and NUL — including spaces, tabs, and newlines. Since the universal interface is newline-delimited by convention, ls output cannot be parsed safely, and neither can any other line-oriented listing. GNU’s answer was find -print0 and xargs -0: a second, NUL-delimited interface bolted alongside the first. That pair is not a footnote, it is an admission — the universal interface needed a parallel non-universal interface to be correct.
The deeper hole is that there is no typed failure channel. An exit status is eight bits. stderr is a second untyped stream, not a fault protocol. SIGPIPE kills a producer silently by default. Everything downstream exists to patch this: set -o pipefail, PIPESTATUS, CI retry conventions, structured logging, and every workflow engine from Airflow to Temporal — which is precisely why observing a distributed workflow is hard work rather than a free property of the model. Adopt the composition model and you must ship the fault protocol yourself, because Unix never did.
The UNIX-HATERS Handbook (IDG Books, 1994; eds. Garfinkel, Weise and Strassmann, with a foreword by Don Norman and an anti-foreword by Dennis Ritchie) is usually remembered as comedy. Its substantive chapters — “‘rm’ Is Forever,” “Error Messages and Error Checking, NOT!,” “Consistently Inconsistent” — are structurally the same complaint: when the connector is untyped, the errors are untyped too. The book is a bug report against the cost side of this trade, and the bug is real.
Inconsistency compounds from the same root. ps(1) documents that it accepts “Unix options … BSD options … GNU long options” and warns that “options of different types may be freely mixed, but conflicts can appear” — one binary, three grammars, because nothing structural ever forced them to reconcile.
Scoring the Trade Honestly
| Dimension | Text streams | Object pipelines |
|---|---|---|
| Prior agreement required | ≈ zero — “bytes, mostly lines” | A shared type system, runtime, and version |
| Composability across authors | Total; tools written decades apart compose | Only within one ecosystem |
| Composability across machines | Native — the stream is the wire format | Requires serialization; fidelity is lost at the boundary |
| Typing | None; every field is a string until proven otherwise | Real types, real members, real null-vs-empty distinction |
| Discoverability | Read the man page, then guess | Reflection — enumerate members at runtime |
| Parsing cost | Paid by every consumer, every time, incorrectly | Paid once, by the runtime |
| Failure channel | 8-bit exit status plus untyped stderr | Typed exceptions with structure |
| Debuggability | You can tee any stage and read it with your eyes | Needs a formatter to be human-visible at all |
| Degradation | Graceful — an unknown field is just more bytes | Brittle — an unknown type is an error |
| Longevity | A 1979 pipeline still runs | Objects do not survive a runtime major version unchanged |
The row that decides most real arguments is the first one. Objects lose at boundaries precisely because they raise the prior-agreement number, and boundaries — between processes, machines, organizations, decades — are exactly where integration cost lives. This is also why “we’ll define a proper schema” is usually correct inside one system and usually wrong between organizations, and it is the same reasoning that puts an anti-corruption layer at the edge of a bounded context rather than a shared type library across the whole estate.
The Three Counter-Experiments
Three serious attempts have been made to change the payload while keeping the pipe. None of them came from people who misunderstood Unix.
PowerShell. Jeffrey Snover’s Monad Manifesto is dated 8 August 2002 and proposed piping .NET objects instead of unstructured text, explicitly to kill brittle parsing. It shipped as Windows PowerShell in 2006. Snover came out of Unix systems management — this was a refusal of one of Unix’s trades, not ignorance of them. The design delivers exactly what it promised inside its boundary: Get-Process | Where-Object CPU -gt 100 needs no parser because CPU is a property with a type. Cross a machine boundary and the promise thins — remoting serializes objects to CLIXML and what arrives on the other side is a deserialized property bag with its methods stripped. The type system is the coupling and the value.
jq. Stephen Dolan’s jq — first commit 18 July 2012, released October 2012 — was pitched as “sed for JSON.” It changed nothing about the pipe. It is an ordinary filter that reads a byte stream on stdin and writes one on stdout.
Nushell. Nushell (2019; Jonathan Turner, Yehuda Katz, Andrés Robalino) makes structured tables the default payload and bills itself as Unix pipelines with PowerShell’s structure. Its structure is real within the shell; the moment it calls an external program it is back to bytes, which is why it ships from json, from csv, from yaml and friends to re-impose structure at the boundary.
jq | PowerShell | Nushell | |
|---|---|---|---|
| Payload on the wire | Bytes (JSON text) | .NET objects in-process; CLIXML across machines | Structured tables in-shell; bytes to external commands |
| Where the schema lives | In-band, self-describing | In a shared type system | In the shell’s own value model |
| Crosses a process boundary | Yes, unchanged | Only within the PowerShell host | Only via serialization |
| Crosses a machine boundary | Yes — it is already a byte stream | With fidelity loss | With fidelity loss |
| Needs a shared runtime | No | Yes | Yes, for structured payloads |
| Works with tools that predate it | Yes — anything emitting JSON | No | Only as text |
| What it actually changed | The format, not the plumbing | The plumbing and the format | The plumbing and the format |
What jq Actually Proved
It is tempting to read jq‘s ubiquity as “objects won.” That is the wrong reading, and the right one is more interesting.
jq proved that the pipe was never the problem — the format was. The 2026 settlement across essentially every serious CLI is: byte stream as transport, JSON as schema, jq as the schema-aware filter. kubectl -o json, aws --output json, gh --json, docker inspect, terraform show -json — all of them emit a self-describing structured document down an ordinary stdout, and all of them are consumed by a filter that understands structure but demands nothing of the producer beyond a meta-format. The number that made text win — prior agreement required, ≈ zero — stays at approximately zero, because JSON’s schema travels in band. Neither end needs a shared type system, a shared runtime, or a shared version. They need a shared grammar, which is a far cheaper thing to agree on.
So jq is not the object pipeline’s victory; it is the object pipeline’s insight delivered on the text stream’s terms. The pipe survived. The format did not. Unstructured line-oriented output lost — quietly, without an argument, over about a decade — and the tools that replaced it did so without touching the plumbing at all.
Score JSON honestly too, since it inherited some of the same problems it fixed. It has no framing for infinite streams, which is why JSON Lines exists and why jq grew streaming modes. RFC 8259 declines to fix a number precision and warns that interoperability is only reliable within IEEE 754 double range — so large integers, IDs especially, get mangled by careless consumers. It carries no binary type, no comments, and no schema versioning of its own. And jq‘s expression language is a genuine second learning curve, which is the tax you pay for putting the structure awareness in the filter instead of the connector.
The Falsification That Matters: isatty()
The usual way to argue that the Unix philosophy failed is to count flags. GNU ls documents on the order of 52 options (coreutils 9.11); POSIX.1-2024, Issue 8 specifies 26; the Seventh Edition synopsis was ls [ -ltasdrucifg ] — eleven. Those counts drift by version and by who is counting, and “the tools got bigger” is an aesthetic complaint that a defender can always absorb.
There is a much better falsification available, and it is a prediction, not a preference. In that 1984 paper, Pike and Kernighan condemned Berkeley ls for switching its output format depending on whether stdout was a terminal, and wrote: this “ad hoc adjustment of the output format depending on the destination is not only distasteful, it is unique — no standard UNIX command has this property.”
In 2026 it is universal. ls --color=auto and its column behavior, Git’s automatic pager and colorization, grep‘s coloring, Docker’s progress rendering, jq‘s own default pretty-printing and coloring — the list is every tool anyone uses daily. That is not a matter of taste that could go either way. It is a testable claim about the design space, made by two of the authors of the model, and the design space falsified it outright.
The reason it lost is the whole article in one mechanism. isatty() is a one-bit type system, negotiated out of band. Because a byte stream carries no channel on which a consumer can say “I am a program, give me the machine form,” the only way a producer can adapt is to sniff its own file descriptor and guess. Every --color=auto, every automatic pager, every progress bar that vanishes under a pipe is a program inferring its consumer’s type from a side channel because the interface has no way to state it. Pike and Kernighan were right that this is ad hoc. They were wrong that it was avoidable, because the alternative — negotiating the format in-band — is precisely the thing text streams cannot do and object pipelines exist to provide.
systemd: Argue the Connector, Not the Binary Count
The same confusion — arguing size when the real variable is coupling — runs through the longest-lived Unix-philosophy fight of the last fifteen years.
Lennart Poettering launched systemd with “Rethinking PID 1” (April 2010) and answered the philosophy charge directly in “The Biggest Myths” (26 January 2013), where he noted that a fully-configured build produces 69 individual binaries (a 2013 figure) and argued that “the design of systemd as a suite of integrated tools that each have their individual purposes but when used together are more than just the sum of the parts, that’s pretty much at the core of UNIX philosophy.” On binary count he is not obviously wrong — GNU coreutils ships over a hundred programs, and nobody calls coreutils monolithic. Counting binaries settles nothing, and any argument that rests on it can be won by either side with a different tally.
The real axis is the connector. Classic Unix composition is late-bound, untyped, human-readable and inspectable: byte streams and exit codes, substitutable at any point, degrading gracefully when one stage doesn’t understand something. systemd’s connector is early-bound and typed: D-Bus interfaces, cgroup hierarchies, socket activation, shared unit state. That buys real things — typed method calls, supervision that actually knows what a service is, a fault channel with more than eight bits — and it costs real things: you cannot tee a D-Bus call and read it with your eyes, and you cannot substitute a component that speaks a slightly different dialect.
That is the text-streams-vs-object-pipelines trade, one layer down, with the same scorecard and the same winners on the same rows. Every “is X Unix-y?” argument reduces to connector coupling, and saying so is what ends the argument productively rather than escalating it. It generalizes past init systems: it is the same axis that separates a REST API from a shared database, a message bus from an in-process call, and MCP’s typed tool contracts from “just shell out and parse the output.”
A Decision Framework for Shipping a Tool
If you ship anything with an output — a CLI, a service, an agent, a build step — this history collapses into a few decisions with defensible answers.
Decide where the schema lives, from the boundary you actually have to cross.
| Boundary | Cheapest correct choice | Why |
|---|---|---|
| Inside one process | Native types | Prior agreement is free — you own both ends |
| Between your own services, one org, one release train | A typed contract (protobuf, typed RPC) | Coordination is affordable and versioning is enforceable |
| Between organizations | Self-describing text (JSON, and a documented grammar) | Neither side can force a runtime, a version, or an upgrade window on the other |
| Across decades, or to consumers you’ll never meet | Bytes with a stable, self-describing grammar | The only contract that survives the disappearance of both authors |
Then follow the four rules the history actually supports:
- Emit formats, not user interfaces.
diffcompounded intopatch, version control, code review, and eventually container images because Hunt and McIlroy’s 1976 algorithm paper shipped a machine-consumable changeset rather than a pretty report. The OCI image spec is that decision fifty years downstream — every layer is a tar changeset with deletions encoded as.wh.whiteout entries, merged by a union filesystem.catdid not compound, because vendors kept adding presentation options to it. The durable artifact of a tool is its output format; the UI is disposable. (That lineage is also why containers look the way they do.) - Make the machine-readable form the honest default, and the pretty form the adaptation. You will end up calling
isatty()— everyone does. Do it in the direction that keeps scripts stable: a documented, versioned structured mode that never changes shape, plus an explicit--json/--formatflag that always wins over detection, so no pipeline ever depends on a guess about a file descriptor. - Ship the fault protocol yourself. Exit codes and
stderrare not it. Decide, at design time, how a consumer distinguishes “no results” from “the query was malformed” from “the backend is down,” and put that distinction in the payload, because the composition model will not give it to you. - Put each general function in exactly one place. This is Pike and Kernighan’s real thesis, and it is about repetition, not size: pagination belongs in the pager, globbing belongs in the shell. The uncomfortable audit question is how many of your services implement their own retry policy, auth parsing, pagination, and log formatting — each duplicate is the Berkeley-
lsmistake in a modern costume.
The transferable lesson is not “text won” or “objects won.” It is that the two things being compared were never the same layer, and the argument only becomes tractable once you name which one you mean: the primitive (the pipe — universally adopted, never seriously challenged), the format (unstructured text — superseded by JSON, decisively), the tool size (falsified, and not worth arguing), or the connector (still genuinely contested, and the only one where reasonable architects disagree). Precision about which of those four you are defending is the entire value of the debate — and it is why jq won without anyone having to lose.
❓ Frequently Asked Questions
Did McIlroy actually say “write programs to handle text streams, because that is a universal interface”?
Not in the 1978 source it is attributed to. The foreword by McIlroy, Pinson and Tague to The Bell System Technical Journal 57(6) Part 2 sets out four numbered maxims, and the phrase “universal interface” appears in none of them. The three-line version is Peter Salus’s condensation in A Quarter Century of UNIX (1994, pp. 52–53), offered as a summary of McIlroy’s thinking. Quote whichever you like — just attribute the wording you actually use.
Is the object pipeline better than text streams?
Better at different things, and the comparison only works dimension by dimension. Object pipelines win on typing, discoverability, parsing cost, and error structure. Text streams win on the metric that decides most integrations: how much the two ends must agree in advance. Objects require a shared type system, runtime and version; bytes require a shared grammar. That is why object pipelines dominate inside a single runtime and byte streams dominate between organizations and across time.
What did jq actually change?
The format, not the plumbing. jq is an ordinary Unix filter — bytes in on stdin, bytes out on stdout — that happens to understand JSON’s structure. Its ubiquity is evidence that unstructured line-oriented output lost, and that the pipe itself never needed replacing. The modern settlement is byte stream as transport, JSON as schema, jq as the schema-aware filter, and the required prior agreement between producer and consumer stays near zero because the schema travels in band.
Why is isatty() format-switching a better argument than counting command-line flags?
Because it is a falsifiable prediction rather than an aesthetic judgment. In 1984 Pike and Kernighan wrote that adapting output format to the destination was “unique — no standard UNIX command has this property,” and by 2026 it is standard behavior in ls, Git, grep, Docker and jq itself. Flag counts drift by version and by counting method, so both sides can pick a number; the isatty() claim was a specific statement about the design space, and the design space overturned it.
Is systemd a violation of the Unix philosophy?
That question is unanswerable as posed, which is why it has run for fifteen years. On binary count Poettering’s defense holds — a fully-configured 2013 build produced 69 binaries against coreutils’ 100-plus programs. The substantive difference is the connector: byte streams and exit codes are late-bound, untyped and inspectable, while D-Bus, cgroups and shared unit state are early-bound and typed. That trade is real, it is the same one this article scores, and naming it converts an identity argument into an engineering one.
If I’m shipping a CLI today, what output format should I choose?
Structured text — JSON or JSON Lines — as a documented, versioned mode behind an explicit flag, with a human-readable rendering as the interactive default. Keep the machine format stable in shape, make the explicit flag override any terminal detection, and put error semantics in the payload rather than trusting an eight-bit exit status to carry them. That combination keeps the coordination cost near zero while giving consumers the structure that unstructured text never provided.
📚 Further Reading
- What Is a Container? The Idea That Powers Modern Software (and AI) — OCI layers are
diff‘s output format applied to filesystems, which is this article’s “emit formats, not interfaces” rule at industrial scale. - What is MCP? The Protocol That Lets AI Connect to Your Apps — a modern typed connector making exactly the early-bound trade this piece scores, in a domain that could have chosen “shell out and parse the output” instead.
- Saga Orchestration vs. Choreography: How to Choose — the centralize-the-connector decision, argued at the distributed-systems layer.
- Observing Distributed Workflows: How to Tell a Saga Is Healthy — what you have to build yourself once you adopt a composition model with no typed failure channel.
- Anti-Corruption Layers: Safely Integrating Untrusted Systems — where to put the parsing tax when you cannot dictate the format your inputs arrive in.
📚 Recommended Resources
Disclosure: some links below are Amazon affiliate links. If you buy through them, I earn a small commission — at no extra cost to you — which helps keep this site running.
🆓 The UNIX-HATERS Handbook
Simson Garfinkel, Daniel Weise & Steven Strassmann (eds.), 1994 · Book
The free PDF is the canonical edition — read it there first. Remembered as comedy, it is better read as the most sustained bug report ever filed against the cost side of the trade this article scores. Skip the punchlines and read “Error Messages and Error Checking, NOT!” and “Consistently Inconsistent” — both are structurally the same complaint, that an untyped connector produces untyped errors. Don Norman wrote the foreword; Dennis Ritchie contributed an anti-foreword. The print edition is a collector’s item more than a reference.
🆓 The Art of UNIX Programming
Eric S. Raymond, 2003 · Book
The full text is free at catb.org (the Harvard mirror is reliable; catb.org itself has a TLS certificate mismatch). §1.6 is where the seventeen rules quoted in this article come from — read it as the doctrinal endpoint of the inflation this piece traces, not as history, and note that several of the rules openly contradict each other. Opinionated, and parts have aged badly; assess it critically rather than treating it as scripture.
🆓 A Quarter Century of UNIX
Peter H. Salus, 1994 · Book
The actual source of “write programs to handle text streams, because that is a universal interface” — pp. 52–53. Worth reading precisely because so many people cite its condensation while believing they are citing the 1978 foreword. ⚠️ Out of print at the publisher; the Amazon listing is used-market only. Salus authorised the Unix Heritage Society to distribute a scan, so tuhs.org is the free and legitimate route.
The Unix Programming Environment
Brian W. Kernighan & Rob Pike, 1983 · Book
The composition model explained by two of the people who then went on to publish the sharpest internal critique of it. The specific tools are dated; the reasoning about how to make programs fit together — and about emitting formats rather than user interfaces — is not. (The hardcover is a separate listing, 0139376992.)
Advanced Programming in the UNIX Environment, 3rd ed.
W. Richard Stevens & Stephen A. Rago, 2013 · Book
Where to go when you want the pipe as a syscall rather than a slogan: PIPE_BUF atomicity, blocking semantics, SIGPIPE/EPIPE teardown, and the process plumbing that makes a shell pipeline work. The standard reference for the layer this article’s decision framework sits on top of. ⚠️ The 2nd edition is a different listing (0321525949) — get the 3rd.
Two free primary sources are worth more than any of the above for this particular argument, and neither has an Amazon listing: POSIX.1-2024 (Base Specifications Issue 8) is free to read in full and is the definitive answer to “what does the standard actually require,” and The Unix Heritage Society hosts the V1–V7 sources — including the Sixth Edition pipe.c quoted in this article — free under the 24 January 2002 Caldera licence.