User Tools

Site Tools


features:pipes

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
Next revisionBoth sides next revision
features:pipes [2015/12/29 20:15] adminfeatures:pipes [2015/12/29 20:30] admin
Line 1: Line 1:
 ====== Pipes ====== ====== Pipes ======
  
-The idea of connecting an output from one process, via an in-kernel memory buffer, to the input of another process is entirely original to Unix.+**Wikipedia: ** [[https://en.wikipedia.org/wiki/Pipeline_(Unix)|Pipeline (Unix)]]
  
-Doug McIlroy, in 1964had proposed that "We should have some ways of connecting programs like garden hose–screw in another segment when it becomes necessary to massage data in another wayThis is the way of IO also." (Dennis Ritchie, [[https://www.bell-labs.com/usr/dmr/www/mdmpipe.html|Advice from Doug McIlroy]])+The idea of connecting an output from one processvia an in-kernel memory bufferto the input of another process is entirely original to Unix.
  
-===== Evolution of Pipes =====+Doug McIlroy, in 1964, had proposed that "We should have some ways of connecting programs like garden hose – screw in another segment when it becomes necessary to massage data in another way. This is the way of IO also." (Dennis Ritchie, [[https://www.bell-labs.com/usr/dmr/www/mdmpipe.html|Advice from Doug McIlroy]]). Implicit in this statement is the idea of //filters// to modify and "massage" the data. 
 +===== The Evolution of Pipes =====
  
 (taken from Dennis Ritchie, [[https://www.bell-labs.com/usr/dmr/www/hist.html|The Evolution of the Unix Time-sharing System]]) (taken from Dennis Ritchie, [[https://www.bell-labs.com/usr/dmr/www/hist.html|The Evolution of the Unix Time-sharing System]])
Line 13: Line 14:
 Pipes appeared in Unix in 1972, well after the PDP-11 version of the system was in operation, at the suggestion (or perhaps insistence) of M. D. McIlroy, a long-time advocate of the non-hierarchical control flow that characterizes coroutines. Some years before pipes were implemented, he suggested that commands should be thought of as binary operators, whose left and right operand specified the input and output files. Thus a `copy' utility would be commanded by Pipes appeared in Unix in 1972, well after the PDP-11 version of the system was in operation, at the suggestion (or perhaps insistence) of M. D. McIlroy, a long-time advocate of the non-hierarchical control flow that characterizes coroutines. Some years before pipes were implemented, he suggested that commands should be thought of as binary operators, whose left and right operand specified the input and output files. Thus a `copy' utility would be commanded by
  
-<code>inputfile copy outputfile +<code> 
- +inputfile copy outputfile 
-</file>+</code>
  
 To make a pipeline, command operators could be stacked up. Thus, to sort //input,// paginate it neatly, and print the result off-line, one would write To make a pipeline, command operators could be stacked up. Thus, to sort //input,// paginate it neatly, and print the result off-line, one would write
- +<code> 
-<code>input sort paginate offprint +input sort paginate offprint
 </code> </code>
  
 In today's system, this would correspond to In today's system, this would correspond to
  
-<code>sort input | pr | opr +<code> 
 +sort input | pr | opr
 </code> </code>
  
Line 33: Line 33:
 Some time later, thanks to McIlroy's persistence, pipes were finally installed in the operating system (a relatively simple job), and a new notation was introduced. It used the same characters as for I/O redirection. For example, the pipeline above might have been written Some time later, thanks to McIlroy's persistence, pipes were finally installed in the operating system (a relatively simple job), and a new notation was introduced. It used the same characters as for I/O redirection. For example, the pipeline above might have been written
  
-<code>sort input >pr>opr> +<code> 
 +sort input>pr>opr>
 </code> </code>
  
Line 42: Line 42:
  
 Soon some problems with the notation became evident. Most annoying was a silly lexical problem: the string after `>' was delimited by blanks, so, to give a parameter to //pr// in the example, one had to quote: Soon some problems with the notation became evident. Most annoying was a silly lexical problem: the string after `>' was delimited by blanks, so, to give a parameter to //pr// in the example, one had to quote:
- +<code> 
-<code>sort input >"pr -2">opr> +sort input>"pr -2">opr>
 </code> </code>
  
 Second, in attempt to give generality, the pipe notation accepted `<' as an input redirection in a way corresponding to `>'; this meant that the notation was not unique. One could also write, for example, Second, in attempt to give generality, the pipe notation accepted `<' as an input redirection in a way corresponding to `>'; this meant that the notation was not unique. One could also write, for example,
  
-<code>opr <pr<"sort input"< +<code> 
 +opr <pr<"sort input"<
 </code> </code>
  
 or even or even
  
-<code>pr <"sort input"< >opr> +<code> 
 +pr <"sort input"<>opr>
 </code> </code>
  
Line 62: Line 61:
  
 I mentioned above in the section on IO redirection that Multics provided a mechanism by which IO streams could be directed through processing modules on the way to (or from) the device or file serving as source or sink. Thus it might seem that stream-splicing in Multics was the direct precursor of Unix pipes, as Multics IO redirection certainly was for its Unix version. In fact I do not think this is true, or is true only in a weak sense. Not only were coroutines well-known already, but their embodiment as Multics spliceable IO modules required that the modules be specially coded in such a way that they could be used for no other purpose. The genius of the Unix pipeline is precisely that it is constructed from the very same commands used constantly in simplex fashion. The mental leap needed to see this possibility and to invent the notation is large indeed. I mentioned above in the section on IO redirection that Multics provided a mechanism by which IO streams could be directed through processing modules on the way to (or from) the device or file serving as source or sink. Thus it might seem that stream-splicing in Multics was the direct precursor of Unix pipes, as Multics IO redirection certainly was for its Unix version. In fact I do not think this is true, or is true only in a weak sense. Not only were coroutines well-known already, but their embodiment as Multics spliceable IO modules required that the modules be specially coded in such a way that they could be used for no other purpose. The genius of the Unix pipeline is precisely that it is constructed from the very same commands used constantly in simplex fashion. The mental leap needed to see this possibility and to invent the notation is large indeed.
 +
 +===== Dating the Creation of Pipes =====
 +
 +According to the [[:anecdotes:pipes|anecdotes]], Ken Thompson added pipes to Unix in one day.
 +
 +The Second Edition of Unix, dated June 1972, didn't have pipes. By January 15, 1973, Unix did have pipes: Doug McIlroy put out the notice for a [[http://www.tuhs.org/Archive/Documentation/Papers/Unix_Users_Talk_Notes_Jan73.pdf|talk which described the state of UNIX]] at that time. Page 4 describes SYS PIPE and its implementation. This is as narrow as we can get on the date when Ken added pipes to the system.
  
features/pipes.txt · Last modified: 2022/09/16 18:47 by 123.219.209.121