Streams

One of the most interesting concepts in computer science to me is the concept of a stream. They are such useful abstractions that they appear in many different contexts

• In Unix, streams are used as an I/O abstraction. Everyone knows that stdin and stdout are streams, but even files are managed using the STREAM datatype when you use formatted I/O functions like fprintf()
  • Actually, in addition to the STREAM datatype, I/O communication in general, including both files and sockets, are thought of as reading and writing to a stream. Most interprocess communication happens through a construct that can be thought of as a stream, either unidirectional or bidirectional.
• A stream is the core metaphor behind TCP all communication, whether mediated by Unix sockets or some other abstraction.
• In Scheme, streams are just lazy lists. This is useful for writing functions which manipulate large (or infinite) lists. You can get the result of a computation on part of your list without doing any more work than necessary.
• The data in a log-based message broker like Kafka is often thought of as a stream (and computation on it is referred to as stream processing).

In fact, you might think of the concept of a stream to as connected with a broader category of programming abstractions like queues, messages and event-based or asynchronous programming models, insofar as they are all used to enable different kinds of incremental computation.