End-to-End Argument

Sometimes, simple arguments can lead to profound effects.

Take the case of the paper titled “End-to-End Arguments in System Design”[1], authored by J.H. Saltzer, D.P. Reed and D.D. Clark. The paper offers a guide for the placement of functions in communication systems in the form of a principle called the “end-to-end argument”.

The argument goes as follows:
“The function in question can completely and correctly be implemented only with the knowledge and help of the application standing at the end points of the communication system. Therefore, providing that questioned function as a feature of the communication system itself is not possible. (Sometimes an incomplete version of the function provided by the communication system may be useful as a performance enhancement.)”

It is as an argument proposing the implementation of functions on the application level as much as it is an argument for discouraging (or at least regulating) the placement of functions in the lower-levels of a system. The reason is quite blunt: place functions on where they will matter, not on where they will just waste resources.

To emphasize the concept being suggested by the argument and to demonstrate the wide range of situations to which this argument applies, they’ve provided several scenarios. I’ll only discuss one of them here, since all of them more or less provide the same idea.

File Transfer Scenario
File transfer is a fairly basic service provided by networks, but this is not necessarily protected from errors. Here are some possible threats enumerated by the authors (based on the setup of two host exchanging files through a network):
1. The file, though originally written correctly onto the disk at host A, if read now may contain incorrect data, perhaps because of hardware faults in the disk storage system.
2. The software of the file system, the file transfer program, or the data communication system might make a mistake in buffering and copying the data of the file, either at host A or host B.
3. The hardware processor or its local memory might have a transient error while doing the buffering and copying, either at host A or host B.
4. The communication system might drop or change the bits in a packet, or lose a packet or deliver a packet more than once.
Given these threats, provisions for ensuring reliable file transfers must be put in place. Following the end-to-end argument, error checking should be implemented in the file transfer application on both hosts, since it is only at that level can the success of the operation can be truly verified.
The communication system may also enforce reliable data transfer using a variety of error checking functions, but this will not guarantee that the file that will arrive at the receiving end is correct. Other forms of errors may occur outside of the communication system before it arrives at its destination disk storage system. As a result, the file transfer application must still perform error checking whether the data coming from the communication system is reliable or not. Not only is the use of functions in the lower levels of the system in this case futile, they are also an instrument for wasteful consumption of resources.

Performance
The authors, however, didn’t completely rule out the implementation of functions on the lower-levels of a communication system. In some cases, particularly in performance issues, this may have beneficial effects too. For example, using functions for ensuring reliability on the lower-levels of a network may increase its performance if the frequency of retrying file transfers outweighs the delay caused by the implementation of these functions. But the authors suggest that such low-level implementations be used only for performance enhancements.

Remark
I agree on the principle being raised by the end-to-end argument. It’s a matter of avoiding the redundant use of resources, and putting functions where they only truly count. Their examples drive home the point of the argument.
I have a minor comment on their argument, though. If they can assume that faults may occur on areas that are expected to be reliable (such as disk storage systems), wouldn’t it be fair to say that the same thing could apply to the applications on the end points of a system, such as the file transfer program? In fact, the second threat in the file transfer scenario highlights this matter. Now, if a file transfer application can commit mistakes in the simple process of buffering and copying data, then much more in the act of error checking (such as the verification of checksums). In short, their end-to-end argument wouldn’t hold if the integrity of end point applications is also compromised.

Conclusion
The authors have presented an argument, which they claim to be similar to “Occam’s razor” (though their scenarios seem to be inspired by Murphy’s Law), for making decisions on the placement of functions in a system. They themselves have admitted that this is not a strict rule to be followed, but a just mere guideline. If applied to a proper context, this simple guide can provide significant benefits for a network.

References
[1] J. H. Saltzer, D.P. Reed and D. D. Clark. End-to-End Arguments in System Design. ACM Transactions in Computer Systems, Vol. 2, No. 4, pp. 277-288. November 1984.