home

TCP for File transfer from ssd to ssd.

http://sne.science.uva.nl/ http://uva.nl/
Introduction
Links
Related Projects
Related publications

Introduction

    Many Internet protocols were designed 30 years ago. The capacities, properties and prices of different components constituting the Internet were vastly different from now. Memory was scarce, is now abundant and cheap. Reading data from peripheral devices was an expensive operation. With the advent of solid state disks (re)reading parts of a file has become a rather cheap and fast operation. Speeds of the links have skyrocketed. Ongoing trials experiment with 100 Gb/s network interface cards. The energy footprint of ICT and the Internet are increasingly becoming under pressure. This spawns new infrastructure models as virtualisation and cloud computing that promise more efficient use of resources that are located at optimal places where environmental and cost parameters are optimal. This development, however, also necessitates the transport of massive amounts of data and computing from the place of the user to/from the place of the cloud centers.
     Internet protocols designed 30 years ago, such as TCP, are still widely used and it is amazing that those still work. However, more and more it becomes clear that in networks with 100 Gb/s speed and 100 ms RTT packet loss and other congestion avoidance protocols can cause a very inefficient behavior of the transport. The current TCP protocol keeps a buffer in memory on both the sending and receiving side to allow retransmission for lost data. The protocol makes sure that when received data is successfully processed  on the receiving side this data is also removed from the buffer on the sending side and makes room for new data. This so called sliding window protocol becomes very inefficient in extreme situations with high speed and round trip time. Lost packets cause the retransmission of data but also block the sending side from removing old data and filling its buffer with new data. Due to the round trip time it will take a while before the sender is notified from success or failure. One of the reasons for this buffering system is the inability or high cost for the sending side protocol to re-read the data that got lost. Obviously in interactive applications (telnet alike) asking the user to type the last line again is a no-go. In case of file transfer one could propose to re-read data from the source media (disk or tape). That was considered an expensive operation as the device needs to mechanically reposition itself what takes considerable time compared with the speeds in the network. However, solid state disks do not have moving parts and read operations have no seek time.

Proposed solutions

     Given the low cost of (re-)reading data from ssd's make a file transfer protocol that reads data, that is lost in the network, again from the source media. In essence this places the sliding window in TCP completely on the file. If we also assume or know that we are operating inside a private lightpath or vlan environment with dedicated capacity one could also study network behavior where the congestion avoidance is completely absent. The senders always transmit at maximum rate and don't care when packets get dropped in the network under the assumption that retransmission is a low cost operation.

Links

Related projects

Related publications




home home