content distribution, network storage
Content Distribution Networks (CDNs) represent an effective and scalable means of publication for content targeted at a wide audience. The benefits of CDNs include reduced load for origin server, reduced latency for end-users, and increased throughput. See [5] for an introductory discussion of content distribution networks.
CDNs are typically provided by large ISPs or third-party companies such as Akamai. While these services are useful for a large class of publications, they are of limited utility for transient content distribution needs. A significant amount of overhead may be involved in setting up content distribution via the traditional channels; e.g. billing arrangements, hardware installation, etc.
As an example of a transient content distribution need, consider the Linux archives at ibiblio.org [2]. When a new Linux distribution is released, the ibiblio servers can expect to see a large increase in traffic. A CDN can help mitigate the increased load and provide increased quality of service to end-users. However, ibiblio does not make use of a CDN because these events are infrequent and transient. It does not make sense to incur the cost and overhead of a commercial CDN provider for infrequent and bursty needs.
Similarly, it does not make sense for organizations to provide their own CDNs to support transient needs: What they need is a solution that provides a mechanism to quickly build short-term ad hoc CDNs. These ad hoc CDNs can be utilized for the duration needed and thrown away.
A suite of technologies enabling such a solution recently came into being through the efforts of the Logistical Computing and Internetworking Lab (LoCI Lab) at the University of Tennessee. At the core of these technologies is the Internet Backplane Protocol (IBP) [4]. IBP is a protocol and server suite that provides time-limited anonymous access to storage resources located in the network. By scattering IBP storage resources across the network and aggregating them into a logistical backbone, or L-Bone, we enable the creation of ad-hoc CDNs.
exDN is an experimental architecture built on top of IBP and related technologies that provides mechanisms for organizations to create short-term ad-hoc CDNs. By utilizing exDN, organizations can meet their transient content distribution needs with a minimum of cost and effort.
exDN makes a number of assumptions about the network; in particular, it assumes that a geographically distributed network of IBP storage depots exists. LoCI Lab has implemented such a network for research purposes, called the L-Bone [3]. Two exDN architectures have been proposed: server-side exDN, and client-side exDN. These architectures differ only in where interaction with IBP takes place and in the number of system components.
Each exDN architecture makes further assumptions about the network environment. In particular, server-side exDN assumes that an HTTP server with processing capabilities is co-located with each IBP storage resource. Client- side exDN assumes that clients capable of interacting directly with IBP storage depots are available.
With these assumptions satisfied, creating ad-hoc CDNs becomes a simple exercise; one need only upload content to the L-Bone and notify the exDN system that a new content channel is available. User requests can then be dynamically redirected to the best available storage resource.
Server-side exDN employs a standard client-server-agent architecture: User requests are initially directed to an agent; the agent couples the client with the server best able to handle the request, and then hands the request off to the server.
Server-side exDN clients are simple web browsers.
exDN servers consist of an IBP server coupled with a modified HTTP server. The exDN server acts as a translator for the client: it accepts standard HTTP requests and converts them to IBP requests. Similarly, the exDN server converts IBP replies to standard HTTP replies consumable by the client.
The exDN agent is the heart of the exDN system, and provides directory and resolution services. Initial client requests are sent to the agent, which chooses an appropriate server and redirects the client request.
Client-side exDN employs a client-server architecture built upon a network storage abstraction called the exNode [1]. In simple terms, an exNode is directly analogous to a Unix inode. An exNode contains pointers to network storage, as well as metadata describing the contents of that storage.
Clients in client-side exDN are web browsers running the exDN plugin. This plugin allows the client to locate and choose an exDN server, and to interpret the contents of an exNode.
Choices about which exDN server to retrieve content from are made based on a variety of performance metrics, including bandwidth, latency, and server load. Once the client has chosen a server, it issues a request for the content.
Client-side exDN employs two types of server. The first is a standard HTTP server and is the entry point into the exDN CDN. Client requests for content are marshaled at this server. Replies from this entry server contain exNodes with pointers to the requested object.
The second server in client-side exDN is a standard IBP server. Once a client has an exNode containing the content it wants, it chooses an IBP server and issues a request.Building an ad-hoc CDN with exDN is a simple task. exDN servers are located throughout the network, and are attached to the L-Bone. To build an ad-hoc CDN with exDN, one need only setup and configure an exDN agent or make exDN clients available. One then uploads content to the exDN content servers. At this point, the CDN is operational.