Lazy Preservation: Reconstructing Websites from the Web Infrastructure

Backup or preservation of websites is often not considered until after a catastrophic event has occurred. In the face of complete website loss, webmasters or concerned third parties have attempted to recover some of their websites from the Internet Archive. Still others have sought to retrieve missing resources from the caches of commercial search engines. Inspired by these post hoc reconstruction attempts, this book introduces the concept of lazy preservation- digital preservation performed as a result of the normal operations of the Web Infrastructure (web archives, search engines and caches). In this book, the Web Infrastructure (WI) is characterized by its preservation capacity and behavior. Methods for reconstructing websites from the WI are investigated, and a new type of crawler is introduced: the web-repository crawler. Several experiments are used to measure and evaluate the effectiveness of lazy preservation for a variety of websites, including dynamic websites where recovery of the server components is necessary. This book is intended for those interested in preserving the Web and using search engines to that effect.