View article

Logging has been a common practice for monitoring and diagnosing performance issues. However, logging comes at a cost, especially for large-scale online service systems. First, the overhead incurred by intensive logging is non-negligible. Second, it is costly to diagnose a performance issue if there are a tremendous amount of redundant logs. Therefore, we believe that it is important to limit the overhead incurred by logging, without sacrificing the logging effectiveness. In this paper we propose Log 2, a cost-aware logging mechanism. Given a “budget”(defined as the maximum volume of logs allowed to be output in a time interval), Log 2 makes the" whether to log" decision through a two-phase filtering mechanism. In the first phase, a large number of irrelevant logs are discarded efficiently. In the second phase, useful logs are cached and output while complying with logging budget. In this way, Log 2 keeps the useful logs and discards the less useful ones. We have implemented Log 2 and evaluated it on an open source system as well as a real-world online service system from Microsoft. The experimental results show that Log 2 can control logging overhead while preserving logging effectiveness.