In-Lab Replay Testing under Real Traffic with a Case Study on WLAN Routers

Abstract

Replaying artificial or real-world traffic is a method used to test networking devices. Using real-world traffic is desirable as it uncovers more realistic properties such as traffic diversity and complicated user behaviors. In this paper, we propose an in-lab replay testing (ILRT) framework, composed of a monitor, a traffic replayer and a library of real-world traffic traces, to replay captured packet traces to test networking devices. To demonstrate its effectiveness, a total of 28 WLAN routers were tested to evaluate whether they could work stably for an extended time. In our experiments, 53.57% and 100% of devices under test (DUTs) were triggered critical (L1) failures and tolerable (L2) failures, respectively. 21.43% and 100% of DUTs were triggered more than one L1 and L2 failures, respectively. Among high-spec DUTs, there was 25% of DUTs which were triggered L1 failures. Furthermore, among 458 test results, chance of pass, L1 failure, and L2 failure is 2.84%, 7.86%, and 89.3%, respectively. The results showed that even though these devices have passed traditional lab tests, there is still a good chance for them to fail under real-world traffic.