At the end of WOPR22, we asked the group for their takeaways. Here is the list that generated, with attribution when we could remember who said it.
1. “We need to think about Performance Testing (in terms) other than just load testing.” – Andy Hohenner
2. “Our first performance test for each build was a waterfall chart.” – Mais Tawfik Ashkar
3. As performance testers, we often have trouble getting enough time and space to review/verify/crosscheck our results enough to report them
4. Pairing performance testers with developers
5. Lack of understanding through reporting chain of Performance Data. This suggests we should provide information, and avoid providing data. At a previous WOPR, Mike Kelly said “Data plus Analysis equals Information”.
6. Consider the value of performance testability from conception, such as good logging and instrumentation
7. Configuration management w/virtualization
8. Let Devs access environment early on
9. Adding performance metrics to CI tests, including UI-based automation
This work is the product of all of the Workshop on Performance and Reliability (WOPR22) attendees. WOPR22 was held May 21-23, 2014, in Malmö, Sweden, on the topic of “Early Performance Testing”. Participants in the workshop included Fredrik Fristedt, Andy Hohenner, Paul Holland, Martin Hynie, Emil Johansson, Maria Kedemo, John Meza, Eric Proegler, Bob Sklar, Paul Stapleton, Andy Still, Neil Taitt, and Mais Tawfik Ashkar.