WOPR1 was held in New York City on October 2-4, 2003.
Attendees
James Bach, Scott Barber, Ross Collard, Linda Hamm, Doug Hoffman, Paul Holland, Dave Jewell, Philip Joung, Jude McQuaid, Alex Podelko, Rob Sabourin, Andrew Sliwkowski, Roland Stens
WOPR ANNOUNCEMENT
Friday, July 25, 2003
From Scott Barber, Ross Collard and Dave Jewell
A. WOPR Themes
The common topic of inquiry for the first two WOPR meetings is: Forming Trustworthy Conclusions.
Performance and reliability testing is a predictive art, and there is usually a “moment of truth” where expectations and predictions meet reality. We will divide this topic into two main themes:
a) Realism in Testing, the theme for WOPR1 in New York in 2003 (details below), and
b) Data Analysis and Interpretation, for WOPR2 in Silicon Valley in 2004 (see www.performance-workshop.org for details).
B. Writing Your Paper
If you would like to give a presentation at WOPR, we want you to write a paper on the topic of your presentation and submit it for circulation to the other participants prior to WOPR. Because of time limitations in the three-day meeting, we cannot guarantee that everybody will be able to present his or her paper, but all papers will be published and circulated within the group.
Although writing a paper is not a mandatory prerequisite for attending WOPR, we strongly encourage everyone to write a paper. Presentations of the papers will be informal, and while we will have a scribe in the meeting we deliberately will not provide an overhead projector.
In your paper and presentation, please share one experience you have had in which you learned a valuable lesson about forming trustworthy conclusions. While your paper does not have to be written in the first person, it should describe your personal experience and your team’s experience on a fairly recent project. We want a report along the lines of: “I was there and here is what I saw and did”. Assume an intermediate-to-advanced level of performance testing experience in the audience, but do not assume that others know your particular environment (e.g., Linux) or tools (e.g., OpenSTA).
We anticipate most papers will be in the range of four to ten pages, though there are no upper and lower limits. Attach graphs and spreadsheets of test data as appropriate, test network topology diagrams, etc. We prefer narrative text but will accept bullet point outlines. Do not include anything which requires a non-disclosure agreement (NDA) or a security clearance.
We have set an aggressive schedule – your paper for WOPR1 is due on August 22, so a prompt start is advisable.
C. Theme: Realism in Testing
The primary issues we would like you to address in your paper and presentation for WOPR1 in October are:
How you developed the usage model(s) to be tested.
How you determined whether the usage model(s) reflected actual usage adequately.
How you visualized, calculated and/or documented the usage model.
How you implemented the usage models using tools, with the trade-offs and compromises you made, if any.
Other related and important issues that you may elect to address, but are not the primary focus of this first WOPR, are:
How you assessed whether you had meaningful and testable performance goals.
How you scaled up or adjusted for differences between the test environment and the live operational environment, to predict performance in production.
D. Qualities of a Good Experience Report (Paper and Presentation).
“What is a good experience report?” Basically, a good experience report has these attributes:
You feel strongly about it.
It taught you a valuable lesson (positive or negative).
It fits the theme of the meeting.
You can describe many relevant aspects of the experience, such as:
– technologies involved
– business environment
– people and their attributes
– equipment and tools
– details of the product or system
– documents
– sequences of events
– relationship to the rest of the project
– decisions made, why and when;
so that …
– other people can compare it meaningfully with their experience
– other people can evaluate your experience
– other people receive enough information to suggest alternative choices you could have made in that situation.