Session-based test management

Session-based Test ManagementChen nengji Original: Session-based test management-James Bach is a method used to measure and manage Exploratory tests. Brief IntroductionExploratory testing is an exploratory test that does not require scripts or drills. Its efficiency depends on several intangible things: the tester's skills, intuition, experience, and ability to track the hunch. However, these intangible things make our test Manager confused when interpreting the test results. For example, when the manager finally asks about the status of the exploratory tester, they will get the answer "ah, you know... I tested it here, and I tested it here. Let's take a look around ." Even if the tester recorded some bugs, the manager was not clear about how they found them. Even if the manager skillfully asks the tester what they did, the tester may have forgotten the details or cannot describe their ideas out loud. This problem also occurs when we perform exploratory testing for our customers. We need to explain our work. We need to provide a status report that reflects our actual work. We need to show that we are creative and skilled explorers, and produce a detailed map of Exploration Progress. How does it work?We have invented the session-based test management method to make these intangible things more practical. You can think of it as a structured Exploratory Test. Of course, it seems self-contradictory, but "structured" does not mean that the test is scripted. It refers to the expectation of what we want to do and how to report. In the recording studio, work is done in the "session. A session ranges from 45 minutes to several hours, but regardless of the length of time, it is the time spent in testing the session chapter. Jonathan Bach described the essence of sessions in an article written for stqe. At the end of a session, the tester submits a session report with important information about their work. Here is an example: Charter
-----------------------------------------------
Create a test coverage outline and risk list for decideright # Areas
Decideright
OS | Win98
Build | 1.2
Strategy | configuration & analysisstart
-----------------------------------------------
4/16/01 11: 15 pmtester
-----------------------------------------------
Jonathan Bach
Tim parkmantask breakdown
----------------------------------------------- # Duration
Short # test design and execution
100 # bug investigation and reporting
0 # Session Setup
0 # Charter vs. Opportunity
100/0 data files
-----------------------------------------------
Tco-jsb-010327-A.txt
Rl-jsb-010327-A.txtTEST notes
-----------------------------------------------
Tim and I got ed through the user guide table of contents and index to create the following TCO: Operating Systems: Win98
Win2000general features:

Installation
User Manual
Online Help
UI
Preferencesprominent windows: main table window
Criteria Weights window
Option ratings window
Documents window
Start-up windowmanagers and wizards: decideright Advisor
Category label Editor
Numeric Editor
Scenario Manager
Report Generator
Quickbuilddemo-elements: Language Elements
Preferences
Sensitivity indicators
Weighting
Input options
Demo-table
Options ratings
Baselineinteroperability: Ole
Import/Export
Graphs
Printingbugs
-----------------------------------------------
# N/aissues
-----------------------------------------------
# Issue
Manual mentions different platforms (Win 3.1, WFW, and WinNT 3.51) and does not mention win2000. We Think Win 2000 is important to test on and that the older OSes are no longer meaningful. # issue
We did this analysis on win98. I have no data to suggest that features may be different on other operating systems, but I'm not sure about that.SessionMeasurementSession measurement is the main method to express the state of the exploratory test process. It includes the following elements: 1. number of sessions completed 2. number of detected problems 3. number of functional areas covered 4. the percentage of session time spent preparing for the test is 5. used to test the percentage of session time spent 6. percentage of session time used to investigate the problemTask reportAt the end of each session, the tester discussed with the manager. We found that the value of sbtm (Session-based test management) depends on the discussion of the work between the test Manager and the tester, helping the tester and manager get the most from the meeting (about 15-20 minutes ), we use a checklist to discuss the problem.Scanning toolsOur scanning tool scans session reports by searching for tags. The scan results are accumulated, that is, all the information with the same mark in all session reports will be collected at each scan.Guidance to managersWe found that this method relies on the skills of the test Manager, so we will focus on a manager guide to discuss session protocols, the Guide also describes the benefits of using sbtm and the problems encountered during use. When this guide is ready, we will post it.IntroductionJonathan Bach first introduced sbtm (Session-based test management) in Star west 2000. His speech was named "How to measure ad hoc testing"