Writing the A2 Project

AQA Specification pages

4. Test Strategy and Testing (8 marks)

From the specification:

Expected contents for this section of the report are:

A test plan that includes:

• details of individual tests using a minimal set of test data
• expected results for clearly defined typical data
• expected results for clearly defined erroneous data
• expected results for clearly defined boundary (extreme) data
• samples of annotated hard copy of actual test runs for typical, erroneous and boundary data, and samples of annotated hard copy showing the system working (system testing)
• all samples cross-referenced to the test plan

A testing strategy is the way that you will test your program. Strategy implies that you will think about the way that you test and will write a sound commentary on the method you have adopted.

Remember that in Analysis you shaped your work to match the objectives of SMART:

• Specific: to the problem being solved and should specify what is to be achieved
• Measurable: you need to be able to tell if you are meeting your objectives
• Achievable: are your objectives achievable (not much use of they are not)
• Realistic: can the objectives be achieved with the resources that you have?
• Time: completed within the specified framework

Your testing will show that you have met these objectives. You will:

• Refer back to the entry in Design where you defined your testing strategy;
• Follow this strategy closely and refer back to it as you perform your tests (cross-reference them);
• Follow the method of testing that you decided on in Design - unit, black box, white box, integration, functionality, usability, system;
• Run the data through your program;
• Show whether the outcomes are as expected or not: provide details through screen shots and/or print-outs and collect these in an appendix with suitable headings for cross-reference purposes;
• Work out the expected outcome according to your algorithms and what transformations of the data they should produce; this could mean doing dry runs and then program runs to compare results;
• Investigate and report on the way data move through and are transformed by critical parts of your program e.g. a key function or procedure (call this unit testing, black box testing, white box testing according to what you are testing and the level of detail you use);
• Drill down to the most basic operations of your code and the ways that it manipulates or transforms the data;
• Comment on the success of your tests; you should have demonstrated beyond any doubt that your program performs as required; report any discrepancies; fix any problems if you have time, depending on their size and scope;
• Test those parts of your program that are not connected to functions (non-functional testing) e.g. security;
• Your program should be working by now; your tests prove this one way or the other;
• Testing should demonstrate that your program works in the way you intended and meets the user's needs. The user should be confident that the tests will make the program acceptable for the purpose agreed at the start. Discuss this in your commentary.

One aim of testing is to discover as yet undiscovered errors and to show that the software works as intended. You should have identified your testing strategy during the Design phase of the project. By thinking about testing in this way you will focus on the things that will make your system work successfully. If you do find errors you may not have time to fix them: minor errors should have been removed during implementation and large errors may be beyond your ability to correct them.

The test plan should include samples of tests done, including the purpose of the test, the data used and the expected results. Samples of hard copy, annotated and cross-referenced, should prove the correct working of the system. Test results will normally be in the form of file dumps, screen shots and test runs. If using screenshots to illustrate messages include the screen behind, not just the message boxes (you could fake message boxes!).

The testing should test the system, not just menus or buttons. The test data should be designed to test the limits of the system, not just operation with normal data. Test the most complex and vital aspects of the system rather than showing repeated validation tests on input data. Each test should be numbered, should specify what is to be tested, should provide an explanation of why the test is being performed, should specify the data to be used and give the expected and actual results.

The full set of test results should be put into an appendix in the back of the project. Test results should be cross-referenced to the test plan.

Use peers to help you with your testing. When you have completed your own testing procedures you should arrange for your user to test the program. Devise some appropriate data to demonstrate the code with your user (you may be able to use existing data if you remove it from the system).

Useful video. There are a number of other videos available on Youtube - search for 'testing software'.

Wikipedia