What statement do you consider to be most important and why?
a) Testing has the primary intent of showing the system meets the users needs.
b) Testing has the primary intent of finding faults
You have run all your tests and they all pass. Is this good news or bad news?
It depends on how good your tests were and what they were testing. To have justified confidence in the software we must have confidence in our tests, data and environment.
What would you do if you were asked to test a system which is unfamiliar to you has out-of-date or inadequate documentation?
Talk to users, developers and analysts to understand what the system is supposed to do. Document this understanding and get it reviewed and use this as a substitute for the Requirements/Design documentation. Talk with testers who have tested the system previously Read whatever is available and clarify assumptions
In running a test you find the actual result does not match the expected result? What would you do?
The tester should first establish whether the reason is because of a test fault (i.e. they have made a mistake) or whether it is an environment fault. If neither of these are true then they should then check to see whether this fault has already been raised. If not then either raise the fault or more preferable ï¿½ talk to the development group to check the fault out.
Do you consider positive or negative testing to be most important or trying to break the system - and why?
They are as important as each other. However testers need to have a different mindset to developers and therefore should actively look for potential faults. If we only concentrate on positive tests (show that the system does what it should do) then we will potentially experience problems when the system goes live. If we only concentrate on negative tests (showing the system doesnï¿½t do what it shouldnï¿½t) then again we could potentially miss significant faults. However if we look primarily at breaking the system then we may find lots of faults (the what if scenarios) but we may not establish if the system is going to meet the users needs and requirements. A balance is needed with all three approaches.
How would you define a good test?
A good test is one that can potentially find a fault in the system. If this test does not find a fault then it will give us a certain amount of confidence. Tests must also be efficient ï¿½ we should not have tests which all do the same thing.
In testing the above application you identify what you believe to be a fault?
In testing the above application you identify what you believe to be a fault ï¿½ instead of printing the message concerning the type of triangle in a separate dialog box the application is printing the message in the space between the 3 text fields and the OK button. What should your next step be (answer and state why)?
a) Continue testing to the end of the script, and then report the bug.
b) Stop testing, report the bug immediately, then continue alternative scripts
c) Stop testing, report the bug and await a fix.
d) Continue testing and report the bug later, along with those found in other scripts
You have raised a fault, but Development are unable to reproduce it. What should your next step be?
You have raised a fault, but Development are unable to reproduce it. What should your next step be? (Give answer and state why)
a) Let development sign off the bug as not reproducible.br> b) Sign off the bug yourself as not reproducible.br> c) Tell development the bug definitely exists and you will not pass it unless fixed. br> d) Re-test and upon confirmation provide more detailed information to Development, talking them through each stage if necessary.
The answer is (d) ï¿½ it might be our environment or it could have been fixed by some other fault fix in the new version.
You have two sets of tests to run on the new version of the software?
You have two sets of tests to run on the new version of the software.
Test Set 1: a test set to provide confidence that software has not regressed from the previous version.
Test Set 2: a detailed test set to investigate potential faults in the new release of software.
Having run test set 1 you discover a number of faults in the new version of software ï¿½ what do you do?
First we should investigate the faults ï¿½ is it because we had run our tests wrongly, or that we were running the tests on the wrong environment? Assuming that it is because the software has regressed ï¿½ then we must establish the nature of the faults and severity of the faults. It is probably inefficient to run any further tests at this stage. We should work with development in getting a new version of the software with the faults fixed and re-tested before running test set 2.
Describe the stages of testing and what the objectives are at each stage?
Lowest level of testing, detail, finding faults, performed by the developers
Combining components, testing interfaces, performed by developers, various types of integration (top-down, functional, bottom up and big bang). Business scenarios and non-functional aspects if possible.
System Testing (functional and non-function)
Testing the system as a whole. Testing requirements and business processes. Also testing non-functional aspects such as Performance, usability etc.
Testing the system with other systems and networks
Acceptance Testing Testing by users/customers to gain confidence that the system is going to support the business as well as meet their requirements.
Explain what you understand by the terms: Regression Testing and Re-Testing?
Running tests to ensure that the software has not regressed in anyway as a result of changes to the software and/or environment. Regression testing is running passed tests again to ensure that they still pass. Re-Testing
This is running a test again that had found a fault to check that the fault has been fixed correctly. Re-testing is running a failed test again to ensure that it now passes.
You have planned to run 600 tests on your own .... What would you do?
Scenario: You have planned to run 600 tests on your own. Each test will take approximately 10 minutes to run. Your manager has told you that you must complete these tests within one week. What would you do?
Assuming there are 7hours per working day. This task would take you: 600x10 = 6000 minutes = 100 hours = 14.286 days
There are a number of options that could be considered:
.. Work overtime (this should not be considered as a first resort)
.. Ask for more staff to help (again this may not be the best approach, particularly if you need to spend time training and mentoring the new staff)
.. Re-prioritise our tests and run the most important tests first
.. Assuming that not all the 600 tests would have been run within this time, risk assessment need to be made as to the consequences of not running the extra tests.
.. After this initial week and the system is implemented there is no reason why the extra tests could not be run (assuming that you are given the time)
Do you consider testing tools to be valuable during the testing process ï¿½ why/why not?
Testing tools are very important to assist the tester in their work. Using tools can also potentially make the tester more efficient in their work ï¿½ they are able to run more tests (using regression testing for example). Or they can quickly compare 3 reports (comparison tool).
The tools in themselves however do not make good testers and also should not be considered if the test process is in ï¿½chaosï¿½.
List 3 test tool categories and describe what each can do?
.. Test Design Tools
.. Test Data Preparation Tools
.. Regression Testing tools
.. Debug Tools
.. Dynamic Analysis Tools
.. Coverage Measurement Tools
.. Static Analysis Tools
.. Performance Testing Tools
.. Test Management Tools
.. Network monitoring tools
.. Test Harness or Simulation tools
The importance of this question is to see if the candidate has any knowledge about tools. We do not want the names of tools but want to know if the candidate can distinguish between the types of tool.
Name 2 standards that refer to testing?
Any of the following:
BS 7925-1 (Glossary of testing terms), BS7925-2 (Component Testing), ISO9000 and ISO9001 (Quality standards), IEEE829 (Test Documentation), IEEE1028 (Reviews), IEEE1044 (Incidents)
How would you test these requirements?
18. How would you test these requirements:
a) The system must be user-friendly
b) The system must be easy to install
c) The following response times are to be achieved with the new system:
ï¿½ Initial loading of the web application must be achieved within 3 seconds
ï¿½ Updating of the information on the web page must be no more than 5 seconds
How would you approach these requirements:
a) The system must be user-friendly
What do we mean by ï¿½user-friendlyï¿½? Questions to ask:
.. Friendly to whom?
.. Who are the users?
.. Talk to the users
.. Document assumptions
.. Compile test scenarios for people who have not seen the system
b) The system must be easy to install
What do we mean by ï¿½easy? Questions to ask:
.. For whom? ..
Is there any installation documentation to follow?
Why do you consider testing to be necessary?
.. There are faults in the software
.. Failures in live operation can be expensive
.. Sometime a ï¿½legalï¿½ or contractual requirement
.. To asses the quality of the software
.. To preserve the quality of the software
.. To help achieve quality software (by finding and removing the faults)
Write a set of test cases to adequately test this telephone system?
A hotel telephone system can perform 3 functions:
ï¿½ Call another hotel room by entering a room number (201 to 500)
ï¿½ Call an external line by entering a 9, followed by the number
ï¿½ Call various hotel services
ï¿½ 0 = Operator
ï¿½ 7 = Room Service
ï¿½ 8 = Reception
Write a set of test cases to adequately test this telephone system
Input Expected Result
201 Room 201 (valid boundary)
405 Room 405 (valid partition)
500 Room 500 (valid boundary)
7 Room Service
9 Outside line
Input Expected Result
200 Error (invalid boundary)
501 Error (invalid boundary)
550 Error (invalid partition)
Any other button Error
What if I accidentally hit multiple buttons?
What if I entered an 0800 number without first
getting an outside line (operator should answer)
Describe what you understand about the term ï¿½Static Testingï¿½ and list 3 static testing techniques?
Static Testing is non-execution of the code. Techniques include; reviews, inspections, walk throughs, individual techniques such as desk checking, data-stepping and proofreading. There is also static analysis (data flow and control flow analysis)
How would you prioritise your tests (list 5)?
ï¿½ ask the customer to prioritise the requirements
ï¿½ ask the customer to prioritise the tests
ï¿½ what is most critical to the customerï¿½s business
ï¿½ test where a failure would be most severe
ï¿½ test where failures would be most visible
ï¿½ test where failures are most likely
ï¿½ areas changed most often
ï¿½ areas with most problems in the past
ï¿½ most complex areas, or technically critical
Choose which program you would re-test and state you reasons?
You are testing 2 programs and have 3 weeks to test them both. Having run all of your tests on both programs you finish testing within 2 weeks. You need to decide which of the 2 programs you would re-visit and run further tests against. Choose which program you would re-test (can choose only one!) ï¿½ and state you reasons:
A Complexity Level: 2
Lines of Code: 2000
Number of tests: 100
Number of bugs found: 10
1 high severity, 3 medium & 6 low)
B Complexity Level: 2
Lines of Code: 2000
Number of tests: 100
Number of bugs found: 50 (10 high severity, 25 medium & 15 low)
1. Different programmers wrote A and B
2. Complexity level of the programs are the same
3. Size of the programs are the same
4. Tester is the same for testing A and B
5. Number of tests run on both programs is the same
6. Number of bugs is higher in program B
Program B seems to have far more faults therefore we would be inclined to spend the further week testing Program B, as there is likely to be more bugs to find. We may also not be very confident at this point with Program B therefore we need to see our confidence increased.
An ATM has been specified to work in the following way?
An ATM has been specified to work in the following way:
Enter a card and if the card is invalid reject the card and exit system. If it is a valid card then enter a PIN number. Check to see if the PIN is invalid
ï¿½ if it is then display a message ï¿½invalid pin number, please re-enterï¿½. If 3 attempts are made with an invalid pin then the machine keeps the card. If it is a valid PIN then the user can select one of the following transactions:
ï¿½ Cash Withdrawal without receipt
ï¿½ Cash Withdrawal with receipt
ï¿½ Balance Inquiry
ï¿½ Statement request
What tests would you produce to test this application? State any assumptions when testing
1. Invalid Card ï¿½ reject card and exit
2. Valid Card and Invalid PIN ï¿½ error message ï¿½invalid pinï¿½ï¿½ (then enter valid pin)
3. Valid Card and Invalid PIN ï¿½ error message ï¿½invalid pinï¿½ï¿½ (then enter another 2 invalid Pins)
4. Valid Card, Valid Pin & Cancel (correct length pin)
5. Valid Card, Valid Pin in a large number ï¿½ but the pin number contains more than the maximum number ï¿½ should error
6. Valid Card, Valid Pin & Cash Withdraw without receipt
7. Valid Card, Valid Pin & Cash Withdraw with receipt
8. Valid Card, Valid Pin & Balance inquiry
9. Valid Card, Valid Pin & Statement Request
.. No date on log as to when raised
.. No keywords (i.e. screen) so that searches can be performed preventing duplication of fault logs
.. No status of the log (opened/fixed/closed/cleared etc.)
.. No owner of the log. .. Has priority ï¿½ but no severity (i.e. risk to the customer)
.. No version number of the system being tested ï¿½ it is very likely that the testers are on a different version to development and that it was a fault but has been inadvertently fixed on this latest software
.. Query the priority of this log (should it be a 3?)
.. No actual error message on the log ï¿½ this may give some clue to the developer about the nature of the fault
.. Response seems to be leading to a dialogue ï¿½ if we are not careful this fault will never be fixed! Tester should talk to the developer rather than sending another message via the fault log.
.. The response by the developer points to another part of the system (security) ï¿½ this may be an indication of developers trying to quickly close the issue without performing sufficient investigation. It could however be because the tester has not spent enough time documenting the problem.