Once the unit tests are completed, the command-line tools must be exercised to ensure their proper function. Begin with the ability to create a repository, then register templates, then submit reports, edit reports, and query reports.
This test case addresses CREATE-REPOSITORY.FR3, CREATE-REPOSITORY.FR4, CREATE-REPOSITORY.FR5, CREATE-REPOSITORY.FR6, CREATE-REPOSITORY.FR7, CREATE-REPOSITORY.FR8
Complete all automatic unit testing
Table 2-1. Test script: CREATE-REPOSITORY.TC
| User actions | System Response/User examination |
| Enter a new directory without a repository. Execute the yaktrack tool with the command line create-repository | System should respond with success, listing the contents of the new repository. Inspect the listing for a recursive display of the new repository's contents. Examine the source default repository and the new repository to ensure that all contents are copied correctly. Examine the output for existence of CM identification. Examine new repository for evidence that the new repository has been placed under CM. |
This test case addresses DEFINE.IS-VALID-TEMPLATE.FR2, DEFINE.IS-VALID-TEMPLATE.FR6A, DEFINE.REGISTER-TEMPLATE.FR1, DEFINE.REGISTER-TEMPLATE.FR2, DEFINE.REGISTER-TEMPLATE.FR3, DEFINE.REGISTER-TEMPLATE.FR4
CREATE-REPOSITORY.TC must be tested
Table 2-2. Test script: REGISTER-TEMPLATE.TC
| User actions | System Response/User examination |
| Enter a directory without a repository and create one via the create-repository command | System should create a repository |
| Copy the bad_template file from the test_data directory of the distribution into the current directory | |
| use the is-valid-template command to test bad_template for validity | System should output the following errors: Invalid subelement originator/email, construct started but never completed, root element does not contain an #IMPLIED text attribute named "yaktrackid" |
| use the register-template command to attempt to register bad_template into the repository | System should respond with a listing of invalidators and the note "no action taken" |
| Copy the good_template file from the test_data directory of the distribution into the current directory. Ensure that the newly-created repository does not already have a template named "defect" | |
| Use the register-template command to register the file good_template | System should respond with "template defect was stored successfully" |
| Repeat the previous entry (try and resubmit good_template) | System should respond that "defect template already exists" and "no action taken" |
This test case addresses SUBMIT.FR1, SUBMIT.FR2, SUBMIT.FR3, SUBMIT.FR4, SUBMIT.FR5, SUBMIT.FR6, SUBMIT.FR7, SUBMIT.FR8
DEFINE.REGISTER-TEMPLATE.TC and its pretests
Table 2-3. Test script: SUBMIT.EDITOR.TC
| User actions | System Response/User examination |
| Create a repository in a temporary directory. Register the good_template template from the test_data directory so that "defect" is registered as a template in the repository | |
| Set the YAKTRACKEDITOR environment variable to "echo" (assuming Unix) | |
| Invoke the submitter with the command submit defect | System should echo the path of the file new_issue.xml and show success at filing a new defect; note path location. Verify that the temporary directory listed in the echoed path is removed after program exit. Verify that the new defect file exists and shows blank (or default) values for the specified template. Verify that the new defect file shows a root attribute element "yaktrackid" with a value equal to the filename. |
| Set the YAKTRACKEDITOR environment variable to an invalid (garbage) command. Invoke the submitter with the command submit defect | Program should exit showing an error executing the editor (your command); no defect should be filed. Verify that the temporary directory created in the process has been removed. |
| Set the YAKTRACKEDITOR environment variable to a valid edit command; invoke the submitter with the command submit defect | Program should invoke the editor specified in the environment variable. |
| Edit the issue and exit the editor | Program should show successful edit and filed report. Check the new file on disk to ensure contents are correctly changed. Check no temporary directory still exists. |
This test case addresses SUBMIT.FR9, SUBMIT.FR10, SUBMIT.FR11, SUBMIT.FR12, SUBMIT.FR13, SUBMIT.FR14
SUBMIT.EDITOR.TC and all its pretests
Table 2-4. Test script: CREATE-REPOSITORY.TC
| User actions | System Response/User examination |
| Set up a test repository as in SUBMIT.EDITOR.TC, with a "defect" template. | |
| Ensure the YAKTRACKEDITOR environment variable is not set. Invoke the submitter with the command submit defect | System should begin displaying a series of prompts and menus |
| Walk through the prompts and menus. Press "return" at least once when a default value is shown, to ensure that default values are tested | System should continue to generate prompts and menus throughout. Verify that enumerated attributes (defect@confidential, for example) are presented as a numbered menu. Verify that text attributes (defect@product_name) are presented as a text line. Verify that content children (defect/submitterid) are presented as text fillins. Verify that each prompt shows "where" the user is in the document structure. Verify that default values are shown where they exist. |
| Ensure that the new issue is filed successfully. Verify that the new file in the repository contains the correct contents |
This test case addresses EDIT.FR1, EDIT.FR2, EDIT.FR3, EDIT.FR4
SUBMIT.TC and all its pretests
Table 2-5. Test script: EDIT.TC
| User actions | System Response/User examination |
| Create a workspace and repository as in SUBMIT.TC | |
| Submit a new issue | System should accept the issue and return an issue ID |
| Set the YAKTRACKEDITOR environment variable to a known editor. Edit the issue with the command edit [your defect id]. | System should bring up the editor with the full text of the desired defect |
| Edit the issue, changing a few known variables, and exit | System should return a success message with the same issue ID |
| Verify that the on-disk issue was in fact changed | |
| Remove the YAKTRACKEDITOR environment variable from the environment | |
| Edit the issue again, noting proper construction of menus and prompts. Verify on at least one entry that pressing "return" keeps default contents. | System should walk through all prompts and file the issue successfully. |
This test case addresses CAT.FR1, CAT.FR2, CAT.FR3
SUBMIT.TC and associated pretests
Table 2-6. Test script: CAT.TC
| User actions | System Response/User examination |
| Enter a new directory with no repository above it. | |
| Execute the cat command with cat | System should respond with "No parent repository exists; cannot cat issues." |
| Create a test repository and submit a new issue | |
| Execute the cat command with a template that does not exist | System should respond by showing that such a template does not exist, listing the current valid templates, and listing the help text for the cat command. |
| Submit a new defect (or other valid issue template) | System should submit the defect and return an ID |
| Cat that issue with the yaktrack command cat defect [your issue id] | System should respond with the text of the issue. Verify that this text is the same as that in the appropriate repository file |
This test case addresses QUERY.FR1 through QUERY.FR10
SUBMIT.TC and associated pretests
Table 2-7. Test script: QUERY.TC
| User actions | System Response/User examination |
| Create a test repository using good_template as the defect template, and file both sample_defect_1 and sample_defect_2. | |
| Query the repository using query defect defect | System should respond by listing both defects |
| Query the repository using query defect defect/submitterid | System should respond by listing only the "submitterid" nodes from both defects |
| Query the repository using query defect defect[@confidential] | System should respond by listing both defects |
| Query the repository using query defect defect[@confidential="yes"] | System should respond by listing only the first defect |
| Query the repository using query defect defect[submitterid="jo.*"] | System should respond by listing only the second defect |
This test case addresses CONFIGURE.FR1, CONFIGURE.FR2.
Automated regression tests
Table 2-8. Test script: CONFIGURE.TC
| User actions | System Response/User examination |
| Enter a directory above which no repository is located. Invoke yaktrack with no arguments | System should respond with a message that no repository is found and recommend creating one. |
| Enter a directory containing a repository; enter a subdirectory a few levels below it (making intermediate directories as necessary). Invoke the tool with no arguments | System should respond with regular help message, minus the message that no repository was found. |
This test case addresses PROGRESSIVE-HELP.FR1 through PROGRESSIVE-HELP.FR8
Automated unit tests
Table 2-9. Test script: PROGRESSIVE-HELP.TC
| User actions | System Response/User examination |
| Inspect source code and ensure that all command objects loaded into the yaktrack command object derive from composite_command. Verify that the composite_command class implements the possible_subcommands_text, is_interactive, has_been_invoked, and last_invocation_succeeded functions. | |
| Inspect source code of each command and ensure they have a short_description_text and long_description_text function. | |
| Invoke each command with invalid or no arguments and ensure that help text is displayed | |
| Invoke any yaktrack command. Verify that feedback is displayed through standard input. This does not verify that the feedback is correct, just that it is displayed. |
This test case addresses FEEDBACK.FR1, FEEDBACK.FR2, FEEDBACK.FR3
Automated unit tests
Table 2-10. Test script: FEEDBACK.TC
| User actions | System Response/User examination |
| Inspect source code and ensure that all command objects loaded into the yaktrack command object derive from composite_command. Verify that the composite_command class implements clear_feedback and append_feedback. | |
| Invoke any yaktrack command. Verify that feedback is displayed through standard input. This does not verify that the feedback is correct, just that it is displayed. |