Ticket #47 (new enhancement)
Adding a test-run report to the test-suite.
| Reported by: | ivan.zapreev | Owned by: | |
|---|---|---|---|
| Priority: | minor | Milestone: | MRMC future |
| Component: | Test Suite | Version: | 1.3.1 |
| Keywords: | Cc: | nguen |
Description (last modified by ivan.zapreev) (diff)
I would suggest to add the accumulated survey (log) on the passed/failed tests at the end of the testing process. This log should also include all the .diff data for the failed tests.
It might be a god idea to have separate log files for internal, functional and performance tests:
=======INTERNAL TESTS=======
TESTS PASSED: ??
TESTS FAILED: ??
------------------
TESTS IN TOTAL: ??
-----FAILED TESTS DIFFS------
.out vs .golden
--- internal/bitset
<The diff file content goes here>
.....
The same kind of file should be present for the functional tests
For the performance tests, the format should be different, something like the following:
==========PERFORMANCE TESTS==========
--- <Full test case name>
TESTS PASSED: ??
TESTS FAILED: ??
--- <Full test case name>
TESTS PASSED: ??
TESTS FAILED: ??
.......
-----FAILED CASE TEST NAMES ------
<Full test case name>
<Full test case name>
<Full test case name>
.......
Note that for performance tests:
1. The diff outputs are not necessary.
2. The test case is repeted several times, so if there is at least one failure of this test case, it should be present in the list of failed test cases.
