Document the display and html report in the spec
summary: Report test results
+story:
+ As a tester I want to have a nice overview of results once
+ the testing if finished.
+description:
+ Report test results according to user preferences.
+/display:
+ summary: Show results in the terminal window
+ story:
+ As a tester I want to see test results in the plain text
+ form in my shell session.
+ description:
+ Test results will be displayed as part of the command line
+ tool output directly in the terminal. Allows to select the
+ desired level of verbosity
+ example: |
+ tmt run -l report # overall summary only
+ tmt run -l report -v # individual test results
+ tmt run -l report -vv # show full paths to logs
+ tmt run -l report -vvv # provide complete test output
+ implemented: tmt/steps/report/display.py
+
+/html:
+ summary: Generate a web page with test results
+ story:
+ As a tester I want to review results in a nicely arranged
+ web page with links to detailed test output.
+ description:
+ Create a local ``html`` file with test results arranged in
+ a table. Optionally open the page in the default browser.
+ example: |
+ tmt run --all report --how html
+ tmt run --all report --how html --open
+ tmt run -l report -h html -o
+ implemented: tmt/steps/report/html.py
+
+/file:
description: |
- Create report for all the test results. The report is saved
- into ``report.yaml`` file with the following format::
+
+ Save the report into a ``report.yaml`` file with the
+ following format::
result: OVERALL_RESULT
plans:
@@ -37,37 +74,23 @@ description: |
* failed - test finished and failed
* error - a problem encountered during test execution
- Note the priority of test results is as written above, with
- ``info`` having the lowest priority and ``error`` has the
- highest. This is important for ``PLAN_RESULT``.
+ Note the priority of test results is as written above,
+ with ``info`` having the lowest priority and ``error`` has
+ the highest. This is important for ``PLAN_RESULT``.
Where ``PLAN_RESULT`` is the overall result or all test
results for the plan run. It has the same values as
``TEST_RESULT``. Plan result is counted according to the
priority of the test outcome values. For example:
- * if the test results are info, passed, passed - the plan
- result will be passed
- * if the test results are info, passed, failed - the plan
- result will be failed
- * if the test results are failed, error, passed - the plan
- result will be error
+ * if the test results are info, passed, passed - the
+ plan result will be passed
+ * if the test results are info, passed, failed - the
+ plan result will be failed
+ * if the test results are failed, error, passed - the
+ plan result will be error
- Where ``LOG_PATH`` is the test log output path, relative to
- the execute step plan run directory. The log can be a single
- log path or a list of log paths, in case the test has produced
- more log files.
-
- This step also defines additional notification settings, which
- can be used by CI or reporting systems to enable and customize
- notifications. The following values are planned:
-
- email
- send email notification
- irc
- notify on irc chat
-
-example: |
- report:
- email:
- - email@address.org
+ Where ``LOG_PATH`` is the test log output path, relative
+ to the execute step plan run directory. The log can be a
+ single log path or a list of log paths, in case the test
+ has produced more log files.
