Obtaining Coverage¶
BugZoo’s Python API provides a number of uniform, language-independent interfaces for collecting line coverage information for individual test cases, entire test suites, and arbitrary executions.
Obtaining Coverage for Bugs¶
Complete test suite coverage information for a given bug can be obtained in
the form of a TestSuiteCoverage object by using
client.bugs.coverage(), as shown below. client.bugs.coverage()
will attempt to load precomputed coverage data for the given bug from its
on-disk cache; if no coverage data can be found, it will be recomputed from
scratch and saved to disk for future requests.
bug = client.bugs['manybugs:python:69223-69224']
coverage = client.bugs.coverage(bug)
Coverage information for a particular test is represented as a
TestCoverage object, and can be retrieved from a
TestSuiteCoverage object by supplying the name of a test to its
[] operator:
coverage_for_suite = client.bugs.coverage(bug)
coverage_for_test = coverage_for_suite["p1"]
TestSuiteCoverage also provides an iterator over the names of the
tests included within its report:
for test_name in coverage_for_suite:
coverage_for_test = coverage_for_suite[test_name]
Discuss test outcomes and file lines.
Obtaining Coverage for Containers¶
Like client.bugs.coverage(), client.containers.coverage() is used
to compute coverage information for an entire test suite. (Note that, unlike
client.bugs.coverage(), client.containers.coverage() does not
cache coverage information to disk as containers are ephemeral.)
coverage_for_suite = client.containers.coverage(container)
Coverage information for individual test executions (and executions of arbitrary shell commands) can also be obtained by first instrumenting the the container, then executing the test (or shell command) for which coverage shoud be collected, before finally extracting a report of the source code lines that were executed:
client.containers.prepare_for_coverage(container)
outcome = client.containers.test(container, test)
lines = client.containers.read_coverage(container)
client.containers.cleanup_coverage(container)
coverage = TestCoverage(test.name, outcome, lines)
Computing Fault Spectra using Coverage¶
Fault spectra, represented as Spectra objects, are used to provide a
concise summary of the number of passing and failing tests that touch (or do
not touch) each line in the program. Each entry, or row, in a fault spectra
is encoded as a LineSpectra object, and represents a line in the
program, and is represented by four numbers: ep, np,
ef, and nf.
API Reference¶
-
class
FileLine[source]¶ Represents a one-indexed line within a specific file.
-
num→ int¶ The one-indexed number of the line.
-
filename→ str¶ The name of the file to which the line belongs.
-
-
class
FileLineSet[source]¶ A set of file lines.
-
__iter__() → Iterator[bugzoo.core.fileline.FileLine][source]¶ Returns an iterator over the lines contained in this set.
-
-
class
TestCoverage[source]¶ Provides complete line coverage information for all files and across all tests within a given project.
-
class
TestSuiteCoverage[source]¶ Holds coverage information for all tests belonging to a particular program version.
-
__getitem__(name: str) → bugzoo.core.coverage.TestCoverage[source]¶ Retrieves coverage information for a given test case.
- Parameters
name – the name of the test case.
- Raises
KeyError – if there is no coverage information for the given test case.
-
-
class
Spectra[source]¶ Contains a summary of the number of passing and failing tests that cover each line in a given project.
-
__iter__() → Iterator[bugzoo.core.fileline.FileLine][source]¶ Returns an iterator over the source code lines that are represented in this spectra.
-
__getitem__(line: bugzoo.core.fileline.FileLine) → bugzoo.core.spectra.LineSpectra[source]¶ Retrieves the spectra information for a given line.
-