Recording metrics

When writing tests, you can use tth_metrics to store values of different metrics collected during the test execution. It provides a global store during a pytest session. In order to use the store, you need to import it as:

from tth.data import tth_metrics

Module tth_metrics provides the metrics store in the form of a dictionary where keys are names of metric groups and values are dictionaries with metric names as dictionary keys and recorded metric values as dictionary values. Metric value can be any of the following types: string, boolean, integer, float, or None. If you want to save datetime instance as a metric value, it is suggested to be saved as string in ISO 8601 format (having timezone suffix like +00:00 is optional). Alternatively, datetime can be saved as integer that represents epoch time (Unix timestamp).

Example of the store:

{
  "group_1": {
    "metric_1_name": "foo",
    "metric_2_name": 234
  },
  "group_2": {
    "metric_1_name": "bar",
    "metric_2_name": 88
  }
}

If the same metric is calculated in several different tests, usually group names would correspond to test names (or test identifiers). Specifying a group name is optional, and 'default' is used as the group name if not specified. After the last test is finished, the store is serialized into a file that is attached to the last executed test and thus included in the Allure report as an attachment.

The following code demonstrates the usage of tth_metrics:

from tth.data import tth_metrics


def test_example():
    tth_metrics.set("voltage", 234)  # add 'voltage' metric with value 234 to 'default' group
    tth_metrics.set("speed", 88, "max_val")  # add 'speed' metric with value 88 to 'max_val' group

    voltage = tth_metrics.get("voltage")  # read value of 'voltage' metric from 'default' group
    speed = tth_metrics.get("speed", "max_val")  # read value of 'speed' metric from 'max_val' group

    tth_metrics.delete("speed")  # delete 'speed' metric from 'default' group
    store_copy = tth_metrics.all()  # obtain copy of metrics store
    tth_metrics.clear()  # clear metrics store

Reference of tth_metrics

all()

Get a copy of all stored metrics.

Returns:

Name	Type	Description
store	dict	A dictionary of all groups and their settings.

clear()

Clear all groups with metrics.

delete(name, group='default')

Delete a metric by name from a specific group.

Parameters:

Name	Type	Description	Default
name	str	The name of the metric.	required
group	str	The name of the group of metrics. Defaults to 'default'.	'default'

delete_group(group='default')

Delete an entire group of metrics and all its stored values.

Parameters:

Name	Type	Description	Default
group	str	The name of the group of metrics to delete.	'default'

get(name, group='default', default=None)

Retrieve a metric value by name from a specific group.

Parameters:

Name	Type	Description	Default
name	str	The name of the setting.	required
group	str	The name of the group of metrics. Defaults to 'default'.	'default'
default	any	The default value to return if the metric or group is not found. Defaults to None.	None

Returns:

Name	Type	Description
value	any	The stored metric value, or the default if not found.

Raises:

Type	Description
ValueError	If the group argument is not a string instance.

get_group(group='default')

Retrieve a specific group.

Parameters:

Name	Type	Description	Default
group	str	The name of the group of metrics. Defaults to 'default'.	'default'

Returns:

Name	Type	Description
data	dict	The copy of group of metrics, or None if not found.

Raises:

Type	Description
ValueError	If the group argument is not a string instance.

is_empty()

Check if the store is empty.

Returns:

Name	Type	Description
indicator	bool	True if no groups exist, False otherwise.

is_group_empty(group='default')

Check if a specific group of metrics is empty.

Parameters:

Name	Type	Description	Default
group	str	The name of the group of metrics to check. Defaults to 'default'.	'default'

Returns:

Name	Type	Description
indicator	bool	True if the group does not exist or has no metrics, False otherwise.

Raises:

Type	Description
ValueError	If the group argument is not a string instance.

set(name, value, group='default')

Sets new metric value.

Parameters:

Name	Type	Description	Default
name	str	The name of the metric.	required
value	str | float | int | bool | NoneType	The value of the metric.	required
group	str	The name of the group of metrics. Defaults to 'default'.	'default'

Raises:

Type	Description
ValueError	Invalid arguments provided