6009595088

Committed 29 Aug 2023 08:07AM UTC coverage: 88.281% (-0.001%) from 88.282%

Build # 6009595088

Build Type

push

github

Committed by

web-flow

Commit Message

Reimplement AnalysisResultTable (#1252)

### Summary

Follow-up of #1133 for data frame. 

### Details and comments

I realized I had some mistake when I wrote a base class of
`ThreadSafeDataFrame`. We plan to move the curve data (XY data) to the
artifact for better reusability of the data points. In addition,
[CurveData](https://github.com/Qiskit-Extensions/qiskit-experiments/blob/c66034c90/qiskit_experiments/curve_analysis/curve_data.py#L88-L113)
object which is a container of the XY data is also replaced with the
data frame in a follow up PR. Since this table should contain predefined
columns such as `xval`, `yval`, `yerr`, I was assuming this container
could be a subclass of `ThreadSafeDataFrame` -- but this was not a right
assumption.

Since the curve analysis always runs on a single thread (analysis
callbacks might be run on multiple threads and thus
`AnalysisResultTable` must be thread-safe), this curve data frame
doesn't need to be thread-safe object. In this PR, a functionality to
define default columns of the table is reimplemented as a Python mixin
class so that the function to add default columns will be used also for
curve data frame without introducing the unnecessary thread safe
mechanisms. `AnalysisResultTable` is a thread safe container but the
functionality to manage default columns is delegated to the mixin.

Run Details

84 of 84 new or added lines in 4 files covered. (100.0%)

12384 of 14028 relevant lines covered (88.28%)

3.53 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

84.95

/qiskit_experiments/framework/analysis_result_table.py

# This code is part of Qiskit.
#
# (C) Copyright IBM 2023.
#
# This code is licensed under the Apache License, Version 2.0. You may
# obtain a copy of this license in the LICENSE.txt file in the root directory
# of this source tree or at http://www.apache.org/licenses/LICENSE-2.0.
#
# Any modifications or derivative works of this code must retain this
# copyright notice, and modified files need to carry a notice indicating
# that they have been altered from the originals.

"""Table representation of analysis results."""

import logging
import threading
import re
import uuid
import warnings
from typing import List, Dict, Union, Optional, Any

import numpy as np
import pandas as pd

from qiskit_experiments.database_service.utils import ThreadSafeContainer

from .table_mixin import DefaultColumnsMixIn

LOG = logging.getLogger(__name__)


class AnalysisResultContainer(pd.DataFrame, DefaultColumnsMixIn):
    """Data container of the thread-safe analysis result table."""

    @classmethod
    def _default_columns(cls) -> List[str]:
        return [
            "name",
            "experiment",
            "components",
            "value",
            "quality",
            "experiment_id",
            "result_id",
            "tags",
            "backend",
            "run_time",
            "created_time",
        ]

    @property
    def _constructor(self):
        # https://pandas.pydata.org/pandas-docs/stable/development/extending.html
        return AnalysisResultContainer


class AnalysisResultTable(ThreadSafeContainer):
    """A thread-safe table form container of analysis results.

    This table is a dataframe wrapper with the thread-safe mechanism with predefined columns.
    This object is attached to the :class:`.ExperimentData` container to store
    analysis results. Each table row contains series of metadata in addition to the
    result value itself.

    User can rely on the dataframe filtering mechanism to analyze large scale experiment
    results, e.g. massive parallel experiment and batch experiment outcomes, efficiently.
    See `pandas dataframe documentation <https://pandas.pydata.org/docs/index.html>`_
    for more details.
    """

    VALID_ID_REGEX = re.compile(r"\A(?P<short_id>\w{8})-\w{4}-\w{4}-\w{4}-\w{12}\Z")

    def _init_container(self, init_values: Any):
        if init_values is None:
            return AnalysisResultContainer()
        return init_values

    def result_ids(self) -> List[str]:
        """Return all result IDs in this table."""
        with self._lock:
            return self._container["result_id"].to_list()

    def filter_columns(self, columns: Union[str, List[str]]) -> List[str]:
        """Filter columns names available in this table.

        Args:
            columns: Specifying a set of columns to return. You can pass a list of each
                column name to return, otherwise builtin column groups are available.

                    * "all": Return all columns, including metadata to communicate
                        with experiment service, such as entry IDs.
                    * "default": Return columns including analysis result with supplementary
                        information about experiment.
                    * "minimal": Return only analysis subroutine returns.

        Raises:
            ValueError: When column is given in string which doesn't match with any builtin group.
        """
        with self._lock:
            if columns == "all":
                return self._container.columns
            if columns == "default":
                return [
                    "name",
                    "experiment",
                    "components",
                    "value",
                    "quality",
                    "backend",
                    "run_time",
                ] + self._container.extra_columns()
            if columns == "minimal":
                return [
                    "name",
                    "components",
                    "value",
                    "quality",
                ] + self._container.extra_columns()
            if not isinstance(columns, str):
                out = []
                for column in columns:
                    if column in self._container.columns:
                        out.append(column)
                    else:
                        warnings.warn(
                            f"Specified column name {column} does not exist in this table.",
                            UserWarning,
                        )
                return out
        raise ValueError(
            f"Column group {columns} is not valid name. Use either 'all', 'default', 'minimal'."
        )

    def get_entry(
        self,
        index: str,
    ) -> pd.Series:
        """Get entry from the dataframe.

        Args:
            index: Name of entry to acquire.

        Returns:
            Pandas Series of acquired entry. This doesn't mutate the table.

        Raises:
            ValueError: When index is not in this table.
        """
        with self._lock:
            if index not in self._container.index:
                raise ValueError(f"Table index {index} doesn't exist in this table.")

            return self._container.loc[index]

    # pylint: disable=arguments-renamed
    def add_entry(
        self,
        result_id: Optional[str] = None,
        **kwargs,
    ) -> pd.Series:
        """Add new entry to the table.

        Args:
            result_id: Result ID. Automatically generated when not provided.
                This must be valid hexadecimal UUID string.
            kwargs: Description of new entry to register.

        Returns:
            Pandas Series of added entry. This doesn't mutate the table.

        Raises:
            ValueError: When the truncated result id causes a collision in the table.
        """
        if not result_id:
            result_id = self._unique_table_index()

        matched = self.VALID_ID_REGEX.match(result_id)
        if matched is None:
            warnings.warn(
                f"The result ID {result_id} is not a valid result ID string. "
                "This entry might fail in saving with the experiment service.",
                UserWarning,
            )
            short_id = result_id[:8]
        else:
            # Short unique index is generated from result id.
            # Showing full result id unnecessary occupies horizontal space of the html table.
            # This mechanism is similar with the github commit hash.
            short_id = matched.group("short_id")

        with self._lock:
            if short_id in self._container.index:
                raise ValueError(
                    f"The short ID of the result_id '{short_id}' already exists in the "
                    "experiment data. Please use another ID to avoid index collision."
                )

            return self._container.add_entry(
                index=short_id,
                result_id=result_id,
                **kwargs,
            )

    def drop_entry(
        self,
        index: str,
    ):
        """Drop specified labels from rows.

        This directly calls :meth:`.drop` of the DataFrame container object.

        Args:
            index: Name of entry to drop.

        Raises:
            ValueError: When index is not in this table.
        """
        with self._lock:
            if index not in self._container.index:
                raise ValueError(f"Table index {index} doesn't exist in this table.")
            self._container.drop(index, inplace=True)

    def clear(self):
        """Remove all elements from this container."""
        with self._lock:
            self._container = AnalysisResultContainer()

    def _unique_table_index(self):
        """Generate unique UUID which is unique in the table with first 8 characters."""
        with self._lock:
            n = 0
            while n < 1000:
                tmp_id = str(uuid.uuid4())
                if tmp_id[:8] not in self._container.index:
                    return tmp_id
        raise RuntimeError(
            "Unique result_id string cannot be prepared for this table within 1000 trials. "
            "Reduce number of entries, or manually provide a unique result_id."
        )

    def _repr_html_(self) -> Union[str, None]:
        """Return HTML representation of this dataframe."""
        with self._lock:
            return self._container._repr_html_()

    def __json_encode__(self) -> Dict[str, Any]:
        with self._lock:
            return {
                "class": "AnalysisResultTable",
                "data": self._container.to_dict(orient="index"),
            }

    @classmethod
    def __json_decode__(cls, value: Dict[str, Any]) -> "AnalysisResultTable":
        if not value.get("class", None) == "AnalysisResultTable":
            raise ValueError("JSON decoded value for AnalysisResultTable is not valid class type.")

        instance = object.__new__(cls)
        instance._lock = threading.RLock()
        instance._container = AnalysisResultContainer.from_dict(
            data=value.get("data", {}),
            orient="index",
        ).replace({np.nan: None})
        return instance

1	# This code is part of Qiskit.
2	#
3	# (C) Copyright IBM 2023.
4	#
5	# This code is licensed under the Apache License, Version 2.0. You may
6	# obtain a copy of this license in the LICENSE.txt file in the root directory
7	# of this source tree or at http://www.apache.org/licenses/LICENSE-2.0.
8	#
9	# Any modifications or derivative works of this code must retain this
10	# copyright notice, and modified files need to carry a notice indicating
11	# that they have been altered from the originals.
12
13	"""Table representation of analysis results."""	4✔
14
15	import logging	4✔
16	import threading	4✔
17	import re	4✔
18	import uuid	4✔
19	import warnings	4✔
20	from typing import List, Dict, Union, Optional, Any	4✔
21
22	import numpy as np	4✔
23	import pandas as pd	4✔
24
25	from qiskit_experiments.database_service.utils import ThreadSafeContainer	4✔
26
27	from .table_mixin import DefaultColumnsMixIn	4✔
28
29	LOG = logging.getLogger(__name__)	4✔
30
31
32	class AnalysisResultContainer(pd.DataFrame, DefaultColumnsMixIn):	4✔
33	"""Data container of the thread-safe analysis result table."""	4✔
34
35	@classmethod	4✔
36	def _default_columns(cls) -> List[str]:	4✔
37	return [	4✔
38	"name",
39	"experiment",
40	"components",
41	"value",
42	"quality",
43	"experiment_id",
44	"result_id",
45	"tags",
46	"backend",
47	"run_time",
48	"created_time",
49	]
50
51	@property	4✔
52	def _constructor(self):	4✔
53	# https://pandas.pydata.org/pandas-docs/stable/development/extending.html
54	return AnalysisResultContainer	4✔
55
56
57	class AnalysisResultTable(ThreadSafeContainer):	4✔
58	"""A thread-safe table form container of analysis results.	4✔
59
60	This table is a dataframe wrapper with the thread-safe mechanism with predefined columns.
61	This object is attached to the :class:`.ExperimentData` container to store
62	analysis results. Each table row contains series of metadata in addition to the
63	result value itself.
64
65	User can rely on the dataframe filtering mechanism to analyze large scale experiment
66	results, e.g. massive parallel experiment and batch experiment outcomes, efficiently.
67	See `pandas dataframe documentation <https://pandas.pydata.org/docs/index.html>`_
68	for more details.
69	"""
70
71	VALID_ID_REGEX = re.compile(r"\A(?P<short_id>\w{8})-\w{4}-\w{4}-\w{4}-\w{12}\Z")	4✔
72
73	def _init_container(self, init_values: Any):	4✔
74	if init_values is None:	4✔
75	return AnalysisResultContainer()	4✔
76	return init_values	×
77
78	def result_ids(self) -> List[str]:	4✔
79	"""Return all result IDs in this table."""
80	with self._lock:	4✔
81	return self._container["result_id"].to_list()	4✔
82
83	def filter_columns(self, columns: Union[str, List[str]]) -> List[str]:	4✔
84	"""Filter columns names available in this table.
85
86	Args:
87	columns: Specifying a set of columns to return. You can pass a list of each
88	column name to return, otherwise builtin column groups are available.
89
90	* "all": Return all columns, including metadata to communicate
91	with experiment service, such as entry IDs.
92	* "default": Return columns including analysis result with supplementary
93	information about experiment.
94	* "minimal": Return only analysis subroutine returns.
95
96	Raises:
97	ValueError: When column is given in string which doesn't match with any builtin group.
98	"""
99	with self._lock:	4✔
100	if columns == "all":	4✔
101	return self._container.columns	4✔
102	if columns == "default":	4✔
103	return [	4✔
104	"name",
105	"experiment",
106	"components",
107	"value",
108	"quality",
109	"backend",
110	"run_time",
111	] + self._container.extra_columns()
112	if columns == "minimal":	4✔
113	return [	4✔
114	"name",
115	"components",
116	"value",
117	"quality",
118	] + self._container.extra_columns()
119	if not isinstance(columns, str):	×
120	out = []	×
121	for column in columns:	×
122	if column in self._container.columns:	×
123	out.append(column)	×
124	else:
125	warnings.warn(	×
126	f"Specified column name {column} does not exist in this table.",
127	UserWarning,
128	)
129	return out	×
130	raise ValueError(	×
131	f"Column group {columns} is not valid name. Use either 'all', 'default', 'minimal'."
132	)
133
134	def get_entry(	4✔
135	self,
136	index: str,
137	) -> pd.Series:
138	"""Get entry from the dataframe.
139
140	Args:
141	index: Name of entry to acquire.
142
143	Returns:
144	Pandas Series of acquired entry. This doesn't mutate the table.
145
146	Raises:
147	ValueError: When index is not in this table.
148	"""
149	with self._lock:	4✔
150	if index not in self._container.index:	4✔
151	raise ValueError(f"Table index {index} doesn't exist in this table.")	×
152
153	return self._container.loc[index]	4✔
154
155	# pylint: disable=arguments-renamed
156	def add_entry(	4✔
157	self,
158	result_id: Optional[str] = None,
159	**kwargs,
160	) -> pd.Series:
161	"""Add new entry to the table.
162
163	Args:
164	result_id: Result ID. Automatically generated when not provided.
165	This must be valid hexadecimal UUID string.
166	kwargs: Description of new entry to register.
167
168	Returns:
169	Pandas Series of added entry. This doesn't mutate the table.
170
171	Raises:
172	ValueError: When the truncated result id causes a collision in the table.
173	"""
174	if not result_id:	4✔
175	result_id = self._unique_table_index()	4✔
176
177	matched = self.VALID_ID_REGEX.match(result_id)	4✔
178	if matched is None:	4✔
179	warnings.warn(	4✔
180	f"The result ID {result_id} is not a valid result ID string. "
181	"This entry might fail in saving with the experiment service.",
182	UserWarning,
183	)
184	short_id = result_id[:8]	4✔
185	else:
186	# Short unique index is generated from result id.
187	# Showing full result id unnecessary occupies horizontal space of the html table.
188	# This mechanism is similar with the github commit hash.
189	short_id = matched.group("short_id")	4✔
190
191	with self._lock:	4✔
192	if short_id in self._container.index:	4✔
193	raise ValueError(	4✔
194	f"The short ID of the result_id '{short_id}' already exists in the "
195	"experiment data. Please use another ID to avoid index collision."
196	)
197
198	return self._container.add_entry(	4✔
199	index=short_id,
200	result_id=result_id,
201	**kwargs,
202	)
203
204	def drop_entry(	4✔
205	self,
206	index: str,
207	):
208	"""Drop specified labels from rows.
209
210	This directly calls :meth:`.drop` of the DataFrame container object.
211
212	Args:
213	index: Name of entry to drop.
214
215	Raises:
216	ValueError: When index is not in this table.
217	"""
218	with self._lock:	4✔
219	if index not in self._container.index:	4✔
220	raise ValueError(f"Table index {index} doesn't exist in this table.")	4✔
221	self._container.drop(index, inplace=True)	4✔
222
223	def clear(self):	4✔
224	"""Remove all elements from this container."""
225	with self._lock:	4✔
226	self._container = AnalysisResultContainer()	4✔
227
228	def _unique_table_index(self):	4✔
229	"""Generate unique UUID which is unique in the table with first 8 characters."""
230	with self._lock:	4✔
231	n = 0	4✔
232	while n < 1000:	4✔
233	tmp_id = str(uuid.uuid4())	4✔
234	if tmp_id[:8] not in self._container.index:	4✔
235	return tmp_id	4✔
236	raise RuntimeError(	×
237	"Unique result_id string cannot be prepared for this table within 1000 trials. "
238	"Reduce number of entries, or manually provide a unique result_id."
239	)
240
241	def _repr_html_(self) -> Union[str, None]:	4✔
242	"""Return HTML representation of this dataframe."""
243	with self._lock:	×
244	return self._container._repr_html_()	×
245
246	def __json_encode__(self) -> Dict[str, Any]:	4✔
247	with self._lock:	4✔
248	return {	4✔
249	"class": "AnalysisResultTable",
250	"data": self._container.to_dict(orient="index"),
251	}
252
253	@classmethod	4✔
254	def __json_decode__(cls, value: Dict[str, Any]) -> "AnalysisResultTable":	4✔
255	if not value.get("class", None) == "AnalysisResultTable":	4✔
256	raise ValueError("JSON decoded value for AnalysisResultTable is not valid class type.")	×
257
258	instance = object.__new__(cls)	4✔
259	instance._lock = threading.RLock()	4✔
260	instance._container = AnalysisResultContainer.from_dict(	4✔
261	data=value.get("data", {}),
262	orient="index",
263	).replace({np.nan: None})
264	return instance	4✔

Qiskit-Extensions / qiskit-experiments / 6009595088

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous