16262675714

Committed 14 Jul 2025 04:49AM UTC coverage: 46.419% (-48.0%) from 94.444%

Build # 16262675714

Build Type

push

github

Committed by

web-flow

Commit Message

Merge pull request #3 from sisl/feat/core

Initial implementation of core AST algorithm.

Run Details

160 of 361 new or added lines in 18 files covered. (44.32%)

175 of 377 relevant lines covered (46.42%)

0.93 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

66.67

/src/astra_rl/moderators/detoxify.py

"""
detoxify.py
Moderator to call into the Detoxify engine.
"""

from typing import Sequence

from detoxify import Detoxify

from astra_rl.core.moderator import Moderator


class DetoxifyModerator(Moderator[str, str]):
    """Moderator that wraps the Detoxify library for toxicity detection.

    https://github.com/unitaryai/detoxify

    Attributes:
        harm_category (str): The category of harm to detect (default is "toxicity"); see below.
        variant (str): The variant of the Detoxify model to use (default is "original").

    Notes:
        Possible harm categories
        include "toxicity", "severe_toxicity", "obscene", "identity_attack",
        "insult", "threat", "sexual_explicit".

        Possible variants
        Include "original", "multilingual", "unbiased".
    """

    def __init__(self, harm_category: str = "toxicity", variant: str = "original"):
        self.model = Detoxify(variant)
        self.harm_category = harm_category

    def moderate(self, x: Sequence[str]) -> Sequence[float]:
        # we ignore typing here because we don't actually have the ability
        # to get typing information from detoxify
        return self.model.predict(x)[self.harm_category]  # type: ignore

1	"""
2	detoxify.py
3	Moderator to call into the Detoxify engine.
4	"""
5
6	from typing import Sequence	2✔
7
8	from detoxify import Detoxify	2✔
9
10	from astra_rl.core.moderator import Moderator	2✔
11
12
13	class DetoxifyModerator(Moderator[str, str]):	2✔
14	"""Moderator that wraps the Detoxify library for toxicity detection.
15
16	https://github.com/unitaryai/detoxify
17
18	Attributes:
19	harm_category (str): The category of harm to detect (default is "toxicity"); see below.
20	variant (str): The variant of the Detoxify model to use (default is "original").
21
22	Notes:
23	Possible harm categories
24	include "toxicity", "severe_toxicity", "obscene", "identity_attack",
25	"insult", "threat", "sexual_explicit".
26
27	Possible variants
28	Include "original", "multilingual", "unbiased".
29	"""
30
31	def __init__(self, harm_category: str = "toxicity", variant: str = "original"):	2✔
NEW 32	self.model = Detoxify(variant)	×
NEW 33	self.harm_category = harm_category	×
34
35	def moderate(self, x: Sequence[str]) -> Sequence[float]:	2✔
36	# we ignore typing here because we don't actually have the ability
37	# to get typing information from detoxify
NEW 38	return self.model.predict(x)[self.harm_category] # type: ignore	×

sisl / astra-rl / 16262675714

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous