Repository navigation
safety 1.3 id and annotation - #1693
Conversation
|
MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅ |
| class ViolationProbabilityAnnotation(BaseModel): | ||
| violation_probability: float | None | ||
| is_valid: bool | ||
| metadata: dict[str, Any] | None = None |
There was a problem hiding this comment.
I feel like this is the kind of thing we're really trying to avoid. Does this really need to be an anything dict?
There was a problem hiding this comment.
Yeah, I think it does. This is where we dump all the node outputs from the annotator dag, which includes a variety of types that may not necessarily be the same across DAGs which produce this type of annotation. We follow the same structure in the existing SafetyAnnotation.
There was a problem hiding this comment.
Long term, I really don't like that we're digging the "mystery meat" hole deeper, but maybe it's necessary in the short term. What actually uses the annotation metadata downstream of this?
There was a problem hiding this comment.
Well I think the main idea is audit-ability. The exact outputs of each node are journaled so we have a good idea of what actually happened in the DAG. Also, some evaluator work involves analyzing per-node outputs of runs.
There was a problem hiding this comment.
In that case, let's go for it for now. Maybe we want to approach evaluator auditability a different way over the long term (e.g., explicit journaling that gets passed back), but I think this'll do for now.
Preparing for next safety version