Description
Publish a short structured document with each model stating how it was built and evaluated, its intended uses and users, performance across groups, and its limitations.
A model card accompanies a trained model and discloses what stakeholders need to judge whether it fits their purpose, covering ethical and fairness considerations as well as metrics. Standard sections are model details; intended use, including primary uses, users and out-of-scope uses; factors (demographic, environmental, technical); metrics, decision thresholds and variation; evaluation data; training data; quantitative analyses, both unitary and intersectional; ethical considerations; and caveats and recommendations. Cards sit within a wider responsible-AI process: finding sources of bias at each stage, understanding data limits with domain experts and affected people, and weighing trade-offs between privacy, compression and fairness. Because cards must be updated with every model version, generation should be automated. For UX, the intended-use, factors and caveats sections are where user research about contexts and affected groups becomes a durable, checkable commitment.
Attributes
- Discipline
- Computational & Machine Learning
- Loss type
- Siloed
- Stage
- Evaluation, Post-launch
- Data
- Mixed
- Effort
- Days
Steps
- Review sources of bias across training data, labelling, features, objective and evaluation.
- Run fine-grained evaluation sliced by relevant groups and their intersections.
- Fill each section: model details, intended use, factors, metrics, evaluation data, training data, quantitative analyses, ethical considerations, caveats and recommendations.
- Involve domain experts and affected users in drafting intended and out-of-scope uses.
- Publish the card with the model and link it from product documentation where users or customers can see it.
- Regenerate the card on every model update, ideally automatically from the evaluation pipeline.
Sources
- Designing Machine Learning Systems (Huyen, 2022)
Also known as
- Model cards for model reporting
- Model documentation
Categories
- Analytics & Behaviour
- Computational & Machine Learning