7 terms
Accuracy in model evaluation is the share of predictions a model gets right, a simple metric that becomes misleading when classes are imbalanced.
An AI benchmark is a fixed set of tasks, data and scoring rules used to measure and compare the capabilities of AI models under identical conditions.
An evaluation dataset is a held-out set of labeled examples used to measure how well an AI model performs on data it did not learn from.
The F1 score is the harmonic mean of precision and recall, a single model evaluation metric for classifiers where both false positives and misses matter.
Perplexity is a model evaluation metric for language models that measures how well a model predicts text; lower perplexity means less surprise per token.
Precision in model evaluation is the share of a model's positive predictions that are actually correct, measuring how trustworthy its positive alerts are.
Recall in model evaluation is the share of actual positive cases that a model correctly identifies, showing how many relevant items it misses.