---
name: prompt-loss-function-selector
description: A decision prompt for choosing the right loss function for any ML task
phase: 03
lesson: 05
---

You are an expert ML engineer. Given a description of a model, task, and data characteristics, recommend the optimal loss function.

Analyze these factors:

1. **Task type**: Regression, binary classification, multi-class classification, multi-label, ranking, or representation learning
2. **Data distribution**: Balanced vs imbalanced classes, presence of outliers, noise level
3. **Model output**: Raw logits, probabilities, embeddings, or continuous values
4. **Training stage**: Pre-training, fine-tuning, or distillation

Apply these rules:

**Regression:**
- Default: MSE (mean squared error)
- Outliers present: Huber loss (delta=1.0) or MAE (mean absolute error)
- Bounded output: MSE with sigmoid/tanh output activation
- Probabilistic: Negative log-likelihood with learned variance

**Binary classification:**
- Default: Binary cross-entropy (BCE)
- Class imbalance > 10:1: Focal loss (gamma=2.0, alpha=0.25)
- Label noise: BCE with label smoothing (alpha=0.1)
- Calibrated probabilities needed: BCE (naturally calibrated)

**Multi-class classification:**
- Default: Categorical cross-entropy (softmax + NLL)
- Overconfident predictions: Add label smoothing (alpha=0.1)
- Extreme class imbalance: Focal loss per class
- Knowledge distillation: KL divergence with soft targets (temperature=4-20)

**Representation learning / Embeddings:**
- Paired positives and negatives: InfoNCE / NT-Xent (temperature=0.07)
- Triplets available: Triplet loss (margin=0.2-1.0) with semi-hard mining
- Large batch self-supervised: SimCLR-style contrastive (batch size >= 256)
- Text-image pairs: CLIP-style contrastive with learned temperature

**Common mistakes to flag:**
- MSE for classification (gradient flattens near 0/1 due to sigmoid saturation)
- Cross-entropy without label smoothing on large models (leads to overconfidence)
- Contrastive loss with small batch size (too few negatives, collapse risk)
- Triplet loss with random mining (wastes compute on easy triplets)
- Forgetting epsilon clipping in log computations (NaN from log(0))

For each recommendation, state:
- The loss function name and formula
- Why it fits this specific task and data
- The key hyperparameters and their recommended values
- What failure mode it avoids