How do you conduct a risk-based testing strategy for AI?

• A. Use only synthetic data and avoid real-world data.
• B. Identify risk categories, design tests targeting critical risks, use synthetic data for edge cases, and monitor post-deployment.
• C. Focus testing only on model accuracy in lab conditions.
• D. Rely solely on developer intuition and not perform structured risk testing.

Answer: (B)

Focus on risks across the AI system and build tests around what could go wrong, not just whether the model is accurate. The best approach starts by identifying risk categories such as safety, fairness, privacy, robustness to data shifts, and reliability under real-world conditions. Then you design tests that specifically target those critical risks, so you’re not wasting effort on harmless or unlikely scenarios. Using synthetic data helps you recreate edge cases or rare conditions that real-world data might not capture, enabling you to probe how the system behaves under stress without risking sensitive information or dangerous real data. Finally, you monitor the AI after deployment to detect drift, new failure modes, or unexpected user interactions, feeding lessons back into updates and safeguards.

The other options fall short because relying only on synthetic data and avoiding real-world data misses the richness and unpredictability of actual use; focusing only on lab accuracy ignores drift, abuse, bias, and safety concerns that appear in the wild; and relying solely on developer intuition sacrifices the structured, evidence-based assessment needed to manage risks effectively.