PMI-CPMAI — PMI Certified Professional in Managing AI Cheat Sheet

Cheat sheet: PMI-CPMAI reference for AI project lifecycle, governance, risk, data, modeling, deployment, and exam decision points.

This Cheat Sheet is for candidates preparing for the PMI Certified Professional in Managing AI (PMI-CPMAI) exam from PMI, exam code PMI-CPMAI. Use it to connect project management judgment with AI-specific lifecycle, data, model, governance, and operational risks.

Use the tables for a quick pre-exam check. Expand a topic’s notes for explanations, examples, and additional distinctions.

Scope and study context

The exam is best approached as a management-of-AI-initiatives exam, not as a pure data science, coding, or machine learning engineering exam. You should understand the technical vocabulary well enough to make sound project, governance, risk, stakeholder, and delivery decisions.

ItemDetail
Vendor/providerPMI
Official exam titlePMI Certified Professional in Managing AI (PMI-CPMAI)
Official exam codePMI-CPMAI
Review focusManaging AI projects, AI lifecycle, business value, data readiness, governance, risk, responsible AI, deployment, and operational monitoring
Practice approachUse PM Mastery practice with original practice questions, topic drills, mock exams, and detailed explanations

After this Cheat Sheet, move into original practice questions rather than rereading notes passively. For PMI Certified Professional in Managing AI (PMI-CPMAI) preparation, effective PM Mastery practice should include:

  • Topic drills on AI lifecycle, data readiness, responsible AI, MLOps, and generative AI.
  • Scenario questions that force tradeoffs between value, risk, governance, and delivery.
  • Mock exams to practice pacing and decision-making.
  • Detailed explanations that explain why the best answer is best and why distractors are weaker.
  • Review of missed questions by concept, not just by answer choice.

Practice Sequence

  1. Do a short mixed quiz to identify weak areas.
  2. Review the relevant Cheat Sheet section.
  3. Complete focused topic drills.
  4. Read detailed explanations for every missed or guessed item.
  5. Build a short error log of recurring traps.
  6. Take a mock exam only after topic weaknesses are improving.
  7. Re-drill missed domains until your reasoning is consistent.

AI Project Lifecycle at a Glance

AI initiatives are not managed like ordinary software delivery. The project manager must handle uncertainty in data, model performance, experimentation, governance, stakeholder trust, and production monitoring.

Lifecycle areaManager focusTypical artifactsExit evidenceCommon exam trap
Business understandingDefine the business problem, value, decision context, and measurable outcomeProblem statement, business case, success metrics, stakeholder map, use-case canvasClear target outcome, success criteria, constraints, acceptance thresholdStarting model selection before validating the business need
Data understandingDetermine what data exists, who owns it, whether it is fit for purposeData inventory, data lineage, data quality profile, access plan, privacy/security reviewData sources identified, quality risks known, permissions understoodAssuming available data is usable data
Data preparationClean, transform, label, enrich, engineer, and split dataData pipeline, labeling guide, feature list, training/validation/test split, data versionReproducible prepared data with documented assumptionsCreating leakage by using future or target-derived data
Model developmentSelect approach, train, tune, compare, and document modelsExperiment log, model candidates, hyperparameters, feature importance, baseline comparisonCandidate model meets technical targets in controlled evaluationTreating the first high-scoring model as production-ready
Model evaluationValidate against business, risk, fairness, security, explainability, and operational criteriaEvaluation report, bias assessment, model card, risk register update, go/no-go decisionModel accepted by business and governance stakeholdersOptimizing only accuracy while ignoring business impact
OperationalizationDeploy, monitor, support, retrain, and manage changeDeployment plan, MLOps pipeline, rollback plan, monitoring dashboard, retraining triggerProduction model monitored with ownership and incident processAssuming deployment is the end of the AI lifecycle
Continuous governanceManage drift, compliance, auditability, accountability, and responsible AI controlsGovernance log, approval records, audit trail, model inventory, incident registerModel remains controlled, explainable, and aligned to valueLeaving governance to technical teams only

Core PMI-CPMAI Distinctions

DistinctionKnow this for the exam
AI project vs software projectAI delivery depends on data quality and experimental model performance, not only requirements and coding. Scope, schedule, and acceptance criteria may need progressive elaboration.
Model output vs business outcomeA statistically strong model is not automatically valuable. Link model metrics to measurable business decisions, cost, risk, adoption, or customer outcomes.
Training vs inferenceTraining builds or tunes the model using data. Inference applies the trained model to new inputs in production or test environments.
Algorithm vs modelAn algorithm is the method. A model is the trained artifact produced by applying an algorithm to data.
Parameter vs hyperparameterParameters are learned during training. Hyperparameters are set before or during experimentation, such as learning rate or tree depth.
Feature vs labelFeatures are input variables. Labels are target outputs used for supervised learning.
Validation vs test setValidation supports model selection/tuning. Test data should provide an unbiased final estimate before deployment.
Model drift vs data driftData drift means input data distribution changes. Model drift/concept drift means the relationship between inputs and target outcome changes.
Explainability vs interpretabilityInterpretability is inherent understandability. Explainability uses techniques to explain complex models.
Automation vs augmentationAutomation replaces a task or decision. Augmentation supports a human decision maker. Human-in-the-loop needs roles, thresholds, and escalation paths.
Proof of concept vs pilot vs productionPOC proves feasibility. Pilot validates in limited real conditions. Production requires monitoring, controls, support, and ownership.

AI Use-Case Pattern Reference

PMI-CPMAI candidates should be able to classify AI opportunities and connect the pattern to data, risks, and success metrics.

AI patternUse when the problem is…Example outcomesUseful metricsKey risks
Predictive analytics and decision supportForecasting, scoring, ranking, or recommending a decisionChurn risk, demand forecast, loan risk, maintenance predictionRMSE, MAE, AUC, precision/recall, business liftBias, false positives/negatives, overreliance
RecognitionIdentifying objects, people, speech, text, images, or signalsImage classification, speech-to-text, document extractionAccuracy, F1, word error rate, detection ratePoor data quality, edge cases, demographic bias
Conversational and human interactionInteracting with users in natural language or multimodal channelsChatbot, service assistant, voice interfaceTask completion, containment, satisfaction, hallucination rateSafety, hallucination, escalation failure
HyperpersonalizationTailoring content, offers, recommendations, or experiencesProduct recommendation, dynamic pricing support, next-best actionConversion, lift, engagement, retentionPrivacy, filter bubbles, unfair targeting
Pattern and anomaly detectionFinding unusual behavior, clusters, outliers, or hidden structuresFraud signal, network anomaly, process deviationDetection rate, false alarm rate, precision, investigation yieldAlert fatigue, shifting baselines
Autonomous systemsActing with limited human intervention in dynamic environmentsRobotics, autonomous routing, adaptive controlSafety incidents, task success, latency, reliabilitySafety, accountability, fail-safe design
Goal-driven systemsOptimizing actions toward a goal under constraintsResource optimization, game agents, planning systemsReward, constraint violations, optimization valueMisaligned reward, unintended behavior

Role and Responsibility Matrix

RolePrimary accountabilityPMI-CPMAI exam signal
SponsorOwns business value, funding, strategic alignment, executive decisionsEscalate unresolved business tradeoffs here, not to the data scientist
Project manager / AI project managerIntegrates scope, schedule, risk, stakeholders, governance, and delivery lifecycleCoordinates uncertainty, decisions, dependencies, and transparency
Product owner / business ownerPrioritizes use cases, accepts outcomes, clarifies decision workflowsNeeded when requirements or value criteria are ambiguous
Data owner / data stewardAuthorizes access, defines meaning, quality expectations, and usage constraintsCritical for permissions, lineage, quality, and retention issues
Subject matter expertValidates business rules, labels, edge cases, and operational usefulnessReduces misunderstood context and labeling errors
Data engineerBuilds data pipelines, transformations, storage, and reproducibilityNeeded when data is fragmented, stale, or not production-ready
Data scientist / ML engineerExplores data, engineers features, trains, tunes, and evaluates modelsDoes not own business value alone
MLOps / platform engineerDeploys, monitors, automates, versions, and supports models in productionCritical when moving from experiment to reliable service
Security / privacy / complianceReviews controls for sensitive data, access, audit, model risk, and obligationsInvolve early, not after model completion
Risk / governance boardReviews responsible AI, model approval, exceptions, and ongoing controlsUsed for high-impact, regulated, sensitive, or material decisions
Change manager / adoption leadManages user readiness, communications, training, and resistanceEspecially important when AI changes work processes
Model ownerAccountable for model performance and control after go-livePrevents orphaned models after project closure

Business Case and Use-Case Selection

High-Yield Evaluation Criteria

CriterionAskGood evidenceWeak evidence
Business valueWhat decision, process, or outcome improves?Quantified benefit, cost avoidance, risk reduction, service improvement“We need AI” or “competitors use AI”
FeasibilityCan the model be built, integrated, and supported?Available skills, architecture, data access, realistic performance targetAssumption that technology alone solves the problem
Data readinessIs relevant, representative, permitted data available?Data profiling, lineage, owner approval, quality assessmentSpreadsheet sample with no provenance
Risk profileWhat could harm customers, employees, operations, or the organization?Risk register, impact analysis, governance pathRisk deferred until deployment
AdoptionWill users trust and use the output correctly?Human workflow design, explanation, training, feedback loop“Users will adopt it if it is accurate”
MeasurabilityCan success be measured objectively?Baseline, target metric, measurement planVague success definition
Notes and examples

Practical Prioritization Formula

Use scoring to compare candidate AI use cases. The exam may not require a specific formula, but it often rewards risk-aware prioritization.

\[ \text{Priority Score} = \frac{\text{Business Value} + \text{Strategic Alignment} + \text{Learning Value}} {\text{Complexity} + \text{Data Risk} + \text{Operational Risk}} \]

Interpretation: higher value and learning increase priority; high complexity and risk reduce priority unless justified by strategic importance.

AI Use Case Selection

A strong AI use case has:

  • A clear business problem or opportunity.
  • A measurable success criterion.
  • Available and usable data.
  • Stakeholder ownership.
  • A realistic path to deployment.
  • Acceptable risk and governance controls.
  • A feedback loop for improvement.

Use Case Screening Table

QuestionWhy It Matters
What decision, workflow, or outcome will improve?Prevents vague “AI transformation” goals
Who will use or be affected by the AI system?Identifies adoption, training, fairness, and human oversight needs
What data is required and who owns it?Surfaces access, quality, privacy, and governance constraints
What happens if the AI is wrong?Determines risk level, controls, and human review requirements
How will performance be measured?Connects technical metrics to business value
Can the solution be integrated into operations?Avoids successful prototypes that never create value

Data Management Cheat Sheet

Data conceptWhat to verifyCommon control
Data provenanceWhere data came from and how it was collectedLineage documentation, source approval
Data ownershipWho can authorize useData owner/steward approval
Data qualityCompleteness, accuracy, validity, consistency, uniqueness, timelinessProfiling, cleansing rules, quality thresholds
RepresentativenessWhether data reflects the target population and future useSampling review, bias analysis
Ground truthReliable target labels or outcomesLabeling guide, SME review, inter-rater checks
Labeling qualityConsistency and correctness of annotationsLabel audit, adjudication process
Sensitive dataPersonal, confidential, regulated, or proprietary dataMinimization, masking, access control
Feature engineeringTransforming raw data into useful predictorsFeature documentation, reproducible pipeline
Data leakageInformation in training that would not be available at prediction timeTime-aware split, feature review
Data versioningAbility to reproduce resultsDataset versions, pipeline hash, experiment tracking
Synthetic dataArtificially generated data used to supplement or protect real dataValidation against target distribution, risk review
RetentionHow long data and outputs are keptRetention schedule and deletion process
Notes and examples

Data Split Decision Table

ScenarioPreferWhy
Randomly distributed independent recordsRandom train/validation/test splitSimple and usually sufficient
Time-series or forecastingTime-based splitPrevents future data from leaking into training
Rare positive casesStratified splitPreserves class balance in each set
Same customer/patient/device appears multiple timesGrouped splitPrevents the same entity from appearing in both train and test
Small datasetCross-validation, with cautionImproves estimate stability but does not solve poor data quality
Production data differs from historical dataBack-testing and pilot monitoringTests real-world stability

Data Readiness Review

AI outcomes depend heavily on data. If a scenario has unclear data ownership, poor quality, incomplete labels, privacy restrictions, or biased samples, those issues must be addressed before relying on model results.

Data Quality Dimensions

DimensionReview Point
AccuracyDoes the data correctly represent reality?
CompletenessAre required fields and records present?
ConsistencyAre formats, definitions, and values aligned across sources?
TimelinessIs the data current enough for the use case?
RepresentativenessDoes it reflect the population or operating environment?
LineageCan the source, transformation, and use of data be traced?
Label qualityAre labels correct, consistent, and relevant?
AccessibilityCan the team lawfully and practically access the data?
PrivacyIs sensitive data protected and used appropriately?

Common Data Traps

  • Training data does not represent future production conditions.
  • Historical data contains past bias.
  • Labels are inconsistent across teams or regions.
  • Sensitive information is used without proper controls.
  • Data is available for prototyping but not for production.
  • Data definitions differ between business units.
  • Duplicate records inflate model performance.
  • Data leakage makes evaluation look better than real-world performance.

Data Leakage

Data leakage occurs when information unavailable at prediction time is used during training or evaluation. It produces misleadingly high performance.

Leakage ExampleWhy It Is a Problem
Including a field that is created after the target eventThe model learns future information
Randomly splitting time-series dataFuture patterns may leak into training
Duplicate customers in both train and test setsTest performance is inflated
Preprocessing before splitting dataTest data influences training transformations
Using target-derived featuresThe answer is indirectly encoded

Decision rule: if the model would not have access to the information at the time of real-world prediction, it should not be used as a feature.

Model Approach Selection

ApproachBest fitExamplesWatch for
Supervised learningLabeled examples exist and target outcome is knownClassification, regressionLabel quality, leakage, imbalance
Unsupervised learningNo labels; goal is grouping, anomaly, or structure discoveryClustering, anomaly detection, topic discoveryHarder validation, business interpretation
Semi-supervised learningLimited labels with larger unlabeled dataDocument classification, image labelingLabel propagation errors
Reinforcement learningAgent learns actions through rewards and penaltiesOptimization, robotics, simulationsSafety, reward misalignment, simulation gap
Deep learningLarge complex data such as images, speech, language, sequencesComputer vision, NLP, generative modelsCompute cost, explainability, data volume
Generative AI / foundation modelCreating or transforming text, code, images, audio, or multimodal contentSummarization, drafting, assistantsHallucination, prompt injection, IP, evaluation difficulty
Rules-based automationLogic is stable, explicit, and deterministicEligibility rules, routing logicNot adaptive; may be better than AI for simple rules
Human-in-the-loop AIDecision impact is high or context is complexMedical support, fraud investigation, hiring supportRole clarity, escalation, accountability
Notes and examples

Build, Buy, or Adapt Decision

OptionChoose whenAdvantagesRisks
Build custom modelNeed differentiated capability, control, special data, or integrationTailored performance and ownershipHigher cost, skills, support burden
Buy vendor AI solutionCommon use case, faster time to value, vendor support acceptableSpeed, packaged featuresLock-in, opaque model, data sharing risk
Adapt/fine-tune foundation modelNeed language or multimodal capability with domain adaptationStrong baseline, faster than building from scratchEvaluation, hallucination, privacy, model updates
Use API/prompting onlyLow customization, rapid experimentation, moderate riskFastest startLess control, variable behavior
Do not use AIRules or reporting solve the problem adequatelyLower risk and complexityMay not capture complex patterns

Evaluation Metrics Reference

Classification Metrics

MetricPlain formulaUse whenTrap
AccuracyCorrect predictions / all predictionsClasses are balanced and errors have similar costMisleading with imbalanced data
PrecisionTP / (TP + FP)False positives are costlyMay miss many actual positives
Recall / sensitivityTP / (TP + FN)False negatives are costlyMay create too many false alerts
SpecificityTN / (TN + FP)Correctly identifying negatives mattersOften ignored in screening use cases
F1 score2 × precision × recall / (precision + recall)Need balance between precision and recallHides business cost differences
AUC-ROCRanking quality across thresholdsComparing classifiers broadlyCan be misleading with severe imbalance
Confusion matrixTP, FP, TN, FN countsExplaining error types to stakeholdersDo not stop at one aggregate score
Notes and examples

Regression, Forecasting, and Ranking Metrics

MetricUse whenLower or higher is betterTrap
MAEAverage absolute error is understandableLowerTreats all errors linearly
RMSELarger errors should be penalized moreLowerSensitive to outliers
MAPEPercentage error is meaningfulLowerFails or distorts near zero values
R-squaredExplaining variance in targetHigherDoes not prove business usefulness
Lift / gainRanking improves targeting vs baselineHigherNeeds baseline comparison
Business KPIModel affects revenue, cost, cycle time, risk, serviceDependsMust be measured after deployment

Generative AI Evaluation

Evaluation areaAskExample evidence
FactualityAre outputs correct and grounded?Human review, retrieval grounding tests, citation checks
RelevanceDoes output answer the user’s task?Rubric scoring, task completion
SafetyDoes it avoid harmful, prohibited, or sensitive output?Red-team tests, policy checks
HallucinationDoes it invent unsupported facts?Grounded evaluation, refusal tests
Toxicity and biasAre outputs discriminatory or harmful?Bias test suites, reviewer audits
RobustnessDoes it resist prompt injection and adversarial inputs?Attack simulations, guardrail tests
ConsistencyAre similar prompts handled consistently?Regression test set
Cost and latencyCan it meet operational constraints?Token cost, response time, throughput tests

Classification Metrics

MetricPlain-Language MeaningBest Used When
AccuracyShare of total predictions that are correctClasses are balanced and errors have similar cost
PrecisionOf predicted positives, how many were truly positiveFalse positives are costly
Recall / sensitivityOf actual positives, how many were foundFalse negatives are costly
SpecificityOf actual negatives, how many were correctly rejectedCorrectly excluding negatives matters
F1 scoreBalance of precision and recallNeed one metric for uneven classes
ROC-AUCAbility to rank positives above negativesComparing classifiers across thresholds
Confusion matrixCounts true positives, false positives, true negatives, false negativesUnderstanding error types

Regression Metrics

MetricPlain-Language MeaningWatch For
MAEAverage absolute errorEasier to explain to stakeholders
MSEAverage squared errorPenalizes large errors heavily
RMSESquare root of MSESame unit as target; sensitive to outliers
MAPEPercentage errorCan fail when actual values are near zero

Generative AI Evaluation

Generative AI evaluation often requires multiple measures because outputs can be fluent but wrong.

Evaluation AreaWhat to Check
FactualityIs the answer true?
GroundednessIs the answer supported by approved sources?
RelevanceDoes it answer the actual user request?
CompletenessDoes it include required information?
SafetyDoes it avoid harmful, toxic, or prohibited content?
PrivacyDoes it avoid exposing sensitive data?
ConsistencyDoes it produce reliable results across similar prompts?
Human reviewAre high-risk outputs reviewed by qualified people?

Generative AI Management Review

Generative AI introduces specific risks because outputs may be plausible, variable, and difficult to verify.

Generative AI Decision Table

NeedLikely ApproachWatch For
General drafting or brainstormingPrompting a foundation modelReview and accuracy controls
Answering from internal documentsRetrieval-augmented generationSource quality, grounding, permissions
Domain-specific style or task adaptationFine-tuning or prompt engineeringData quality, overfitting, cost
High-risk factual responsesRAG plus human reviewHallucination, outdated sources
Structured extractionModel plus validation rulesInconsistent outputs
Customer-facing chatbotGuardrails, escalation, monitoringSafety, privacy, brand risk

Generative AI Risks

RiskMitigation
HallucinationGrounding, retrieval, citations, human review
Prompt injectionInput filtering, tool restrictions, security testing
Sensitive data exposureData loss prevention, access control, redaction
Copyright / IP concernApproved content sources, legal review
Toxic or unsafe outputSafety filters, evaluation, escalation
OverrelianceUser training, confidence limits, review workflows
Inconsistent outputTemplates, structured prompts, deterministic settings where possible
Unauthorized actionTool permissions, approval gates, audit logs

RAG vs. Fine-Tuning

ApproachUse WhenNot Best For
RAGNeed answers grounded in current or proprietary documentsTeaching the model broad new behavior by itself
Fine-tuningNeed model behavior, tone, format, or task adaptationKeeping rapidly changing knowledge current
Prompt engineeringNeed low-cost, flexible task guidanceComplex governance or high-risk reliability needs alone
Rules / workflow automationNeed deterministic behaviorOpen-ended language understanding or generation

Risk, Governance, and Responsible AI Controls

Risk exposure is often summarized as:

\[ \text{Risk Exposure} = \text{Probability} \times \text{Impact} \]

For AI projects, include both project delivery risk and model behavior risk.

RiskWarning signsPM actionEvidence/artifact
Bias or unfair impactUnequal errors across groups, nonrepresentative dataPerform bias assessment, involve SMEs/governance, adjust data/model/processFairness report, mitigation log
Privacy misuseSensitive data used without clear purpose or approvalApply minimization, masking, access controls, privacy reviewData use approval, privacy assessment
Security riskExposed model endpoints, weak access, prompt injectionInvolve security early, threat model, test controlsSecurity review, penetration/red-team results
HallucinationModel generates plausible unsupported contentGround outputs, add human review, define refusal/escalationEvaluation report, guardrail design
Explainability gapStakeholders cannot understand or challenge decisionsSelect interpretable model or explainability technique; document limitationsModel card, explanation method
Model driftProduction performance degrades over timeMonitor data/performance, set retraining triggersDrift dashboard, retraining plan
Data driftIncoming data differs from training dataMonitor input distributions and qualityData monitoring report
Automation biasUsers over-trust model outputTrain users, design confidence indicators, require review for high impactAdoption plan, UI decision controls
Accountability gapNo owner after deploymentAssign model owner and support process before go-liveRACI, operations handbook
Third-party/vendor riskBlack-box model, unclear data use, dependency riskDue diligence, contract review, exit planVendor assessment, SLA/support terms
IP/copyright riskUnclear rights for training data or generated contentLegal/IP review and content policyUsage policy, approval record
Safety riskAI action may cause physical, financial, or human harmFail-safe design, simulation, staged rollout, human overrideSafety case, rollback plan

Governance Artifacts to Recognize

ArtifactPurposeCreated/updated when
AI use-case canvasCaptures business problem, users, data, value, and risksInitiation and refinement
Data inventoryLists sources, owners, sensitivity, quality, and access statusData understanding
Data sheet / dataset documentationDocuments dataset origin, contents, limitations, and intended useBefore model development and reuse
Model cardSummarizes model purpose, training data, performance, limitations, ethical considerationsBefore approval and deployment
Experiment logTracks runs, parameters, datasets, and resultsDuring model development
Risk registerTracks project, data, model, governance, and operational risksThroughout lifecycle
Decision logRecords major tradeoffs and approvalsWhen selecting use case, data, model, threshold, deployment
Monitoring planDefines metrics, thresholds, alerts, owners, and retraining triggersBefore production
Change management planPrepares users, process changes, training, communicationsBefore pilot and rollout
Incident response planDefines what to do when AI fails, drifts, or harmsBefore go-live
Notes and examples

AI Governance Artifacts

You may see scenario questions where the best answer is to add governance, documentation, or monitoring rather than to tune the model again.

ArtifactPurpose
AI use case inventoryTracks where AI is being used
Risk assessmentIdentifies severity, likelihood, controls, and owners
Data sheet / data documentationDescribes data source, quality, limits, and permitted uses
Model cardSummarizes model purpose, performance, limitations, and intended use
Evaluation reportRecords test design, results, assumptions, and acceptance criteria
Human oversight planDefines review, override, escalation, and accountability
Monitoring planDefines metrics, thresholds, alerts, drift checks, and retraining triggers
Incident response planDefines response to harmful, incorrect, or unexpected AI behavior
Change logTracks model, data, prompt, feature, and configuration changes

Project Management Decision Tables

What Should the Manager Do Next?

SituationBest next actionWhy
Stakeholders ask for “an AI solution” but cannot define the decision or valueFacilitate business understanding and define measurable outcomesAI selection should follow problem definition
Data scientists want to start modeling before data access is approvedResolve data ownership, permission, and governancePrevents rework and compliance issues
Model accuracy is high but false negatives are costlyReview confusion matrix and adjust metric/threshold with business ownersBusiness impact depends on error type
Test performance is much worse than training performanceInvestigate overfitting, data leakage, split strategy, and data representativenessTechnical score must generalize
Sponsor wants full rollout after a successful lab demoRecommend pilot or staged deployment with monitoring and rollbackLab success does not prove operational readiness
Users distrust model recommendationsAdd explanation, training, feedback channels, and human review designAdoption risk is a project risk
Model behavior changes after deploymentTrigger monitoring response: assess drift, impact, rollback/retrain if neededAI requires lifecycle management
Compliance team raises concerns latePause affected work, assess impact, update risk and governance planGovernance cannot be bypassed for schedule
Vendor model is opaqueConduct vendor risk review and define transparency, audit, and performance requirementsBlack-box risk must be managed
Product owner keeps changing target metricReconfirm business objective and update change control/backlog priorityPrevents uncontrolled experimentation
Notes and examples

Predictive, Agile, or Hybrid Delivery

Delivery approachBest fitAI-specific tailoring
PredictiveStable compliance, procurement, infrastructure, or deployment constraintsStill allow experimentation within planned stages
AgileUncertain model approach, evolving requirements, iterative discoveryUse spikes, experiments, model backlog, frequent stakeholder review
HybridMost AI initiatives: governed phases plus iterative modelingStage gates for data/model/governance; agile iterations inside phases
Kanban/flowOperational model monitoring, incident handling, labeling queuesManage WIP, service levels, and continuous improvement
Experimental POCFeasibility unknownTimebox, define learning goals, avoid production promises

Change Control in AI Projects

Change typeExamplesManage through
Business objective changeNew target outcome, different user groupSponsor/product owner approval; update business case
Data changeNew source, changed definition, quality issueData owner approval; update lineage and tests
Feature changeAdd/remove predictorsExperiment log; validation and leakage review
Model changeNew algorithm, retraining, fine-tuningModel governance, evaluation, versioning
Threshold changeAdjust fraud alert cutoff or risk scoreBusiness owner decision; document error tradeoff
Deployment changeNew integration, scaling, endpoint, user workflowRelease plan, security review, rollback
Governance exceptionUse of sensitive data, reduced explainability, high-risk automationFormal risk acceptance or rejection

MLOps and Operationalization Workflow

    flowchart LR
	    A[Business objective] --> B[Data access and profiling]
	    B --> C[Data preparation and feature pipeline]
	    C --> D[Model training and experiments]
	    D --> E[Validation and risk review]
	    E --> F{Go / no-go}
	    F -- No --> B
	    F -- Yes --> G[Deploy to pilot or production]
	    G --> H[Monitor performance, drift, cost, safety]
	    H --> I{Trigger?}
	    I -- Drift or incident --> J[Rollback, retrain, or remediate]
	    J --> E
	    I -- Stable --> H
Notes and examples

Production Readiness Checklist

AreaMust be true before go-live
OwnershipSponsor, product owner, model owner, support owner identified
DataSource permissions, quality checks, lineage, and versioning established
ModelPerformance, limitations, bias, explainability, and thresholds documented
IntegrationInterfaces, latency, reliability, and fallback paths tested
SecurityAccess controls, endpoint protection, secrets management, and testing complete
MonitoringMetrics, alerts, drift checks, cost tracking, and review cadence defined
Human processReview, override, escalation, and feedback loops designed
Change controlRetraining, model updates, and rollback procedures defined
AuditabilityDecisions, versions, approvals, and evidence retained as required by the organization
AdoptionTraining, communications, and user readiness completed

Thresholds, Tradeoffs, and Human-in-the-Loop

Decision pointIf you increase thresholdIf you decrease thresholdManager focus
Fraud alert thresholdFewer alerts, higher precision, more missed fraudMore alerts, higher recall, more false alarmsBalance investigation capacity and loss tolerance
Medical screening sensitivityFewer false positives but more missed casesMore detected cases but more unnecessary follow-upAlign with clinical risk and governance
Content moderation thresholdLess overblocking, more harmful content may passMore blocking, more legitimate content removedAlign with policy, appeal process, safety
Credit/risk cutoffFewer approvals, lower default riskMore approvals, higher default riskEnsure fairness, explainability, and business policy
Human review triggerLess human workload, more automation riskMore oversight, slower processMatch review level to impact and uncertainty

Quality and Acceptance Criteria

AI Quality Dimensions

DimensionMeaningEvidence
Technical performanceModel meets metric targets on appropriate dataEvaluation results, confidence intervals where useful
Business performanceModel improves real KPI or decision qualityPilot results, A/B test, operational KPI
RobustnessHandles edge cases, noise, and adversarial conditionsStress tests, red-team tests
FairnessPerformance and impact are acceptable across relevant groupsBias analysis, mitigation actions
ExplainabilityUsers and reviewers can understand enough to trust and challenge outputExplanations, documentation, training
ReliabilityService works consistently under expected loadUptime, latency, error rate
MaintainabilityModel and pipeline can be updated and reproducedVersioning, automation, documentation
Compliance/governanceMeets organizational policies and approval requirementsReview records, audit trail
Notes and examples

Definition of Done for AI Work

Work itemDone means…
Data source onboardedApproved, documented, profiled, secured, and accessible
Labeling completedGuide followed, quality checked, conflicts resolved
Model candidate trainedReproducible run with dataset/model version and metrics
Model selectedCompared to baseline and alternatives; tradeoffs documented
Risk review completedMaterial risks assessed, mitigations accepted or required
Pilot completedReal users/workflow tested; feedback and metrics reviewed
Production release completedMonitoring, support, rollback, and ownership active

Common Exam Traps

TrapBetter answer pattern
Choosing an algorithm firstDefine problem, value, data, and constraints first
Equating high accuracy with successEvaluate business impact, error costs, fairness, and adoption
Ignoring data ownershipConfirm access, permissions, lineage, and stewardship
Treating AI as a one-time deliverablePlan monitoring, retraining, governance, and lifecycle ownership
Deferring ethics and complianceIntegrate responsible AI and risk controls early
Assuming more data is always betterPrefer relevant, representative, high-quality, permitted data
Deploying directly from POCUse pilot, production readiness checks, rollback, and monitoring
Letting technical teams own all decisionsBusiness owners decide value and error tradeoffs; governance decides risk acceptance
Using a black-box model without justificationMatch explainability to impact, risk, and stakeholder need
Ignoring user workflowDesign how humans receive, challenge, override, and improve AI output

Rapid Review Checklist

Before the exam, be able to answer these quickly:

  • What business decision or process does the AI system improve?
  • Which AI pattern fits the use case?
  • What data is needed, who owns it, and is it fit for purpose?
  • What could make the model unfair, unsafe, insecure, or noncompliant?
  • Which metric matches the business cost of errors?
  • What is the baseline, and how does the model improve on it?
  • How will stakeholders validate, accept, and adopt the solution?
  • What governance artifacts are needed before deployment?
  • Who owns the model after project closure?
  • How will production performance, drift, incidents, and retraining be managed?
Notes and examples

Final Pre-Practice Checklist

Before starting a mock exam, make sure you can quickly answer:

  • When is AI appropriate, and when is a simpler solution better?
  • What makes data ready or not ready for AI?
  • How do you connect model metrics to business outcomes?
  • What are the main risks of deploying AI into production?
  • How do fairness, explainability, privacy, and accountability affect project decisions?
  • What is the difference between a prototype and an operational AI system?
  • How should generative AI be evaluated and controlled?
  • What does monitoring need to detect after deployment?
  • Who must be involved in AI governance and delivery?

Core Mental Model for the Exam

AI projects are not just software projects with a model added. The key difference is that project outcomes depend on data quality, model behavior, uncertainty, continuous evaluation, and responsible use.

A practical AI initiative usually moves through this logic:

    flowchart LR
	    A[Business problem] --> B[Use case and value hypothesis]
	    B --> C[Data availability and readiness]
	    C --> D[Model or AI solution approach]
	    D --> E[Evaluation against success criteria]
	    E --> F[Deployment and integration]
	    F --> G[Monitoring, governance, and improvement]
	    G --> C

High-yield exam principle:

A good AI manager does not start with the algorithm. A good AI manager starts with the business problem, validates that AI is appropriate, confirms data readiness, manages uncertainty, governs risk, and measures value after deployment.

High-Yield Review Map

AreaWhat to Know QuicklyExam Trap
AI opportunity selectionIdentify business value, feasibility, data availability, stakeholder impact, and measurable outcomesChoosing AI because it is fashionable rather than because it solves a defined problem
Data readinessData quality, completeness, labeling, representativeness, lineage, access, privacy, and governanceAssuming a model can compensate for poor or unavailable data
AI lifecycleIterative discovery, experimentation, evaluation, deployment, monitoring, and retrainingTreating AI delivery as a one-time linear build
Model evaluationMetrics must match the business objective and risk profileOptimizing accuracy when false positives or false negatives matter more
Responsible AIFairness, transparency, accountability, privacy, safety, security, and human oversightTreating ethics as a final compliance checklist
MLOps / operationsVersioning, monitoring, drift detection, rollback, retraining, and incident responseDeclaring success at deployment instead of monitoring real-world performance
StakeholdersBusiness owners, users, data owners, SMEs, legal, risk, security, IT, data scientists, and operationsEngaging only technical teams
Generative AIHallucination, grounding, prompt design, RAG, evaluation, IP, data leakage, and guardrailsAssuming fluent output means reliable output

AI Project Management: The Big Differences

Traditional Software vs. AI Initiatives

DimensionTraditional SoftwareAI / ML Initiative
Primary logicRules and deterministic behaviorProbabilistic behavior learned from data
Main uncertaintyRequirements, integration, delivery capacityData quality, model performance, bias, drift, user trust
TestingExpected outputs for defined inputsStatistical evaluation across datasets and scenarios
Success criteriaFeatures delivered, defects controlled, user acceptanceBusiness outcome, model performance, risk controls, operational stability
Change over timeCode changes when modifiedModel performance may degrade as data changes
Governance needSecurity, quality, complianceSecurity plus fairness, explainability, privacy, model risk, monitoring

Key Decision Rule

If the problem can be solved with simple rules, workflow automation, or conventional analytics, do not assume AI is the best answer. AI is most appropriate when the task involves prediction, classification, pattern detection, natural language, perception, recommendation, or generation and when sufficient data and governance are available.

Notes and examples

Common Scenario Decision Rules

If the Scenario Says…Prefer This Type of Answer
“The model performs well in testing but poorly in production”Investigate drift, data mismatch, monitoring, and deployment conditions
“Stakeholders disagree about success”Clarify business objectives and acceptance criteria
“Data quality is unknown”Conduct data assessment before full development
“The AI output may affect people significantly”Add risk review, human oversight, explainability, and appeal path
“A vendor model is high-performing but opaque”Evaluate explainability, risk, contractual rights, and monitoring
“A team wants to automate immediately”Confirm risk level, human review needs, and operational readiness
“A prototype works in a lab”Validate production data, integration, scalability, and support
“Accuracy is high but minority group errors are high”Investigate fairness and subgroup performance
“Generative AI gives confident wrong answers”Add grounding, evaluation, guardrails, and human review
“Model performance is declining”Check data drift, concept drift, upstream changes, and retraining criteria

Business Value and Success Metrics

AI projects should link technical performance to business value. The exam may present scenarios where a technically impressive model is not the best project choice.

Common Value Measures

GoalPossible Business MetricPossible AI Metric
Reduce fraudFraud loss reduction, investigation efficiencyPrecision, recall, false positive rate
Improve customer supportResolution time, satisfaction, cost per caseAccuracy, containment rate, hallucination rate
Forecast demandInventory cost, stockout rate, forecast errorMAE, RMSE, MAPE
Detect defectsDefect escape rate, inspection throughputRecall, precision, F1 score
Recommend productsConversion, revenue per user, retentionClick-through rate, ranking quality
Generate contentProductivity, quality review pass rateHuman evaluation score, groundedness, toxicity rate
Notes and examples

Candidate Trap

Do not select a model metric without understanding business consequences. A medical screening, fraud detection, or safety use case may require high recall even if precision falls. A customer-facing recommendation system may care about conversion, relevance, and user trust more than raw model accuracy.

Common Candidate Mistakes

  1. Starting with the technology instead of the business problem. The exam often rewards defining value, feasibility, and governance before selecting a tool.

  2. Confusing prototype success with production readiness. A proof of concept does not prove operational scalability, monitoring, security, or adoption.

  3. Choosing accuracy as the default metric. Match metrics to the cost of errors and business objective.

  4. Ignoring data governance. Data access, privacy, lineage, and quality are core AI management issues.

  5. Treating bias as only a technical issue. Bias can originate in history, process, measurement, labels, sampling, and deployment.

  6. Assuming more data always improves the model. More low-quality, biased, irrelevant, or unauthorized data can make the system worse.

  7. Assuming a complex model is best. Simpler models may be more explainable, cheaper, safer, and easier to govern.

  8. Forgetting monitoring after deployment. AI performance can degrade without code changes.

  9. Overlooking human adoption. AI initiatives fail when users do not trust, understand, or integrate the system into work.

  10. Confusing generative fluency with correctness. Generative AI can sound authoritative while being inaccurate or unsupported.

AI, Machine Learning, and Generative AI Basics

Core Terms

TermQuick Meaning
Artificial intelligenceBroad field of systems performing tasks associated with human intelligence
Machine learningSystems that learn patterns from data rather than relying only on explicit rules
Deep learningMachine learning using neural networks with many layers
Supervised learningLearns from labeled examples
Unsupervised learningFinds patterns or groupings without labeled outcomes
Reinforcement learningLearns actions through rewards and penalties
Generative AICreates new content such as text, images, code, audio, or summaries
Foundation modelLarge pretrained model adaptable to many tasks
Large language modelModel designed to process and generate language
PromptInput or instruction given to a generative AI model
RAGRetrieval-augmented generation; grounds responses using retrieved external content
Fine-tuningAdditional training to adapt a model for a task or domain
Notes and examples

Algorithm-Level Awareness

You generally need to recognize which technique fits which problem.

Problem TypeTypical Approach
Predict a numberRegression
Predict a categoryClassification
Group similar recordsClustering
Detect unusual behaviorAnomaly detection
Recommend itemsRecommendation systems
Extract meaning from textNatural language processing
Identify objects in imagesComputer vision
Generate text or summariesGenerative AI / LLM
Optimize sequential decisionsReinforcement learning

Overfitting, Underfitting, and Generalization

ConceptMeaningManagement Response
UnderfittingModel is too simple and performs poorly on training and test dataImprove features, model choice, or data quality
OverfittingModel memorizes training data and performs poorly on new dataUse validation, regularization, simpler model, more data
GeneralizationModel performs well on unseen dataUse proper train/validation/test design
BaselineSimple reference model or current processCompare improvements honestly
Holdout test setData reserved for final evaluationProtect against tuning to test data

Candidate trap: the best model is not automatically the most complex model. Simpler, explainable, stable, and governable models may be preferable, especially in regulated, safety-critical, or stakeholder-sensitive contexts.

Bias, Fairness, and Responsible AI

Responsible AI should be built into the lifecycle, not added at the end.

Responsible AI Principles

PrinciplePractical Meaning
FairnessAvoid unjustified disparate impact or discriminatory outcomes
TransparencyMake system purpose, limitations, and behavior understandable
ExplainabilityProvide reasons for outputs where appropriate
AccountabilityAssign ownership for design, deployment, monitoring, and remediation
PrivacyProtect personal and sensitive information
SecurityDefend models, data, and pipelines from misuse or attack
SafetyPrevent unacceptable harm to people, organizations, or society
Human oversightKeep humans involved where risk, judgment, or accountability requires it
ReliabilityEnsure consistent performance under expected conditions
ContestabilityAllow affected users to challenge or appeal decisions when appropriate
Notes and examples

Bias Sources

SourceExample
Historical biasPast decisions reflect unfair treatment
Sampling biasTraining data excludes certain groups
Measurement biasVariables measure outcomes differently across populations
Label biasLabels reflect subjective or inconsistent judgments
Deployment biasUsers apply the AI system differently than intended
Feedback-loop biasModel outputs influence future data in a self-reinforcing way

Fairness Trap

Removing protected attributes from data does not automatically remove bias. Proxy variables such as location, income, education, language, or device type may still encode sensitive patterns.

AI Risk Management

AI risk depends on both likelihood and impact. High-risk use cases need stronger controls, documentation, testing, monitoring, and escalation.

Risk Review Table

Risk TypeExamplesCommon Controls
Data riskPoor quality, privacy exposure, biased samplesData governance, quality checks, access controls
Model riskInaccurate, unstable, biased, unexplainable modelValidation, documentation, monitoring, independent review
Operational riskFailed integration, poor adoption, unreliable workflowPilot, change management, rollback plan
Security riskPrompt injection, model theft, data poisoningSecurity testing, input filtering, access control
Compliance riskImproper use of sensitive data, inadequate recordsLegal review, audit trail, policy alignment
Reputational riskHarmful outputs, unfair treatment, public failureHuman oversight, communication plan, guardrails
Vendor riskLock-in, opaque model behavior, data misuseContract review, due diligence, exit plan
Notes and examples

Risk-Based Decision Rule

The higher the impact of a wrong AI output, the more the project needs:

  • Human review.
  • Explainability.
  • Traceability.
  • Testing across edge cases.
  • Monitoring after deployment.
  • Formal approval and escalation paths.
  • Clear accountability.

AI Lifecycle Review

A practical AI lifecycle includes both management and technical work.

PhaseKey QuestionsCommon Deliverables
Identify opportunityWhat problem are we solving and why AI?Use case statement, value hypothesis, stakeholder map
Assess feasibilityIs data available, lawful, useful, and representative?Data assessment, risk screen, feasibility decision
Prepare dataCan data be cleaned, labeled, governed, and accessed?Data pipeline, labeling approach, quality checks
Develop model / solutionWhat approach best fits the objective and constraints?Baseline, model experiments, prompt/RAG design
EvaluateDoes it meet technical, business, risk, and user criteria?Validation report, acceptance decision
DeployCan it be integrated safely into workflow?Deployment plan, training, controls, rollback
Operate and monitorIs it performing as expected over time?Monitoring dashboard, drift alerts, incident process
Improve or retireShould the system be retrained, changed, scaled, or stopped?Retraining decision, change record, retirement plan

Project Roles and Stakeholders

AI initiatives are cross-functional. A common exam trap is choosing a technically correct answer that ignores stakeholder ownership, governance, or operational adoption.

RoleTypical Contribution
Executive sponsorStrategic direction, funding, prioritization
Product owner / business ownerBusiness problem, value definition, user needs
Project manager / AI managerCoordination, risk, schedule, scope, stakeholders
Data scientistModeling, experimentation, evaluation
Data engineerData pipelines, integration, data quality
Machine learning engineerProduction model implementation and automation
Domain SMEContext, labels, edge cases, validation
Legal / compliancePolicy, contractual, privacy, regulatory review
SecurityThreat modeling, access controls, vulnerability management
Operations / supportMonitoring, incident handling, user support
End usersWorkflow fit, usability, feedback, adoption

Scope, Schedule, and Estimation in AI Projects

AI project estimates are uncertain because feasibility depends on data and model performance. Manage this with discovery, proof of concept, staged decisions, and explicit assumptions.

Good Practices

  • Separate discovery from full-scale delivery.
  • Use timeboxed experiments.
  • Define minimum acceptable model and business performance.
  • Keep a baseline comparison.
  • Document assumptions about data, access, quality, and integration.
  • Use go/no-go checkpoints.
  • Avoid promising exact model performance before data assessment.

Poor Practices

  • Committing to production dates before data is accessible.
  • Treating proof of concept success as production readiness.
  • Ignoring model monitoring work in the schedule.
  • Excluding legal, security, data owners, or operations until late.
  • Defining “done” as model training rather than operational value.

Agile, Iterative, and Stage-Gate Thinking

AI work benefits from iteration, but not uncontrolled experimentation. Strong AI management combines learning cycles with governance checkpoints.

SituationBest Management Approach
Uncertain data qualityDiscovery spike or data assessment
Uncertain model feasibilityTimeboxed proof of concept
High-risk decision automationFormal risk review and human oversight
Production deploymentControlled release with monitoring and rollback
Model performance degradationDrift analysis, retraining decision, incident review
Expanding to new population or regionRevalidate data, fairness, performance, and controls

MLOps and Operational Monitoring

MLOps is the discipline of reliably deploying, monitoring, updating, and governing models in production.

MLOps Capabilities

CapabilityWhy It Matters
Version controlTracks code, data, model, prompt, and configuration changes
Model registryMaintains approved models and metadata
Automated testingCatches defects before deployment
CI/CD or controlled releaseSupports repeatable deployment
MonitoringDetects performance, data, and operational issues
Drift detectionIdentifies when production data changes
Retraining processUpdates models safely and consistently
RollbackRestores prior version if the new one fails
Audit trailSupports accountability and review
Notes and examples

Drift Types

Drift TypeMeaning
Data driftInput data distribution changes
Concept driftRelationship between inputs and target changes
Prediction driftOutput distribution changes
Performance driftBusiness or model metrics degrade

Candidate trap: retraining is not always the first answer. First determine whether the issue is data quality, upstream system change, user behavior change, model drift, integration failure, or metric definition change.

Deployment and Change Management

A model that works in a notebook may fail in operations. Deployment planning should address workflow, people, systems, risk, and support.

Deployment Checklist

  • Integration with existing systems.
  • User training and communication.
  • Defined human review steps.
  • Access controls and security testing.
  • Monitoring dashboard and alert thresholds.
  • Rollback or fallback process.
  • Support ownership.
  • Incident escalation.
  • Documentation for users and operators.
  • Post-deployment review.

Adoption Trap

Users may reject an AI system if they do not trust it, understand it, or see how it improves their work. Change management is part of AI delivery, not an optional extra.

Build vs. Buy vs. Partner

AI solutions may be built internally, purchased from vendors, or delivered through partners. The best option depends on strategic value, data sensitivity, expertise, time, cost, control, and risk.

OptionAdvantagesRisks
Build internallyControl, customization, internal knowledgeRequires talent, time, MLOps maturity
Buy vendor solutionFaster deployment, packaged capabilitiesLock-in, opacity, data rights concerns
Use API / platformRapid experimentation, scalable capabilitiesCost variability, dependency, privacy
Partner / outsourceAccess to expertiseKnowledge transfer and accountability issues

Vendor Evaluation Questions

  • What data is sent to the vendor?
  • Who owns inputs, outputs, derived data, and model improvements?
  • Can the vendor explain model limitations?
  • How is performance validated?
  • What security and privacy controls exist?
  • Are logs retained, and who can access them?
  • What happens if the vendor changes the model?
  • Is there an exit or migration plan?
  • Can the organization audit or review relevant controls?

Security Review for AI Projects

AI systems introduce conventional cybersecurity issues plus AI-specific threats.

ThreatMeaningControl Examples
Data poisoningTraining data is manipulatedData validation, trusted sources, anomaly checks
Model inversionSensitive training information is inferredPrivacy controls, minimization, testing
Model extractionAttacker copies model behaviorRate limits, monitoring, access controls
Adversarial examplesInputs crafted to fool the modelRobust testing, detection, fallback
Prompt injectionMalicious instructions override intended behaviorInput sanitization, tool restrictions, guardrails
Data exfiltrationModel reveals confidential dataAccess control, redaction, output filtering
Unauthorized tool useAI agent takes harmful actionsPermission boundaries, approvals, audit logs

Candidate trap: security is not only an IT issue after deployment. It should be considered during data acquisition, model design, vendor selection, testing, deployment, and monitoring.

Explainability and Transparency

Explainability needs depend on the use case. A low-risk recommendation may need less explanation than a model used for credit, employment, healthcare, safety, or disciplinary decisions.

ConceptMeaning
InterpretabilityThe model is understandable by design
ExplainabilityMethods help explain model outputs or behavior
TransparencyUsers and stakeholders understand purpose, limits, and use
Black-box modelInternal logic is difficult to understand
Local explanationExplains one prediction
Global explanationExplains overall model behavior

Decision rule: when decisions significantly affect people, increase explainability, documentation, review, appeal, and accountability.

Human-in-the-Loop Design

Human involvement should be purposeful. It is not enough to say “a human is involved” if the human lacks time, expertise, authority, or information to intervene.

PatternDescriptionGood Use
Human-in-the-loopHuman reviews before final actionHigh-risk or uncertain outputs
Human-on-the-loopHuman monitors automated systemLower-risk operations with alerts
Human-out-of-the-loopFully automated actionLow-risk, well-validated, controlled settings
Human overrideHuman can reverse or stop AI actionSafety, compliance, customer impact
EscalationAI routes uncertain cases to expertsComplex or high-impact decisions

Human Oversight Trap

A human review step can become ineffective if reviewers rubber-stamp outputs due to automation bias, time pressure, poor explanations, or unclear accountability.

Rapid Review Tables

AI Project “Go / No-Go” Signals

Go SignalNo-Go or Pause Signal
Clear business ownerNo accountable sponsor
Defined outcome and metricVague innovation goal
Data is accessible and governedData ownership unclear
Risk level understoodHarm impact unknown
Feasible deployment pathPrototype only, no integration plan
Monitoring and support plannedNo post-deployment owner
Stakeholders engagedUsers excluded
Controls match riskEthics and security deferred
Notes and examples

Technical vs. Management Response

ProblemTechnical ResponseManagement Response
Poor model performanceImprove features, data, model, tuningRevisit feasibility, timeline, success criteria
Biased outcomesTest subgroups, adjust data/modelDefine fairness goals, governance, review process
HallucinationsRAG, prompts, evaluationLimit use case, add human review, communicate risks
DriftMonitor, retrain, recalibrateDefine thresholds, ownership, incident process
Poor adoptionImprove UX, workflow integrationChange management, training, stakeholder alignment
Vendor opacityModel documentation requestRisk assessment, contract terms, exit plan

Put the review into practice