Free Google Cloud Professional ML Engineer Practice Questions: Monitoring AI Solutions

Topic snapshot

Sample questions

These are original IT Mastery practice questions aligned to this topic area. They are not official Google Cloud questions, copied live-exam content, or exam dumps. Use them to preview question style and explanation depth before continuing with topic drills, mixed sets, and timed mocks in IT Mastery.

Question 1

Topic: Monitoring AI Solutions

A retail lender has a custom tabular risk model deployed on Agent Platform Inference. Business and compliance stakeholders need to understand which input factors drive individual high-risk predictions and review aggregate model behavior during monthly responsible AI reviews. Which setup best meets these requirements?

Options:

• A. Enable model explainability and collect feature attributions from predictions

• B. Enable only data drift alerts in Model Monitoring

• C. Move the model to a GPU endpoint for faster inference

• D. Store all input features in Agent Platform Feature Store

Best answer: A

Explanation: Model explainability on Agent Platform is used when stakeholders need to understand model behavior and prediction factors. For a tabular model, feature attributions can show which inputs contributed most to a specific prediction and can be aggregated for responsible AI review. This supports compliance and business interpretation without changing the model architecture or moving the serving workload.

Drift monitoring and feature serving are useful operational capabilities, but they do not explain why a prediction was made. The key takeaway is to configure explainability at the model/serving layer where predictions are produced.

• Drift-only monitoring can detect input distribution changes, but it does not identify prediction-driving factors.
• Feature Store inspection exposes feature values, not their contribution to the model output.
• GPU serving may improve latency for some models, but it does not add interpretability.

Question 2

Topic: Monitoring AI Solutions

A team is running a 10% canary of a new recommendation model on Agent Platform Inference. They must decide whether to promote it, roll it back, or start retraining. Labels arrive after 24 hours, and recent traffic has short-lived spikes. What workflow setup best supports an evidence-based decision?

Options:

• A. Promote after offline validation, then monitor only the promoted model

• B. Gate rollout on canary-vs-baseline monitoring and delayed quality labels

• C. Trigger retraining immediately on any input drift alert

• D. Rollback only when endpoint latency exceeds the SLO

Best answer: B

Explanation: A production decision gate should use evidence from the right lifecycle stage. For a canary, compare the candidate model against the current baseline using serving metrics, business or model-quality metrics when delayed labels arrive, and Model Monitoring signals such as drift or skew. Short traffic spikes should not automatically cause retraining; retraining is appropriate when quality degradation or persistent distribution change is confirmed. Rollback is appropriate when the canary shows a clear regression, especially in reliability or quality, before broader promotion.

• Offline-only promotion misses production behavior and delayed label evidence from the canary.
• Immediate retraining treats any drift alert as root cause without confirming persistence or quality impact.
• Latency-only rollback ignores model-quality regressions, drift, and baseline comparison.

Question 3

Topic: Monitoring AI Solutions

A company is deploying a Gemini-based support assistant that can use internal knowledge articles. The security team needs a protection and monitoring setup that blocks accidental leakage of PII or secrets and also detects deliberate prompt-injection or jailbreak attempts to exfiltrate internal content. Which setup best meets these requirements?

Options:

• A. Add Regex checks only to the training data pipeline before fine-tuning.

• B. Use Model Armor on prompts and responses with sensitive-data filters, safety filters, and prompt-injection/jailbreak detection.

• C. Deploy the assistant on a private endpoint with strict IAM access only.

• D. Enable Model Monitoring for drift and trigger retraining when input distributions change.

Best answer: B

Explanation: The key requirement is content-level protection for a generative AI application at inference time. Model Armor is designed for securing LLM interactions by inspecting prompts and responses, applying safety controls, detecting prompt injection or jailbreak patterns, and filtering sensitive data such as PII or secrets. This addresses two risk types: unintentional exposure by legitimate users and deliberate misuse by attackers. Logging or alerting on policy violations then supports ongoing monitoring of attempted abuse.

Network controls, retraining triggers, and training-time validation can be useful, but they do not inspect live LLM inputs and outputs for both accidental leakage and adversarial prompts.

• Drift monitoring tracks model behavior or data distribution changes, but it is not the primary control for prompt injection or sensitive-data leakage.
• Private endpoint only reduces exposure but still allows authorized users or compromised clients to send unsafe prompts.
• Training-only Regex misses inference-time prompts and responses, where accidental leakage and deliberate misuse occur.

Question 4

Topic: Monitoring AI Solutions

A customer support assistant built with Gemini and RAG is running in production. Model Monitoring and an LLM-as-a-judge evaluation raise an incident for unsafe escalation advice.

The service must restore safe behavior quickly and preserve evidence for review. What should you do first?

Options:

• A. Rebuild the RAG index from the source documents

• B. Roll back the prompt template to the last approved version

• C. Start a new fine-tuning job on recent incidents

• D. Disable safety filters to capture unmodified model output

Best answer: B

Explanation: This is an incident response for a production generative AI regression. The timing and replay evidence point to the prompt template release, not the model, retriever, traffic pattern, or serving infrastructure. The first operation should reduce user harm by returning to the last known safe prompt version, while retaining logs, prompts, responses, evaluation results, and deployment metadata for root-cause analysis. After mitigation, the team can add regression tests, strengthen evaluation gates, and review prompt-change controls before re-releasing. The key is to contain the unsafe behavior before starting slower corrective work.

• Disabling safety filters increases user risk and violates the requirement to restore safe behavior quickly.
• Fine-tuning is too slow and is not indicated when replay tests isolate the issue to a prompt change.
• Rebuilding the RAG index does not match the evidence because the index did not change and only the new prompt fails.

Question 5

Topic: Monitoring AI Solutions

After Model Monitoring on Gemini Enterprise Agent Platform detects drift, a team triggers an Agent Platform Pipelines retraining run for a custom TensorFlow model. The training job fails before the first epoch. What is the most likely cause?

Exhibit: Training log excerpt

INFO Input path: gs://ml-data/churn/retrain/*
INFO Loaded schema: purchase_amount FLOAT, tenure_months INT64, label INT64
INFO Parsed records from dt=2026-07-17 successfully
ERROR Failed parsing dt=2026-07-18/part-00003.jsonl
ERROR Feature purchase_amount: expected float32, got bytes value '19.99'

Options:

• A. The custom container is missing TensorFlow dependencies

• B. The worker ran out of accelerator memory

• C. The training service account lacks Cloud Storage read access

• D. The new data partition has a feature type mismatch

Best answer: D

Explanation: The failure happens during input parsing, before the first epoch, and the log identifies a specific feature type problem. The training schema expects purchase_amount as a numeric FLOAT, but the failing file contains it as a bytes/string value ('19.99'). Because the previous partition parsed successfully, the issue is likely an upstream schema change or export/casting problem in the new retraining data partition. The next diagnostic step would be to inspect the transform or export job for dt=2026-07-18 and enforce the expected numeric schema before training.

• Read access is unlikely because the job reads records and reports a specific parse failure.
• Accelerator memory is unlikely because the job fails before training steps begin.
• Missing dependencies would usually appear as import or startup errors, not a feature parsing mismatch.

Question 6

Topic: Monitoring AI Solutions

A credit-risk model served on Agent Platform Inference has stable overall accuracy and latency. A compliance review reports that applicants from one demographic group are receiving unfavorable predictions at a much higher rate than other groups. The team needs to confirm whether the production outputs show biased behavior and investigate contributing factors. What is the best next diagnostic step?

Options:

• A. Retrain immediately after removing the demographic field

• B. Monitor CPU, memory, and endpoint latency by region

• C. Monitor prediction and error metrics by demographic slice with explanations

• D. Alert only on overall accuracy and AUC changes

Best answer: C

Explanation: Bias monitoring should examine model behavior across relevant population slices, not only aggregate performance. In this case, overall accuracy is stable, but the reported symptom is a disparity in unfavorable predictions for a demographic group. The best diagnostic approach is to track production prediction rates and error metrics by demographic slice, then use explainability or feature attribution to investigate which inputs may be contributing to the disparity. Removing a sensitive field without analysis is not sufficient because proxy variables can still encode group membership. The key takeaway is to monitor and investigate fairness at the slice level.

• Aggregate metrics can hide subgroup harm because high overall AUC may coexist with biased predictions for a protected group.
• Infrastructure metrics help diagnose serving health, but they do not measure output fairness or prediction disparities.
• Removing one field is premature and may not remove proxy bias; diagnosis should precede remediation.

Question 7

Topic: Monitoring AI Solutions

A healthcare company uses a Gemini Enterprise Agent Platform RAG assistant to answer internal clinical policy questions. The assistant retrieves snippets from patient-care notes that can contain PHI. For LLM-as-a-judge evaluation, the pipeline writes raw prompts, retrieved chunks, model outputs, and model artifacts to a BigQuery dataset shared with a broad analytics group. A monitoring review finds exact patient-note text in some outputs. Which engineering decision best addresses the data exfiltration risk?

Options:

• A. Encrypt the dataset while keeping current access.

• B. Store only redacted workflow artifacts in a restricted dataset.

• C. Filter only delivered responses with safety settings.

• D. Increase evaluation sampling of raw traces.

Best answer: B

Explanation: Data exfiltration risk in AI workflows is not limited to the final answer. Prompts, retrieved context, model outputs, evaluation traces, and model artifacts can all expose sensitive training or source data if they are stored raw or shared too broadly. In this scenario, raw PHI-bearing artifacts are being written to BigQuery and made available to a broad analytics group, so the engineering decision should minimize what is retained and restrict who can access it. Redaction or tokenization before logging, plus a least-privilege dataset boundary, lets evaluation continue without spreading raw PHI. Filtering the user-visible answer alone does not protect raw traces or artifacts already captured by the workflow.

• Response-only filtering misses raw prompts, retrieved chunks, and evaluation logs that can still leak PHI.
• Encryption only protects data at rest but does not fix overly broad authorized access.
• More raw sampling may improve detection, but it increases the amount of sensitive material exposed.

Question 8

Topic: Monitoring AI Solutions

An ML engineer launches an Agent Platform custom training job that uses a custom container. The same code reads the data successfully from the engineer’s notebook, but the managed training job fails before the first batch. The job is configured to run as train-runner@project.iam.gserviceaccount.com.

Exhibit: Training log

Starting task: trainer.task
Reading gs://ml-prod-data/churn/train/*.csv
ERROR: 403 Forbidden
Permission denied: storage.objects.get
Principal: train-runner@project.iam.gserviceaccount.com

What is the best configuration change?

Options:

• A. Mount the notebook user credentials into the container

• B. Increase worker memory for the custom training job

• C. Grant bucket read access to the training service account

• D. Register the model artifact before starting training

Best answer: C

Explanation: Managed custom training runs under the service account configured for the job, not the engineer’s notebook credentials. The visible failure is 403 Forbidden on storage.objects.get for the training service account before any batches are read, so the problem is data access. Configure least-privilege IAM so the job’s service account can read the required Cloud Storage objects, then rerun the job.

Changing compute resources would not fix an authorization error, and model registry actions happen after a model artifact exists.

• More memory addresses out-of-memory or resource exhaustion symptoms, not a 403 Forbidden object-read failure.
• Model registration is a later lifecycle step and cannot resolve failure to load training data.
• Notebook credentials would be an insecure workaround and ignores that managed jobs should use configured service accounts.

Question 9

Topic: Monitoring AI Solutions

A team deployed a Gemini RAG assistant for internal policy search. Red-team evaluation shows that prompts containing instruction-override attempts sometimes cause the assistant to return restricted retrieved snippets. Toxicity and violence safety filters are already enabled, and Regex redaction catches employee IDs in normal outputs. Which control should the engineer add next to address the observed risk?

Options:

• A. Apply Model Armor to prompts and responses

• B. Add more Regex redaction patterns

• C. Tighten content safety filters

• D. Lower the model temperature

Best answer: A

Explanation: The failure is a generative AI security issue: adversarial instructions are causing the app to reveal restricted retrieved content. Safety filters are mainly for harmful-content categories, and Regex is useful for deterministic strings like employee IDs. Neither reliably handles semantically varied prompt-injection attempts or broader data-exfiltration behavior. Model Armor is the appropriate control to place around the application’s model traffic so prompts and responses can be checked for these AI-specific risks. The existing controls are useful, but they are aimed at different failure modes.

• Safety filters target harmful content categories, so tightening them does not directly address instruction override and retrieval leakage.
• Regex rules work for known patterns, but prompt injection is semantically varied and the leaked snippets are not just employee IDs.
• Temperature reduction may make outputs more stable, but it is not a security control for adversarial prompts.

Question 10

Topic: Monitoring AI Solutions

A fraud model is deployed on Agent Platform Inference. Input feature distributions and validation accuracy look stable, but an explainability report shows that the features driving predictions have changed since training.

Which monitoring setup best detects this production issue?

Options:

• A. Enable feature attribution drift monitoring against the training baseline.

• B. Run training-serving skew checks for schema differences.

• C. Enable only input data drift monitoring on raw feature values.

• D. Compare daily predicted class distributions from batch predictions.

Best answer: A

Explanation: Feature attribution drift is the right monitoring signal when the model’s explanations change, even if raw input distributions and headline accuracy appear stable. In this scenario, the important clue is that feature contribution ranks shifted: features that were influential during training are no longer driving predictions, while other features became more influential. Model Monitoring on Gemini Enterprise Agent Platform should compare production feature attributions with a training or reference baseline and alert when contribution patterns change unexpectedly.

Input data drift, prediction-distribution checks, and training-serving skew can be useful, but they monitor different failure modes. The key takeaway is that attribution drift is about changing model reliance, not just changing input values.

• Raw data drift can miss cases where feature values look stable but the model’s reliance on those features changes.
• Training-serving skew targets inconsistent preprocessing or schema differences, not changing explanation patterns after deployment.
• Prediction distribution checks show output shifts, but they do not identify which features are driving predictions differently.

Continue in the web app

Use IT Mastery for interactive Google Cloud Professional ML Engineer practice with mixed sets, timed mocks, topic drills, explanations, and progress tracking.

Try Google Cloud Professional ML Engineer on Web

Related focused pages

• Free Google Cloud Professional ML Engineer Full-Length Practice Exam
• Low-Code AI Architecture
• Data and Model Collaboration
• Scaling ML Models
• Serving and Scaling Models
• ML Pipeline Automation