Pass the Amazon Web Services AWS Certified AI Practitioner AIF-C01 Questions and answers with ExamsMirror

The company is building a chatbot using an LLM to answer questions about HR policies, with access to a large digital documentation base. Retrieval Augmented Generation (RAG) optimizes the LLM’s responses by retrieving relevant information from the documentation base and using it to generate accurate, contextually grounded answers, reducing hallucinations and improving response quality.

Exact Extract from AWS AI Documents:

From the AWS Bedrock User Guide:

"Retrieval Augmented Generation (RAG) enhances the performance of large language models by retrieving relevant information from external knowledge bases, such as documentation or databases, and incorporating it into the generation process. This technique ensures responses are accurate and grounded in the provided data, making it ideal for applications like policy chatbots."

(Source: AWS Bedrock User Guide, Retrieval Augmented Generation)

Detailed Explanation:

Option A: Use Retrieval Augmented Generation (RAG).This is the correct answer. RAG leverages the documentation base to provide the LLM with relevant HR policy information, optimizing the chatbot’s responses for accuracy and relevance.

Option B: Use few-shot prompting.Few-shot prompting provides a few examples in the prompt to guide the LLM, but it is less effective than RAG for large documentation bases, as it cannot dynamically retrieve specific policy details.

Option C: Set the temperature to 1.Setting the temperature to 1 controls the randomness of the LLM’s output but does not optimize responses using external documentation. This option is unrelated to the documentation base.

Option D: Decrease the token size.Decreasing token size (likely referring to limiting input/output tokens) may reduce response length but does not optimize the quality of responses using the documentation base.

Adjusting the prompt is the correct solution to align the LLM outputs with the company's expectations for short, specific language responses.

Adjust the Prompt:

Modifying the prompt can guide the LLM to produce outputs that are shorter and tailored to the desired language.

A well-crafted prompt can provide specific instructions to the model, such as "Answer in a short sentence in Spanish."

Why Option A is Correct:

Control Over Output: Adjusting the prompt allows for direct control over the style, length, and language of the LLM outputs.

Flexibility: Prompt engineering is a flexible approach to refining the model’s behavior without modifying the model itself.

Why Other Options are Incorrect:

B. Choose an LLM of a different size: The model size does not directly impact the response length or language.

C. Increase the temperature: Increases randomness in responses but does not ensure brevity or specific language.

D. Increase the Top K value: Affects diversity in model output but does not align directly with response length or language specificity.

The ecommerce company’s chatbot, powered by AI, automates customer order submissions and is accessible 24/7 via the website. Prompt injection is an AI system input vulnerability where malicious users craft inputs to manipulate the chatbot’s behavior, such as bypassing safeguards or accessing unauthorized information. This vulnerability must be resolved before the chatbot is made available to ensure security.

Exact Extract from AWS AI Documents:

From the AWS Bedrock User Guide:

"Prompt injection is a vulnerability in AI systems, particularly chatbots, where malicious inputs can manipulate the model’s behavior, potentially leading to unauthorized actions or harmful outputs. Implementing guardrails and input validation can mitigate this risk."

(Source: AWS Bedrock User Guide, Security Best Practices)

Detailed Explanation:

Option A: Data leakageData leakage refers to the unintended exposure of sensitive data during model training or inference, not an input vulnerability affecting a chatbot’s operation.

Option B: Prompt injectionThis is the correct answer. Prompt injection is a critical input vulnerability for chatbots, where malicious prompts can exploit the AI to produce harmful or unauthorized responses, a risk that must be addressed before launch.

Option C: Large language model (LLM) hallucinationsLLM hallucinations refer to the model generating incorrect or ungrounded responses, which is an output issue, not an input vulnerability.

Option D: Concept driftConcept drift occurs when the data distribution changes over time, affecting model performance. It is not an input vulnerability but a long-term performance issue.

Increasing the number of epochs during model training allows the model to learn from the data over more iterations, potentially improving its accuracy up to a certain point. This is a common practice when attempting to reach a specific level of accuracy.

Option B (Correct): "Increase the epochs": This is the correct answer because increasing epochs allows the model to learn more from the data, which can lead to higher accuracy.

Option A: "Decrease the batch size" is incorrect as it mainly affects training speed and may lead to overfitting but does not directly relate to achieving a specific accuracy level.

Option C: "Decrease the epochs" is incorrect as it would reduce the training time, possibly preventing the model from reaching the desired accuracy.

Option D: "Increase the temperature parameter" is incorrect because temperature affects the randomness of predictions, not model accuracy.

AWS AI Practitioner References:

Model Training Best Practices on AWS: AWS suggests adjusting training parameters, like the number of epochs, to improve model performance.

To determine which model generates responses in a style that the company's employees prefer, the best approach is to use a human workforce to evaluate the models with custom prompt datasets. This method allows for subjective evaluation based on the specific stylistic preferences of the company's employees, which cannot be effectively assessed through automated methods or pre-built datasets.

Option B (Correct): "Evaluate the models by using a human workforce and custom prompt datasets": This is the correct answer as it directly involves human judgment to evaluate the style and quality of the responses, aligning with employee preferences.

Option A: "Evaluate the models by using built-in prompt datasets" is incorrect because built-in datasets may not capture the company's specific stylistic requirements.

Option C: "Use public model leaderboards to identify the model" is incorrect as leaderboards typically measure model performance on standard benchmarks, not on stylistic preferences.

Option D: "Use the model InvocationLatency runtime metrics in Amazon CloudWatch" is incorrect because latency metrics do not provide any information about the style of the model's responses.

AWS AI Practitioner References:

Model Evaluation Techniques on AWS: AWS suggests using human evaluators to assess qualitative aspects of model outputs, such as style and tone, to ensure alignment with organizational preferences

Amazon SageMaker Model Cards provide a centralized and standardized repository for documenting machine learning models. They capture key details such as the model's intended use, training and evaluation datasets, performance metrics, ethical considerations, and other relevant information. This documentation facilitates governance and reporting by ensuring that all stakeholders have access to consistent and comprehensive information about each model. While Amazon SageMaker Debugger is used for real-time debugging and monitoring during training, and Amazon SageMaker Model Monitor tracks deployed models for data and predictionquality, neither offers the comprehensive documentation capabilities of Model Cards. Amazon SageMaker JumpStart provides pre-built models and solutions but does not focus on governance documentation.

Amazon Bedrock provides large language models (LLMs) that are optimized for natural language understanding and text summarization tasks, making it the best choice for creating concise summaries of user reviews. Time-series forecasting, classification, and image analysis (Rekognition) are not suitable for summarizing textual data. References: AWS Bedrock Documentation.

AWS Glue is the correct service for transforming petabytes of unstructured data into a structured format suitable for machine learning tasks.

AWS Glue:

A fully managed extract, transform, and load (ETL) service that makes it easy to prepare and transform unstructured data into a structured format.

Provides a range of tools for cleaning, enriching, and cataloging data, making it ready for data scientists to use in ML models.

Why Option D is Correct:

Data Transformation: AWS Glue can handle large volumes of data and transform unstructured data into structured formats efficiently.

Integrated ML Support: Glue integrates with other AWS services to support ML workflows.

Why Other Options are Incorrect:

A. Amazon Lex: Used for building chatbots, not for data transformation.

B. Amazon Rekognition: Used for image and video analysis, not for data transformation.

C. Amazon Kinesis Data Streams: Handles real-time data streaming, not suitable for batch transformation of large volumes of unstructured data.

Generative AI is a type of AI that creates new content, such as text, images, or audio, often mimicking human-like outputs. A practical use case for generative AI is employing a chatbot to provide human-like responses to customer queries in real time, as it leverages the ability of large language models (LLMs) to generate natural language responses dynamically.

Exact Extract from AWS AI Documents:

From the AWS Bedrock User Guide:

"Generative AI enables applications like chatbots to produce human-like text responses in real time, enhancing customer support by providing natural and contextually relevant answers to user queries."

(Source: AWS Bedrock User Guide, Introduction to Generative AI)

Detailed Explanation:

Option A: Using an ML model to forecast product demandForecasting product demand typically involves predictive analytics using supervised learning (e.g., regression models), not generative AI, which focuses on creating new content.

Option B: Employing a chatbot to provide human-like responses to customer queries in real timeThis is the correct answer. Generative AI, particularly LLMs, is commonly used to power chatbots that generate human-like responses, making this a practical use case.

Option C: Using an analytics dashboard to track website traffic and user behaviorAn analytics dashboard involves data visualization and analysis, not generative AI, which is about creating new content.

Option D: Implementing a rule-based recommendation engine to suggest products to customersA rule-based recommendation engine relies on predefined rules, not generative AI. Generative AI could be used for more dynamic recommendations, but this scenario does not describe such a case.

To build an AI model responsibly and minimize bias, it is essential to ensure fairness and transparency throughout the model development and deployment process. This involves detecting and mitigating data imbalances and thoroughly evaluating the model's behavior to understand its impact on different groups.

Option A (Correct): "Detect imbalances or disparities in the data": This is correct because identifying and addressing data imbalances or disparities is a critical step in reducing bias. AWS provides tools like Amazon SageMaker Clarify to detect bias during data preprocessing and model training.

Option C (Correct): "Evaluate the model's behavior so that the company can provide transparency to stakeholders": This is correct because evaluating the model's behavior for fairness and accuracy is key to ensuring that stakeholders understand how the model makes decisions. Transparency is a crucial aspect of responsible AI.

Option B: "Ensure that the model runs frequently" is incorrect because the frequency of model runs does not address bias.

Option D: "Use the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) technique to ensure that the model is 100% accurate" is incorrect because ROUGE is a metric for evaluating the quality of text summarization models, not for minimizing bias.

Option E: "Ensure that the model's inference time is within the accepted limits" is incorrect as it relates to performance, not bias reduction.

AWS AI Practitioner References:

Amazon SageMaker Clarify: AWS offers tools such as SageMaker Clarify for detecting bias in datasets and models, and for understanding model behavior to ensure fairness and transparency.

Responsible AI Practices: AWS promotes responsible AI by advocating for fairness, transparency, and inclusivity in model development and deployment.