Pass the Databricks ML Data Scientist Databricks-Machine-Learning-Professional Questions and answers with ExamsMirror

 In a continuous integration, continuous deployment (CI/CD) process for machine learning pipelines, the arrival of a new model version in the MLflow Model Registry is a common event that triggers the execution of automated testing. This is because the MLflow Model Registry is a centralized repository for managing the full lifecycle of MLflow models. It provides model lineage, model versioning, stage transitions, and annotations. When a new model version is registered, it can be automatically tested for quality, performance, and compliance using various tools and frameworks, such as MLflow Projects, MLflow Models, MLflow Tracking, and MLflow Model Serving. The automated testing can help ensure that the new model version meets the desired criteria and standards before it is deployed to production or other stages. The automated testing can also help detect and prevent any errors, bugs, or drifts that may affect the model functionality or accuracy. References:

MLflow Model Registry Documentation, p. 1-2

MLOps: Continuous delivery and automation pipelines in machine learning, Continuous integration and delivery (CI/CD) for ML, p. 5-6

Machine Learning Engineering with MLflow, Testing and Validating ML Models, p. 1-2

CI/CD for Machine Learning: What it is & Benefits in 2023 - AIMultiple, CI/CD for Machine Learning, p. 1-2

Structured Streaming is a scalable and fault-tolerant stream processing engine built on the Spark SQL engine. It allows users to express streaming computations the same way as batch computations on static data, using DataFrame and Dataset APIs. Structured Streaming can handle both unbounded and bounded data sources, and can process data in micro-batches or continuously. Structured Streaming can be used to provide continuous processing of data for machine learning applications, such as data ingestion, preprocessing, feature engineering, and model inference. StructuredStreaming can also integrate with MLflow and Delta Lake to enable end-to-end machine learning pipelines with tracking, reproducibility, and governance123 References:

Structured Streaming Programming Guide - Spark 3.2.0 Documentation

Structured Streaming + Kafka Integration Guide (Kafka broker version 0.10.0 or higher) - Spark 3.2.0 Documentation

Machine Learning with Structured Streaming - Databricks

[Continuous Machine Learning with Structured Streaming and MLflow - Databricks]

MLflow Model Serving is a feature that allows users to host MLflow models as REST endpoints on Databricks1. To query a model version that is in the Production stage of the MLflow Model Registry, the user can use the following model URI format: https:// /model-serving///invocations2. In this case, the model name is recommender and the stage is Production, so the correct model URI is https:// /model-serving/recommender/Production/invocations. The other options are incorrect because they either use the wrong prefix (model instead of model-serving), the wrong stage name (stage-production instead of Production), or require the version number of the model version in Production, which is not necessary for querying the model by stage. References: Announcing MLflow Model Serving on Databricks, [MLflow Model Serving], [MLflow Model Registry]

Concept drift in machine learning and data mining refers to the change in the relationships between input and output data in the underlying problem over time. In other domains, this change maybe called “covariate shift,” “dataset shift,” or “nonstationarity.” Concept drift can affect the performance and accuracy of predictive models that assume a static relationship between input and output variables. Concept drift can be caused by various factors, such as changes in user behavior, environmental conditions, market trends, etc. Concept drift can be detected and handled by various methods, such as periodic retraining, online learning, ensemble methods, etc References:

Concept drift - Wikipedia

A Gentle Introduction to Concept Drift in Machine Learning

Model Drift & Machine Learning: Concept Drift, Feature Drift, Etc.

Data Drift vs. Concept Drift: What Is the Difference?

The mlflow.register_model operation can be used to add a new version of an MLflow Model to an existing MLflow Model Registry model. This operation takes the URI of the model source (such as an MLflow run or a local path) and the name of the registered model as arguments, and returns a ModelVersion object that contains metadata about the registered model version. The mlflow.register_model operation can also create a new registered model if the name does not exist in the Model Registry.

The other options are incorrect for the following reasons:

B. MlflowClient.update_registered_model is used to update the metadata of an existing registered model, such as the name or description, not to add a new version of the model.

C. mlflow.add_model_version does not exist as a valid MLflow operation.

D. MlflowClient.get_model_version is used to fetch the metadata of a specific version of a registered model, not to add a new version of the model.

E. The machine learning engineer does not need to create an entirely new MLflow Model Registry model, as they can use the mlflow.register_model operation to add a new version of the model to the existing “project” model.

References: Adding an MLflow Model to the Model Registry, mlflow.register_model

MLflow provides a module called mlflow.shap that can be used to automatically calculate and log a Shapley feature importance plot using the SHAP library. The mlflow.shap.log_explanation function takes a predict function and a sample of input data, and computes the SHAP values and base values for each feature. It then logs the following artifacts to the current run:

Base values

SHAP values (computed using shap.KernelExplainer)

Summary bar plot (shows the average impact of each feature on model output) The mlflow.shap.log_explanation function can be used with any model that has a predict function, such as scikit-learn, xgboost, etc1234 References:

mlflow.shap — MLflow 2.9.1 documentation

Extract feature importance from a mlflow 1.9 PyFuncModel model

9.6 SHAP (SHapley Additive exPlanations) | Interpretable Machine Learning

load MLFlow model and plot feature importance with feature names

To move a model version from the Staging stage to the Production stage using the MLflow Client, you can use the client.transition_model_version_stage method. This method takes the name of the registered model, the version of the model, and the target stage as arguments. It transitions the model version to the specified stage and returns the updated model version object. The syntax of the method is as follows:

Python

client.transition_model_version_stage (name, version, stage)

AI-generated code. Review and use carefully. More info on FAQ.

The other options are incorrect because:

Option A: client.transition_model_version_stage with the stage argument set to “Staging” will not move the model version from the Staging stage to the Production stage, but rather keep it in the same stage.

Option B: client.transition_model_stage is not a valid method name, as it is missing an underscore. The correct method name is client.transition_model_version_stage.

Option D: client.transition_model_stage is not a valid method name, as it is missing an underscore. The correct method name is client.transition_model_version_stage. Also, the method does not take from and to arguments, but only a stage argument.

Option E: client.transition_model_version_stage does not take from and to arguments, but only a stage argument. The from argument is redundant, as the method will automatically transition the model version from its current stage tothe target stage. References: MLflow Model Registry, MlflowClient.transition_model_version_stage

To enable MLflow Autologging for all supported machine learning libraries in a notebook, you can use the mlflow.autolog() function. This function enables autologging for each library by invoking the corresponding autologging function, such as mlflow.sklearn.autolog() or mlflow.spark.autolog(). This way, you do not need to call each autologging function separately. The mlflow.autolog() function also takes care of the compatibility and configuration issues that may arise from different versions of the Databricks Runtime for Machine Learning or workspace-wide settings in the Admin Console. The other options are incorrect because:

Option A: mlflow.sklearn.autolog() will only enable autologging for scikit-learn, not for other libraries.

Option B: mlflow.spark.autolog() will only enable autologging for Spark MLlib, not for other libraries.

Option C: spark.conf.set(“autologging”, True) is not a valid way to enable autologging, as there is no such configuration option in Spark.

Option D: It is possible to automatically log MLflow runs using the autologging functions provided by MLflow. References: Automatic Logging with MLflow Tracking, MLflow quickstart (Python)