abacusai.model_monitor

Classes

`ModelMonitorVersion`	A version of a model monitor
`RefreshSchedule`	A refresh schedule for an object. Defines when the next version of the object will be created
`AbstractApiClass`
`ModelMonitor`	A model monitor

Module Contents

class abacusai.model_monitor.ModelMonitorVersion(client, modelMonitorVersion=None, status=None, modelMonitorId=None, monitoringStartedAt=None, monitoringCompletedAt=None, trainingFeatureGroupVersion=None, predictionFeatureGroupVersion=None, error=None, pendingDeploymentIds=None, failedDeploymentIds=None, metricConfigs=None, featureGroupMonitorConfigs=None, metricTypes=None, modelVersion=None, batchPredictionVersion=None, edaConfigs=None, trainingForecastConfig=None, predictionForecastConfig=None, forecastFrequency=None, monitorDriftConfig=None, predictionDataUseMappings=None, trainingDataUseMappings=None)

Bases: abacusai.return_class.AbstractApiClass

A version of a model monitor

Parameters:

client (ApiClient) – An authenticated API Client instance
modelMonitorVersion (str) – The unique identifier of a model monitor version.
status (str) – The current status of the model.
modelMonitorId (str) – A reference to the model monitor this version belongs to.
monitoringStartedAt (str) – The start time and date of the monitoring process.
monitoringCompletedAt (str) – The end time and date of the monitoring process.
trainingFeatureGroupVersion (list[str]) – Feature group version IDs that this refresh pipeline run is monitoring.
predictionFeatureGroupVersion (list[str]) – Feature group version IDs that this refresh pipeline run is monitoring.
error (str) – Relevant error if the status is FAILED.
pendingDeploymentIds (list) – List of deployment IDs where deployment is pending.
failedDeploymentIds (list) – List of failed deployment IDs.
metricConfigs (list[dict]) – List of metric configs for the model monitor instance.
featureGroupMonitorConfigs (dict) – Configurations for feature group monitor
metricTypes (list) – List of metric types.
modelVersion (list[str]) – Model version IDs that this refresh pipeline run is monitoring.
batchPredictionVersion (str) – The batch prediction version this model monitor is monitoring
edaConfigs (list) – The list of eda configs for the version
trainingForecastConfig (dict) – The training forecast config for the monitor version
predictionForecastConfig (dict) – The prediction forecast config for the monitor version
forecastFrequency (str) – The forecast frequency for the monitor version
monitorDriftConfig (dict) – The monitor drift config for the monitor version
predictionDataUseMappings (dict) – The mapping of prediction data use to feature group version
trainingDataUseMappings (dict) – The mapping of training data use to feature group version

__repr__(): Return repr(self).

to_dict()

Get a dict representation of the parameters in this class

Returns:: The dict value representation of the class parameters
Return type:: dict

get_prediction_drift()

Gets the label and prediction drifts for a model monitor.

Parameters:: model_monitor_version (str) – Unique string identifier for a model monitor version created under the project.
Returns:: Object describing training and prediction output label and prediction distributions.
Return type:: DriftDistributions

refresh()

Calls describe and refreshes the current object’s fields

Returns:: The current object
Return type:: ModelMonitorVersion

describe()

Retrieves a full description of the specified model monitor version.

Parameters:: model_monitor_version (str) – The unique version ID of the model monitor version.
Returns:: A model monitor version.
Return type:: ModelMonitorVersion

delete()

Deletes the specified model monitor version.

Parameters:: model_monitor_version (str) – Unique identifier of the model monitor version to delete.

metric_data(metric_type, actual_values_to_detail=None)

Provides the data needed for decile metrics associated with the model monitor.

Parameters:

metric_type (str) – The type of metric to get data for.
actual_values_to_detail (list) – The actual values to detail.

Returns:

Data associated with the metric.

Return type:

ModelMonitorVersionMetricData

list_monitor_alert_versions_for_monitor_version()

Retrieves the list of monitor alert versions for a specified monitor instance.

Parameters:: model_monitor_version (str) – The unique ID associated with the model monitor.
Returns:: A list of monitor alert versions.
Return type:: list[MonitorAlertVersion]

get_drift_for_feature(feature_name, nested_feature_name=None)

Gets the feature drift associated with a single feature in an output feature group from a prediction.

Parameters:

feature_name (str) – Name of the feature to view the distribution of.
nested_feature_name (str) – Optionally, the name of the nested feature that the feature is in.

Returns:

An object describing the training and prediction output feature distributions.

Return type:

FeatureDistribution

get_outliers_for_feature(feature_name=None, nested_feature_name=None)

Gets a list of outliers measured by a single feature (or overall) in an output feature group from a prediction.

Parameters:

feature_name (str) – Name of the feature to view the distribution of.
nested_feature_name (str) – Optionally, the name of the nested feature that the feature is in.

wait_for_monitor(timeout=1200)

A waiting call until model monitor version is ready.

Parameters:: timeout (int) – The waiting time given to the call to finish, if it doesn’t finish by the allocated time, the call is said to be timed out.

get_status()

Gets the status of the model monitor version.

Returns:: A string describing the status of the model monitor version, for e.g., pending, complete, etc.
Return type:: str

class abacusai.model_monitor.RefreshSchedule(client, refreshPolicyId=None, nextRunTime=None, cron=None, refreshType=None, error=None)

Bases: abacusai.return_class.AbstractApiClass

A refresh schedule for an object. Defines when the next version of the object will be created

Parameters:

client (ApiClient) – An authenticated API Client instance
refreshPolicyId (str) – The unique identifier of the refresh policy
nextRunTime (str) – The next run time of the refresh policy. If null, the policy is paused.
cron (str) – A cron-style string that describes the when this refresh policy is to be executed in UTC
refreshType (str) – The type of refresh that will be run
error (str) – An error message for the last pipeline run of a policy

__repr__(): Return repr(self).

to_dict()

Get a dict representation of the parameters in this class

Returns:: The dict value representation of the class parameters
Return type:: dict

class abacusai.model_monitor.AbstractApiClass(client, id)

__eq__(other): Return self==value.

_get_attribute_as_dict(attribute)

class abacusai.model_monitor.ModelMonitor(client, modelMonitorId=None, name=None, createdAt=None, projectId=None, trainingFeatureGroupId=None, predictionFeatureGroupId=None, predictionFeatureGroupVersion=None, trainingFeatureGroupVersion=None, alertConfig=None, biasMetricId=None, metricConfigs=None, featureGroupMonitorConfigs=None, metricTypes=None, modelId=None, starred=None, batchPredictionId=None, monitorType=None, edaConfigs=None, trainingForecastConfig=None, predictionForecastConfig=None, forecastFrequency=None, trainingFeatureGroupSampling=None, predictionFeatureGroupSampling=None, monitorDriftConfig=None, predictionDataUseMappings=None, trainingDataUseMappings=None, refreshSchedules={}, latestMonitorModelVersion={})

Bases: abacusai.return_class.AbstractApiClass

A model monitor

Parameters:

client (ApiClient) – An authenticated API Client instance
modelMonitorId (str) – The unique identifier of the model monitor.
name (str) – The user-friendly name for the model monitor.
createdAt (str) – Date and time at which the model was created.
projectId (str) – The project this model belongs to.
trainingFeatureGroupId (list[str]) – Feature group IDs that this model monitor is monitoring.
predictionFeatureGroupId (list[str]) – Feature group IDs that this model monitor is monitoring.
predictionFeatureGroupVersion (list[str]) – Feature group versions that this model monitor is monitoring.
trainingFeatureGroupVersion (list[str]) – Feature group versions that this model monitor is monitoring.
alertConfig (dict) – Alerting configuration for this model monitor.
biasMetricId (str) – The bias metric ID
metricConfigs (dict) – Configurations for model monitor
featureGroupMonitorConfigs (dict) – Configurations for feature group monitor
metricTypes (dict) – List of metric types
modelId (str) – Model ID that this model monitor is monitoring.
starred (bool) – Whether this model monitor is starred.
batchPredictionId (str) – The batch prediction ID this model monitor monitors
monitorType (str) – The type of the monitor, one of MODEL_MONITOR, or FEATURE_GROUP_MONITOR
edaConfigs (dict) – The configs for EDA
trainingForecastConfig (dict) – The tarining config for forecast monitors
predictionForecastConfig (dict) – The prediction config for forecast monitors
forecastFrequency (str) – The frequency of the forecast
trainingFeatureGroupSampling (bool) – Whether or not we sample from training feature group
predictionFeatureGroupSampling (bool) – Whether or not we sample from prediction feature group
monitorDriftConfig (dict) – The monitor drift config for the monitor
predictionDataUseMappings (dict) – The data_use mapping of the prediction features
trainingDataUseMappings (dict) – The data_use mapping of the training features
latestMonitorModelVersion (ModelMonitorVersion) – The latest model monitor version.
refreshSchedules (RefreshSchedule) – List of refresh schedules that indicate when the next model version will be trained.

__repr__(): Return repr(self).

to_dict()

Get a dict representation of the parameters in this class

Returns:: The dict value representation of the class parameters
Return type:: dict

rerun()

Re-runs the specified model monitor.

Parameters:: model_monitor_id (str) – Unique string identifier of the model monitor to re-run.
Returns:: The model monitor that is being re-run.
Return type:: ModelMonitor

refresh()

Calls describe and refreshes the current object’s fields

Returns:: The current object
Return type:: ModelMonitor

describe()

Retrieves a full description of the specified model monitor.

Parameters:: model_monitor_id (str) – Unique string identifier associated with the model monitor.
Returns:: Description of the model monitor.
Return type:: ModelMonitor

get_summary()

Gets the summary of a model monitor across versions.

Parameters:: model_monitor_id (str) – A unique string identifier associated with the model monitor.
Returns:: An object describing integrity, bias violations, model accuracy and drift for the model monitor.
Return type:: ModelMonitorSummary

list_versions(limit=100, start_after_version=None)

Retrieves a list of versions for a given model monitor.

Parameters:

limit (int) – The maximum length of the list of all model monitor versions.
start_after_version (str) – The ID of the version after which the list starts.

Returns:

A list of model monitor versions.

Return type:

list[ModelMonitorVersion]

rename(name)

Renames a model monitor

Parameters:: name (str) – The new name to apply to the model monitor.

delete()

Deletes the specified Model Monitor and all its versions.

Parameters:: model_monitor_id (str) – Unique identifier of the Model Monitor to delete.

list_monitor_alerts_for_monitor(realtime_monitor_id=None)

Retrieves the list of monitor alerts for a specified monitor. One of the model_monitor_id or realtime_monitor_id is required but not both.

Parameters:: realtime_monitor_id (str) – The unique ID associated with the real-time monitor.
Returns:: A list of monitor alerts.
Return type:: list[MonitorAlert]