Manage Model Health

The Health page has two views into the health of your model: Project Overview and Targets. It's also where you can run a rebuild of the project.

Analyze Model Health by Project

The Project Overview view shows a color-coded distribution chart visualizing the health of the best models. The Overall Health Score is a scaled metric ranging between -1 and 1, which indicates improvement or degradation in a model’s predictive accuracy.

A positive (green) health score signals that a model’s predictive accuracy has improved since it was deployed. A negative (red) health score signals a decrease in predictive accuracy since initial deployment.

The AI Service Messages provides information on how the overall health score has changed with each prediction. The Online Builds pane lists details of all builds for the project.

Analyze Model Health by Target

In the Targets view, you can see how the evaluation metric score for a given target/model/metric score combination changed over time.

Click a target in the Targets pane to see the average metric score for each of its models, based on the error metric selected in the Metric Name field. Use the Metric Name drop-down to select the type of error metric. See Appendix 3: Error Metrics for more information on the error metrics Sensible Machine Learning uses.

For each model of the selected target, you can view a line graph that shows the metric scores for each of its prediction runs. Click a model name in the top pane to view the model's prediction run information in the line chart. This includes the date of each of the model's prediction runs and the average metric score for each.

TIP: A forecast is not included in the line chart if no actuals are present for the metrics to be calculated. The date displays on the x-axis, but no corresponding value displays.

Rebuild the Model

You can rebuild the model when needed. The rebuild process can be automated to run as a data management job in the background, or you can click Run Rebuild on the Health page to run it manually. There are twol rebuild methods:

• Auto Rebuild: Automatically rebuilds and deploys the entire project using the same configurations from the latest build. The new build’s models are trained on more historical data than in the initial build when additional data is being used since the latest build.

• Manual Rebuild: Returns you to the Dataset page of the Data Section in the Model Build phase. You can walk through the Model Build phase, setting different configurations as needed.