neptunecontrib.monitoring.sklearn
¶
Module Contents¶
Functions¶
|
Log sklearn regressor summary. |
|
Log sklearn classifier summary. |
|
Log estimator parameters. |
|
Log pickled estimator. |
|
Log test predictions. |
|
Log test predictions probabilities. |
|
Log estimator scores on |
|
Log learning curve chart. |
|
Log feature importance chart. |
|
Log residuals chart. |
|
Log prediction error chart. |
|
Log feature importance chart. |
|
Log classification report chart. |
|
Log confusion matrix. |
|
Log ROC-AUC chart. |
|
Log precision recall chart. |
|
Log class prediction error chart. |
|
Log sklearn kmeans summary. |
|
Log index of the cluster label each sample belongs to. |
|
Log K-elbow chart for KMeans clusterer. |
|
Log Silhouette Coefficients charts for KMeans clusterer. |
|
-
neptunecontrib.monitoring.sklearn.
log_regressor_summary
(regressor, X_train, X_test, y_train, y_test, model_name=None, nrows=1000, experiment=None, log_charts=True)¶ Log sklearn regressor summary.
This method automatically logs all regressor parameters, pickled estimator (model), test predictions as table, model performance visualizations and test metrics.
Regressor should be fitted before calling this function.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
regressor (
regressor
) –Fitted sklearn regressor objectX_train (
ndarray
) –Training data matrixX_test (
ndarray
) –Testing data matrixy_train (
ndarray
) –The regression target for trainingy_test (
ndarray
) –The regression target for testingmodel_name (str, optional, default is
None
) –If logging picked model, define a name of the file to be logged to model/<model_name>IfNone
- model/estimator.skl is used.nrows (int, optional, default is 1000) –
Log firstnrows
rows of test predictions.experiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.( (log_charts) – bool:, optional, default is
True
): | If True, calculate and send chart visualizations. | | NOTE: calculating visualizations is potentially expensive depending on input data and regressor, and | may take some time to finish. | | This is equivalent to calling log_learning_curve_chart, log_feature_importance_chart, | log_residuals_chart, log_prediction_error_chart, log_cooks_distance_chart functions from this module. | | If not all visualizations are needed, it’s recommended to set this parameter toFalse
and call | only the desired log functions
- Returns
None
Examples
Log random forest regressor summary
rfr = RandomForestRegressor() rfr.fit(X_train, y_train) neptune.init('my_workspace/my_project') neptune.create_experiment() log_regressor_summary(rfr, X_train, X_test, y_train, y_test)
-
neptunecontrib.monitoring.sklearn.
log_classifier_summary
(classifier, X_train, X_test, y_train, y_test, model_name=None, nrows=1000, experiment=None, log_charts=True)¶ Log sklearn classifier summary.
This method automatically logs all classifier parameters, pickled estimator (model), test predictions, predictions probabilities as table, model performance visualizations and test metrics.
Classifier should be fitted before calling this function.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
classifier (
classifier
) –Fitted sklearn classifier objectX_train (
ndarray
) –Training data matrixX_test (
ndarray
) –Testing data matrixy_train (
ndarray
) –The classification target for trainingy_test (
ndarray
) –The classification target for testingmodel_name (str, optional, default is
None
) –If logging picked model, define a name of the file to be logged to model/<model_name>IfNone
- estimator.skl is used.nrows (int, optional, default is 1000) –
Log firstnrows
rows of test predictions and predictions probabilities.experiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.( (log_charts) – bool:, optional, default is
True
): | If True, calculate and send chart visualizations. | | NOTE: calculating visualizations is potentially expensive depending on input data and classifier, and | may take some time to finish. | | This is equivalent to calling log_classification_report_chart, log_confusion_matrix_chart, | log_roc_auc_chart, log_precision_recall_chart, log_class_prediction_error_chart functions from this | module. | | If not all visualizations are needed, it’s recommended to set this parameter toFalse
and call | only the desired log functions
- Returns
None
Examples
Log random forest classifier summary
rfc = RandomForestClassifier() rfc.fit(X_train, y_train) neptune.init('my_workspace/my_project') neptune.create_experiment() log_classifier_summary(rfc, X_train, X_test, y_train, y_test)
-
neptunecontrib.monitoring.sklearn.
log_estimator_params
(estimator, experiment=None)¶ Log estimator parameters.
Log all estimator parameters as experiment properties.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
estimator (
estimator
) –Scikit-learn estimator from which to log parameters.experiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.
- Returns
None
Examples
rfr = RandomForestRegressor() neptune.init('my_workspace/my_project') neptune.create_experiment() log_estimator_params(rfr)
-
neptunecontrib.monitoring.sklearn.
log_pickled_model
(estimator, model_name=None, experiment=None)¶ Log pickled estimator.
Log estimator as pickled file to Neptune artifacts.
Estimator should be fitted before calling this function.
Path to file in the Neptune artifacts is
model/<model_name>
.Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
estimator (
estimator
) –Scikit-learn estimator to log.model_name (str, optional, default is
None
) –Name of the file.IfNone
-estimator.skl
is used.experiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.
- Returns
None
Examples
rfr = RandomForestRegressor() rfr.fit(X_train, y_train) neptune.init('my_workspace/my_project') neptune.create_experiment() log_pickled_model(rfr, 'my_model')
-
neptunecontrib.monitoring.sklearn.
log_test_predictions
(estimator, X_test, y_test, y_pred=None, nrows=1000, experiment=None)¶ Log test predictions.
Calculate and log test predictions and have them as csv file in the Neptune artifacts.
If you pass
y_pred
, then predictions are logged without computing fromX_test
data.Estimator should be fitted before calling this function.
Path to predictions in the Neptune artifacts is ‘csv/test_predictions.csv’.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
estimator (
estimator
) –Scikit-learn estimator to compute predictions.X_test (
ndarray
) –Testing data matrix.y_test (
ndarray
) –Target for testing.y_pred (
ndarray
, optional, default isNone
) –Estimator predictions on test data.nrows (int, optional, default is 1000) –
Number of rows to log.experiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.
- Returns
None
Examples
rfr = RandomForestRegressor() rfr.fit(X_train, y_train) neptune.init('my_workspace/my_project') neptune.create_experiment() log_test_predictions(rfr, X_test, y_test)
-
neptunecontrib.monitoring.sklearn.
log_test_preds_proba
(classifier, X_test, y_pred_proba=None, nrows=1000, experiment=None)¶ Log test predictions probabilities.
Calculate and log test predictions probabilities and have them as csv file in the Neptune artifacts.
If you pass
y_pred_proba
, then predictions probabilities are logged without computing fromX_test
data.Estimator should be fitted before calling this function.
Path to predictions probabilities in the Neptune artifacts is ‘csv/test_preds_proba.csv’.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
classifier (
classifier
) –Scikit-learn classifier to compute predictions probabilities.X_test (
ndarray
) –Testing data matrix.y_pred_proba (
ndarray
, optional, default isNone
) –Classifier predictions probabilities on test data.nrows (int, optional, default is 1000) –
Number of rows to log.experiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.
- Returns
None
Examples
rfc = RandomForestClassifier() rfc.fit(X_train, y_train) neptune.init('my_workspace/my_project') neptune.create_experiment() log_test_preds_proba(rfc, X_test, y_test)
-
neptunecontrib.monitoring.sklearn.
log_scores
(estimator, X, y, y_pred=None, name=None, experiment=None)¶ Log estimator scores on
X
.Calculate and log scores on data and have them as metrics in Neptune. If you pass
y_pred
, then predictions are not computed fromX
data.Estimator should be fitted before calling this function.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Regressor
For regressors that outputs single value, following scores are logged:
explained variance
max error
mean absolute error
r2
For multi-output regressor:
r2
Classifier
For classifier, following scores are logged:
precision
recall
f beta score
support
Tip
Check Neptune documentation for the full example.
- Parameters
estimator (
estimator
) –Scikit-learn estimator to compute scores.X (
ndarray
) –Data matrix.y (
ndarray
) –Target for testing.y_pred (
ndarray
, optional, default isNone
) –Estimator predictions on data.name (str, optional, default is
None
) –Use ‘train’, ‘valid’, ‘test’ to better define on what data scores are logged.experiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.
- Returns
None
Examples
rfc = RandomForestClassifier() rfc.fit(X_train, y_train) neptune.init('my_workspace/my_project') neptune.create_experiment() log_scores(rfc, X, y, name='test', experiment=exp)
-
neptunecontrib.monitoring.sklearn.
log_learning_curve_chart
(regressor, X_train, y_train, experiment=None)¶ Log learning curve chart.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
regressor (
regressor
) –Fitted sklearn regressor objectX_train (
ndarray
) –Training data matrixy_train (
ndarray
) –The regression target for trainingexperiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.
- Returns
None
Examples
rfr = RandomForestRegressor() rfr.fit(X_train, y_train) neptune.init('my_workspace/my_project') neptune.create_experiment() log_learning_curve_chart(rfr, X_train, y_train)
-
neptunecontrib.monitoring.sklearn.
log_feature_importance_chart
(regressor, X_train, y_train, experiment=None)¶ Log feature importance chart.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
regressor (
regressor
) –Fitted sklearn regressor objectX_train (
ndarray
) –Training data matrixy_train (
ndarray
) –The regression target for trainingexperiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.
- Returns
None
Examples
rfr = RandomForestRegressor() rfr.fit(X_train, y_train) neptune.init('my_workspace/my_project') neptune.create_experiment() log_feature_importance_chart(rfr, X_train, y_train)
-
neptunecontrib.monitoring.sklearn.
log_residuals_chart
(regressor, X_train, X_test, y_train, y_test, experiment=None)¶ Log residuals chart.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
regressor (
regressor
) –Fitted sklearn regressor objectX_train (
ndarray
) –Training data matrixX_test (
ndarray
) –Testing data matrixy_train (
ndarray
) –The regression target for trainingy_test (
ndarray
) –The regression target for testingexperiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.
- Returns
None
Examples
rfr = RandomForestRegressor() rfr.fit(X_train, y_train) neptune.init('my_workspace/my_project') exp = neptune.create_experiment() log_residuals_chart(rfr, X_train, X_test, y_train, y_test, experiment=exp)
-
neptunecontrib.monitoring.sklearn.
log_prediction_error_chart
(regressor, X_train, X_test, y_train, y_test, experiment=None)¶ Log prediction error chart.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
regressor (
regressor
) –Fitted sklearn regressor objectX_train (
ndarray
) –Training data matrixX_test (
ndarray
) –Testing data matrixy_train (
ndarray
) –The regression target for trainingy_test (
ndarray
) –The regression target for testingexperiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.
- Returns
None
Examples
rfr = RandomForestRegressor() rfr.fit(X_train, y_train) neptune.init('my_workspace/my_project') neptune.create_experiment() log_prediction_error_chart(rfr, X_train, X_test, y_train, y_test)
-
neptunecontrib.monitoring.sklearn.
log_cooks_distance_chart
(regressor, X_train, y_train, experiment=None)¶ Log feature importance chart.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
regressor (
regressor
) –Fitted sklearn regressor objectX_train (
ndarray
) –Training data matrixy_train (
ndarray
) –The regression target for trainingexperiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.
- Returns
None
Examples
rfr = RandomForestRegressor() rfr.fit(X_train, y_train) neptune.init('my_workspace/my_project') neptune.create_experiment() log_cooks_distance_chart(rfr, X_train, y_train)
-
neptunecontrib.monitoring.sklearn.
log_classification_report_chart
(classifier, X_train, X_test, y_train, y_test, experiment=None)¶ Log classification report chart.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
classifier (
classifier
) –Fitted sklearn classifier objectX_train (
ndarray
) –Training data matrixX_test (
ndarray
) –Testing data matrixy_train (
ndarray
) –The classification target for trainingy_test (
ndarray
) –The classification target for testingexperiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.
- Returns
None
Examples
rfc = RandomForestClassifier() rfc.fit(X_train, y_train) neptune.init('my_workspace/my_project') exp = neptune.create_experiment() log_classification_report_chart(rfc, X_train, X_test, y_train, y_test, experiment=exp)
-
neptunecontrib.monitoring.sklearn.
log_confusion_matrix_chart
(classifier, X_train, X_test, y_train, y_test, experiment=None)¶ Log confusion matrix.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
classifier (
classifier
) –Fitted sklearn classifier objectX_train (
ndarray
) –Training data matrixX_test (
ndarray
) –Testing data matrixy_train (
ndarray
) –The classification target for trainingy_test (
ndarray
) –The classification target for testingexperiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.
- Returns
None
Examples
rfc = RandomForestClassifier() rfc.fit(X_train, y_train) neptune.init('my_workspace/my_project') neptune.create_experiment() log_confusion_matrix_chart(rfc, X_train, X_test, y_train, y_test)
-
neptunecontrib.monitoring.sklearn.
log_roc_auc_chart
(classifier, X_train, X_test, y_train, y_test, experiment=None)¶ Log ROC-AUC chart.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
classifier (
classifier
) –Fitted sklearn classifier objectX_train (
ndarray
) –Training data matrixX_test (
ndarray
) –Testing data matrixy_train (
ndarray
) –The classification target for trainingy_test (
ndarray
) –The classification target for testingexperiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.
- Returns
None
Examples
rfc = RandomForestClassifier() rfc.fit(X_train, y_train) neptune.init('my_workspace/my_project') exp = neptune.create_experiment() log_roc_auc_chart(rfc, X_train, X_test, y_train, y_test, experiment=exp)
-
neptunecontrib.monitoring.sklearn.
log_precision_recall_chart
(classifier, X_test, y_test, y_pred_proba=None, experiment=None)¶ Log precision recall chart.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
classifier (
classifier
) –Fitted sklearn classifier objectX_test (
ndarray
) –Testing data matrixy_test (
ndarray
) –The classification target for testingy_pred_proba (
ndarray
, optional, default isNone
) –Classifier predictions probabilities on test data.experiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.
- Returns
None
Examples
rfc = RandomForestClassifier() rfc.fit(X_train, y_train) neptune.init('my_workspace/my_project') neptune.create_experiment() log_precision_recall_chart(rfc, X_test, y_test)
-
neptunecontrib.monitoring.sklearn.
log_class_prediction_error_chart
(classifier, X_train, X_test, y_train, y_test, experiment=None)¶ Log class prediction error chart.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
classifier (
classifier
) –Fitted sklearn classifier objectX_train (
ndarray
) –Training data matrixX_test (
ndarray
) –Testing data matrixy_train (
ndarray
) –The classification target for trainingy_test (
ndarray
) –The classification target for testingexperiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.
- Returns
None
Examples
rfc = RandomForestClassifier() rfc.fit(X_train, y_train) neptune.init('my_workspace/my_project') exp = neptune.create_experiment() log_class_prediction_error_chart(rfc, X_train, X_test, y_train, y_test, experiment=exp)
-
neptunecontrib.monitoring.sklearn.
log_kmeans_clustering_summary
(model, X, nrows=1000, experiment=None, **kwargs)¶ Log sklearn kmeans summary.
This method fit KMeans model to data and logs cluster labels, all kmeans parameters and clustering visualizations: KMeans elbow chart and silhouette coefficients chart.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
model (
KMeans
) –KMeans object.X (
ndarray
) –Training instances to cluster.nrows (int, optional, default is 1000) –
Number of rows to log in the cluster labelsexperiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.kwargs – KMeans parameters.
- Returns
None
Examples
km = KMeans(n_init=11, max_iter=270) X, y = make_blobs(n_samples=579, n_features=17, centers=7, random_state=28743) neptune.init('my_workspace/my_project') neptune.create_experiment() log_kmeans_clustering_summary(km, X=X)
-
neptunecontrib.monitoring.sklearn.
log_cluster_labels
(model, X, nrows=1000, experiment=None, **kwargs)¶ Log index of the cluster label each sample belongs to.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
model (
KMeans
) –KMeans object.X (
ndarray
) –Training instances to cluster.nrows (int, optional, default is 1000) –
Number of rows to log.experiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.kwargs – KMeans parameters.
- Returns
None
Examples
km = KMeans(n_init=11, max_iter=270) X, y = make_blobs(n_samples=579, n_features=17, centers=7, random_state=28743) neptune.init('my_workspace/my_project') neptune.create_experiment() log_cluster_labels(km, X=X)
-
neptunecontrib.monitoring.sklearn.
log_kelbow_chart
(model, X, experiment=None, **kwargs)¶ Log K-elbow chart for KMeans clusterer.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
model (
KMeans
) –KMeans object.X (
ndarray
) –Training instances to cluster.experiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.kwargs – KMeans parameters.
- Returns
None
Examples
km = KMeans(n_init=11, max_iter=270) X, y = make_blobs(n_samples=579, n_features=17, centers=7, random_state=28743) neptune.init('my_workspace/my_project') neptune.create_experiment() log_kelbow_chart(km, X=X)
-
neptunecontrib.monitoring.sklearn.
log_silhouette_chart
(model, X, experiment=None, **kwargs)¶ Log Silhouette Coefficients charts for KMeans clusterer.
Charts are computed for j = 2, 3, …, n_clusters.
Make sure you created an experiment by using
neptune.create_experiment()
before you use this method.Tip
Check Neptune documentation for the full example.
- Parameters
model (
KMeans
) –KMeans object.X (
ndarray
) –Training instances to cluster.experiment (
neptune.experiments.Experiment
, optional, default isNone
) –NeptuneExperiment
object to control to which experiment you log the data.IfNone
, log to currently active, and most recent experiment.kwargs – KMeans parameters.
- Returns
None
Examples
km = KMeans(n_init=11, max_iter=270) X, y = make_blobs(n_samples=579, n_features=17, centers=7, random_state=28743) neptune.init('my_workspace/my_project') neptune.create_experiment() log_silhouette_chart(km, X=X, n_clusters=12)
-
neptunecontrib.monitoring.sklearn.
_validate_experiment
(experiment)¶