Neptune `0.x` API#

The 0.x client is deprecated

With the 1.0 release of the Neptune client library, we're ending active maintenance and support for the Neptune 0.x API.

From 1.0 onward, you must import the 0.x API with the following statement:

import neptune.legacy as neptune

We strongly encourage you to migrate to the new client at your earliest convenience.

Migrating to the `1.x` API#

This guide walks you through migrating your legacy code to the 1.x Python API.

Initialization#

0.x

import neptune.legacy as neptune

neptune.init(project_qualified_name="my-workspace/my-project")

neptune.create_experiment(tags=["resnet"])

1.x

import neptune

run = neptune.init_run(project="my-workspace/my-project", tags=["resnet"])

Note

The names of the environment variables have not changed.

Instead of specifying the project name or API token in the code, you can provide them by setting the NEPTUNE_PROJECT and NEPTUNE_API_TOKEN environment variables.

Hierarchical structure#

Runs can be viewed as nested dictionary-like structures that you can define in your code. This lets you organize your metadata in a way that is most convenient for you.

The hierarchical structure that you apply to your metadata is reflected when you view the metadata in the app.

The structure of a run consists of fields that are organized into namespaces.

The path of a field is a combination of the namespaces and its name: if you store a value (such as a float, 0.8) in a field named momentum under a namespace params, the full path of the field is params/momentum.

You can organize any type of metadata this way – such as images, parameters, metrics, scores, model checkpoints, and CSV files.

Let's look at the following code:

import neptune

run = neptune.init_run(project="my_workspace/my_project")

run["about/JIRA"] = "NPT-952"

run["parameters/batch_size"] = 5
run["parameters/algorithm"] = "ConvNet"

for epoch in range(100):
    acc_value = ...
    loss_value = ...
    run["train/accuracy"].append(acc_value)
    run["train/loss"].append(loss_value)

exp["trained_model"].upload("model.pt")

The resulting structure of the run will be:

"about":
    "JIRA": String
"parameters": 
    "batch_size": Float
    "algorithm": String
"train":
    "accuracy": FloatSeries
    "loss": FloatSeries
"trained_model": File

Batch assign#

You can assign values to multiple fields in batch by using a dictionary.

With the legacy API, you had to pass parameters when creating an experiment and it was not possible to change them afterward. In addition, nested dictionaries were not fully supported.

0.x

import neptune.legacy as neptune

PARAMS = {"epoch_nr": 100, "lr": 0.005, "use_nesterov": True}

neptune.init()
neptune.create_experiment(params=PARAMS)

With the new neptune API, it's up to you when and where you want to specify parameters. You can also update them later:

1.x

import neptune

PARAMS = {"epoch_nr": 100, "lr": 0.005, "use_nesterov": True}

run = neptune.init_run()
run["my_params"] = PARAMS

# You can also specify parameters one by one
run["my_params/batch_size"] = 64

# Update lr value
run["my_params/lr"] = 0.007

The artificial distinction between parameters and properties is also gone. You can log and access them in a single, unified way.

Interacting with files (artifacts)#

You are no longer bound to store files only in the artifacts folder.

Whether it's a model checkpoint, custom interactive visualization, or audio file, you can track it in the same hierarchical structure with the rest of the metadata:

0.x	1.x
`neptune.log_artifact("model_viz.png")`	`run[model/viz"].upload("model_viz.png")`
`neptune.log_artifact("model.pt")`	`run[trained_model"].upload("model.pt")`
`neptune.download_artifact("model.pt")`	`run["trained_model"].download()`

Note

In the legacy API, artifacts mimicked a file system. What you uploaded was saved under the same name, with the extension, etc.

The idea behind the new Python API is more database-like: We have a field (with a path) and under it, we store some content - Float, String, series of String, or File. In this model, the extension is part of the content.

Example: If under the path "model/last" you upload a .pt file, the file will be displayed as "last.pt" in the app.

When it's unambiguous, we implicitly convert an object to type File and there is no need for explicit conversion.

Example: For Matplotlib charts, you can do

run["conf_matrix"].upload(plt_fig)

instead of

run["conf_matrix"].upload(File.as_image(plt_fig))

neptune-contrib#

We've integrated neptune-contrib file-related functionalities into the core library.

The conversion methods are available as File factory methods:

Interactive charts (Altair, Bokeh, Plotly, Matplotlib)#

0.x	1.x
`from neptunecontrib.api import log_chart`	`from neptune.types import File`
`log_chart("int_chart", chart)`	`run["int_chart"].upload(File.as_html(chart))`

pandas DataFrame#

0.x	1.x
`from neptunecontrib.api import log_table`	`from neptune.types import File`
`log_table("pred_df", df)`	`run["pred_df"].upload(File.as_html(df))`

Audio and video#

We've expanded the range of files that are natively supported in the Neptune app, so for audio and video files you no longer need to use conversion methods:

0.x 1.x

from neptunecontrib.api import log_audio

log_audio("sample.mp3")

run["sample"].upload("sample.mp3")

from neptunecontrib.api import log_video

log_audio("sample.mp4")

run["sample"].upload("sample.mp4")

Pickled objects#

0.x	1.x
`from neptunecontrib.api import log_pickle`	`from neptune.types import File`
`log_pickle("model.pkl", model)`	`run["model"].upload(File.as_pickle(model))`

HTML strings#

0.x	1.x
`from neptunecontrib.api import log_html`	`from neptune.types import File`
`log_html("custom_viz", html_string)`	`run["custom_viz"].upload(File.from_content(model), extension="html")`

Scores and metrics#

Logging metrics is quite similar, except that you can now organize them in a hierarchical structure:

0.x	1.x
`neptune.log_metric("acc", 0.97)`	`run["acc"].append(0.97)`
`neptune.log_metric("train_acc", 0.97)`	`run["train/acc"].append(0.97)`
`neptune.log_metric("loss", 0.8)`	`run["key"].append()`

To log scores you don't need to use Series fields anymore as you can track single values anytime, anywhere:

0.x	1.x
`neptune.log_metric("final_accuracy", 0.8)`	`run["final_accuracy"] = 0.8`

Text and image series#

Similar changes need to be applied for text and image series:

0.x	1.x
`neptune.log_text("train_log", custom_log_msg)`	`run["train/log"].append(custom_log_msg)`
`neptune.log_image("misclassified", filepath)`	`run["misclassified"].append(File(filepath))`
`neptune.log_image("pred_dist", hist_chart)`	`run["pred_dist"].append(hist_chart)`

To add a single image that you want to view with the rest of the metrics you no longer need to use Series fields. As you control whether they are grouped in the same namespace you can upload it as a single File field.

0.x	1.x
`neptune.log_image("train_hist", hist_chart)`	`run["train/hist"].upload(hist_chart)`

Integrations#

We've redesigned how our integrations work:

They're more tightly integrated with our base library and the API is more unified.
You can update the version of each integration separately in case of dependency conflicts.
There's a smaller number of dependencies if you're not using all integrations.

There is no longer a neptune-contrib library for the new Python API – instead, each integration has two parts:

Boilerplate code for ease of use in the main library:

import neptune.integrations.framework_name
The actual integration that can be installed separately:

pip install neptune-framework-name

or as an extra together with neptune:

pip install "neptune[framework-name]"

Existing integrations from neptune-contrib are still fully functional. You can use them both with projects using the previous structure and the new structure. However, integrations from neptune-contrib use the legacy Python API, while the new integrations are rewritten to fully leverage the new Python API for better metadata organization.

Let's look at how this works like in the case of the Keras integration:

Installation

0.x 1.x

pip install neptune-contrib

pip install "neptune[tensorflow-keras]"

or

pip install neptune-tensorflow-keras

Usage

0.x

import neptune.legacy as neptune

neptune.init()

from neptunecontrib.monitoring.keras import NeptuneMonitor

model.fit(
    x_train,
    y_train,
    epochs=5,
    batch_size=64,
    callbacks=[NeptuneMonitor()],
)

1.x

import neptune

run = neptune.init_run()

from neptune.integrations.tensorflow_keras import NeptuneCallback

model.fit(
    x_train,
    y_train,
    epochs=5,
    batch_size=64,
    callbacks=[NeptuneCallback(run=run)],
)

Tags#

Interaction with tags is quite similar: tags are stored as a StringSet field under the sys/tags field.

0.x

import neptune.legacy as neptune

neptune.init()

neptune.create_experiment(params=PARAMS, tags=["maskRCNN"])

neptune.append_tag("prod_v2.0.1")
neptune.append_tags("finetune", "keras")

1.x

import neptune

run = neptune.init_run(tags=["maskRCNN"])

run["sys/tags"].add("prod_v2.0.1")
run["sys/tags"].add(["finetune", "keras"])

Logging to a project with 0.x runs#

If you have a project with a set of runs (previously Experiments) that were created using the 0.x API and you would like to add runs using the 1.x API, there are a few things to remember.

Metadata structure#

Each type of metadata from the previous version (parameters, properties, logs, artifacts, and monitoring) are mapped into a separate fixed namespace. If you want to be able to compare those, you need to make sure that the field path is backward compatible:

0.x	1.x
`neptune.create_experiment(params=params_dict)`	`run["parameters"] = params_dict`
`neptune.set_property("SEED", 2458)`	`run["properties/SEED"] = str(2458)`
`neptune.log_metric("acc", 0.98)`	`run["logs/acc"].append(0.98)`
`neptune.log_artifact("model.pt", "model.pt")`	`run["artifacts/model"].upload("model.pt")`

Data types#

Neptune is type-sensitive. For example, a column in the experiments table represents a field with a specific path and type. In particular, FloatSeries and Float are two different types and Neptune by default doesn't know how to compare them.

With the 0.x API, it wasn't possible to set Float fields outside of the create_experiment() function, and a common practice was to store those as FloatSeries with one entry through the use of log_metric() function. To achieve backward compatibility, you need to create a FloatSeries with the new API as well, by using the append() method.

Properties set through 0.x API are always String fields. To achieve backward compatibility, use assignment with explicit casting to str to ensure that the resulting field will be a string.

Neptune 0.x API#

Migrating to the 1.x API#