Experiment Groups

Caliban supports grouping experiments into a collection called an experiment group. This allows you to do things like monitor all of the jobs in a given group, stop all running jobs in a group, or re-run all of the jobs in a group.

Each of the caliban compute backends supports specifying an experiment group via the --xgroup flag:

$ caliban run --xgroup my-xgroup ...
$ caliban cloud --xgroup my-xgroup ...
$ caliban cluster job submit --xgroup my-xgroup ...

If you don’t specify an experiment group when submitting jobs via caliban, a new experiment group will be generated for you, so you don’t need to use them if you don’t want to. Also, the existence of this group should be transparent to you.

You can add new jobs to an existing experiment group simply by specifying the same group on different caliban job submission calls:

caliban cloud --xgroup my-xgroup ... foo.py --
...
(some time later...)
caliban cloud --xgroup my-xgroup ... bar.py --

The experiment group my-xgroup will contain the experiments generated by both of the caliban calls, and you can then perform different operations on these as described in the sections below.