Setting up Google Cloud¶
This document will guide you through the process of configuring your machine to submit jobs to Google’s Cloud AI Platform with caliban cloud.
Caliban will eventually support other Cloud providers like AWS, but Google Cloud will remain a great starting option. Google Cloud gives you $300 of credit, so you can get started immediately with caliban cloud.
These are the steps we’ll cover:
To make it through, you’ll need to know:
Know how to save environment variables on your machine by saving them in your
To submit jobs to AI Platform you need to create a Google Cloud account with an active “project”. Every job you submit to Cloud will be associated with this project.
Visit the Google Cloud Console and click “Select a Project”:
Click “New Project” in the dialogue that appears:
Give your project a memorable name like “totoro-lives” and click Create. This should take you to your new project’s dashboard.
Note the Project ID in the “Project info” panel:
Caliban will use this project ID to submit jobs to the correct project in Cloud.
Add the following line to the file
~/.bash_profile on your
machine, to make the ID available to Caliban:
If you don’t know what this means, see this page for a tutorial on environment variables. We’ll remind you at the end of the tutorial which variables you’ll need.
Every new Google Cloud project comes with a free $300 credit. To activate this, click “Activate” at the top right of your new Cloud account’s console and follow the prompts.
You’ll have to set up a billing account as well.
The system will ask you for a credit card to verify your identity, but if you use up the entire credit it won’t automatically charge you. You can decide at that point whether you’d like to continue or not.
Follow the instructions at this link to enable the Container Registry API by selecting the project you created above and clicking “Continue”.
Click this link to Enable the AI Platform Jobs API by clicking “Enable API” and waiting for the spinner to stop.
The final step is to install the Google Cloud SDK on your machine.
Visit the Google Cloud SDK installation page for a full set of installation instructions. Here is the distilled version:
If you didn’t do it during installation, you’ll need to initialize the SDK with this command:
When you see this output:
You are logged in as: [firstname.lastname@example.org]. This account has a lot of projects! Listing them all can take a while.  Enter a project ID  Create a new project  List projects Please enter your numeric choice:
1, then type in your project ID you noted earlier. (You should have
saved it as an environment variable called
If you’d like to set a default zone, anything beginning with
a great choice.
us-central1 has the most capability of any region.
the Cloud SDK is quite powerful, and gives you access to Cloud buckets and all sorts of Google services. You might want to peruse the full set of SDK documentation once you’ve got everything working.
Configure Docker Authentication¶
To submit a job with caliban cloud, Caliban needs to push Docker images to the Container Registry service that you enabled earlier. To allow Docker to push to the Container Registry, run this command at a terminal:
gcloud auth configure-docker
You should see output that includes the text
Adding credentials for all GCR
To check if your SDK installation was successful, run
gcloud auth list in
your terminal. You should see your email address listed as the active account:
[totoro@totoro ~]$ gcloud auth list Credentialed Accounts ACTIVE ACCOUNT * email@example.com To set the active account, run: $ gcloud config set account `ACCOUNT`
As a final step, confirm that you’ve set the following environment variables.
(If you set a custom region above, add it here as a
export REGION="us-central1" export PROJECT_ID="research-3141"
If you have all of this, you’re set!
Now that you have a working Cloud configuration and a new project, you can use caliban cloud to submit jobs to Cloud AI platform.
The “Getting Started with Caliban” tutorial ends with a nice demo that has you training models in Cloud. Head over to the tutorial and complete the final step to train a digit-classifying neural network on AI Platform.