Skip to content

tobegit3hub/advisor

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Nov 11, 2019
f450353 · Nov 11, 2019
Dec 6, 2018
Nov 4, 2019
Oct 18, 2018
Oct 11, 2018
Oct 11, 2018
Oct 4, 2018
Oct 2, 2018
Oct 2, 2018
Oct 1, 2018
Oct 2, 2018
Sep 14, 2017
Oct 11, 2018
Oct 2, 2018
Oct 2, 2018
Oct 18, 2018

Repository files navigation

Advisor

Introduction

Advisor is the hyper parameters tuning system for black box optimization.

It is the open-source implementation of Google Vizier with these features.

  • Easy to use with API, SDK, WEB and CLI
  • Support abstractions of Study and Trial
  • Included search and early stop algorithms
  • Recommend parameters with trained model
  • Same programming interfaces as Google Vizier
  • Command-line tool just like Microsoft NNI.

Supported Algorithms

  • Grid Search
  • Random Search
  • Bayesian Optimization
  • TPE(Hyperopt)
  • Random Search(Hyperopt)
  • Simulate Anneal(Hyperopt)
  • Quasi Random(Chocolate)
  • Grid Search(Chocolate)
  • Random Search(Chocolate)
  • Bayes(Chocolate)
  • CMAES(Chocolate)
  • MOCMAES(Chocolate)
  • SMAC Algorithm
  • Bayesian Optimization(Skopt)
  • Early Stop First Trial Algorithm
  • Early Stop Descending Algorithm
  • Performance Curve Stop Algorithm

Quick Start

It is easy to setup advisor service in local machine.

pip install advisor

advisor_admin server start

Then go to http://127.0.0.1:8000 in the browser and submit tuning jobs.

git clone --depth 1 https://github.com/tobegit3hub/advisor.git && cd ./advisor/

advisor run -f ./advisor_client/examples/python_function/config.json

advisor study describe -s demo

Advisor Server

Run server with official package.

advisor_admin server start

Or run with official docker image.

docker run -d -p 8000:8000 tobegit3hub/advisor

Or run with docker-compose.

wget https://raw.githubusercontent.com/tobegit3hub/advisor/master/docker-compose.yml

docker-compose up -d

Or run in Kubernetes cluster.

wget https://raw.githubusercontent.com/tobegit3hub/advisor/master/kubernetes_advisor.yaml

kubectl create -f ./kubernetes_advisor.yaml

Or run from scratch with source code.

git clone --depth 1 https://github.com/tobegit3hub/advisor.git && cd ./advisor/

pip install -r ./requirements.txt

./manage.py migrate

./manage.py runserver 0.0.0.0:8000

Advisor Client

Install with pip or use docker container.

pip install advisor

docker run -it --net=host tobegit3hub/advisor bash

Use the command-line tool.

export ADVISOR_ENDPOINT="http://127.0.0.1:8000"

advisor study list

advisor study describe -s "demo"

advisor trial list --study_name "demo"

Use admin tool to start/stop server.

advisor_admin server start

advisor_admin server stop

Use the Python SDK.

client = AdvisorClient()

# Create the study
study_configuration = {
        "goal": "MAXIMIZE",
        "params": [
                {
                        "parameterName": "hidden1",
                        "type": "INTEGER",
                        "minValue": 40,
                        "maxValue": 400,
                        "scalingType": "LINEAR"
                }
        ]
}
study = client.create_study("demo", study_configuration)

# Get suggested trials
trials = client.get_suggestions(study, 3)

# Complete the trial
trial = trials[0]
trial_metrics = 1.0
client.complete_trial(trial, trial_metrics)

Please checkout examples for more usage.

Configuration

Study configuration describe the search space of parameters. It supports four types and here is the example.

{
  "goal": "MAXIMIZE",
  "randomInitTrials": 1,
  "maxTrials": 5,
  "maxParallelTrials": 1,
  "params": [
    {
      "parameterName": "hidden1",
      "type": "INTEGER",
      "minValue": 1,
      "maxValue": 10,
      "scalingType": "LINEAR"
    },
    {
      "parameterName": "learning_rate",
      "type": "DOUBLE",
      "minValue": 0.01,
      "maxValue": 0.5,
      "scalingType": "LINEAR"
    },
    {
      "parameterName": "hidden2",
      "type": "DISCRETE",
      "feasiblePoints": "8, 16, 32, 64",
      "scalingType": "LINEAR"
    },
    {
      "parameterName": "optimizer",
      "type": "CATEGORICAL",
      "feasiblePoints": "sgd, adagrad, adam, ftrl",
      "scalingType": "LINEAR"
    },
    {
      "parameterName": "batch_normalization",
      "type": "CATEGORICAL",
      "feasiblePoints": "true, false",
      "scalingType": "LINEAR"
    }
  ]
}

Here is the configuration file in JSON format for advisor run.

{
  "name": "demo",
  "algorithm": "BayesianOptimization",
  "trialNumber": 10,
  "concurrency": 1,
  "path": "./advisor_client/examples/python_function/",
  "command": "./min_function.py",
  "search_space": {
      "goal": "MINIMIZE",
      "randomInitTrials": 3,
      "params": [
          {
              "parameterName": "x",
              "type": "DOUBLE",
              "minValue": -10.0,
              "maxValue": 10.0,
              "scalingType": "LINEAR"
          }
      ]
  }
}

Or use the equivalent configuration file in YAML format.

name: "demo"
algorithm: "BayesianOptimization"
trialNumber: 10
path: "./advisor_client/examples/python_function/"
command: "./min_function.py"
search_space:
  goal: "MINIMIZE"
  randomInitTrials: 3
  params:
    - parameterName: "x"
      type: "DOUBLE"
      minValue: -10.0
      maxValue: 10.0

Screenshots

List all the studies and create/delete the studies easily.

study_list.png

List the detail of study and all the related trials.

study_detail.png

List all the trials and create/delete the trials easily.

trial_list.png

List the detail of trial and all the related metrics.

trial_detail.png

Development

You can edit the source code and test without re-deploying the server and client.

git clone git@github.com:tobegit3hub/advisor.git

cd ./advisor/advisor_client/

python ./setup.py develop

export PYTHONPATH="/Library/Python/2.7/site-packages/:$PYTHONPATH"