DataCrunch Python SDK

The official DataCrunch.io Python SDK.

The SDK's documentation is available on ReadTheDocs

DataCrunch's Public API documentation is available here.

• Install via pip:

  pip3 install datacrunch

• Generate your client credentials - instructions in the public API docs.

• Add your client id and client secret to an environment variable (don't want it to be hardcoded):

  Linux (bash):

  export DATACRUNCH_CLIENT_ID=YOUR_ID_HERE
  export DATACRUNCH_CLIENT_SECRET=YOUR_SECRET_HERE

• To enable sending inference requests from SDK you must generate an inference key - Instructions on inference authorization

• Add your inference key to an environment variable

  Linux (bash):

  export DATACRUNCH_INFERENCE_KEY=YOUR_API_KEY_HERE

  Other platforms: https://en.wikipedia.org/wiki/Environment_variable

• Example for creating a new instance:

  import os
  from datacrunch import DataCrunchClient
  
  # Get credentials from environment variables
  CLIENT_ID = os.environ.get('DATACRUNCH_CLIENT_ID')
  CLIENT_SECRET = os.environ['DATACRUNCH_CLIENT_SECRET']
  
  # Create datcrunch client
  datacrunch = DataCrunchClient(CLIENT_ID, CLIENT_SECRET)
  
  # Get all SSH keys
  ssh_keys = datacrunch.ssh_keys.get()
  ssh_keys = list(map(lambda key: key.id, ssh_keys))
  
  # Create a new instance
  instance = datacrunch.instances.create(instance_type='1V100.6V',
                                        image='ubuntu-24.04-cuda-12.8-open-docker',
                                        ssh_key_ids=ssh_keys,
                                        hostname='example',
                                        description='example instance')
  
  # Delete instance
  datacrunch.instances.action(instance.id, datacrunch.constants.instance_actions.DELETE)

  More examples can be found in the /examples folder or in the documentation.

• Clone the repository:

  git clone

• Create local virtual environment:

  python3 -m venv datacrunch_env && source ./datacrunch_env/bin/activate

  or if using fish shell:

  python3 -m venv datacrunch_env && source ./datacrunch_env/bin/activate.fish

• Install Dependencies:

  pip3 install -e .[test]
  pip3 install -U pytest

We use pytest for testing.

• To execute all tests

  pytest

• To execute a single test file

  pytest ./tests/unit_tests/test_file.py

Create this file in the root directory of the project:

from datacrunch.datacrunch import DataCrunchClient

CLIENT_SECRET = 'secret'
CLIENT_ID = 'your-id'

# Create datcrunch client
datacrunch = DataCrunchClient(CLIENT_ID, CLIENT_SECRET, base_url='http://localhost:3001/v1')

If added a new service, create a documentation template under api/services for that service.

cd docs
make html

Use autopep8 for auto code formatting:

# Install
pip3 install autopep8

# Apply to an entire directory
autopep8 directory_name --recursive --in-place --pep8-passes 2000 --verbose

# Or a single file
autopep8 file.py --in-place

You can contact us here, or open an issue in the repo.