Skip to content

About

Stacktape starter project

Resources

Stars

1 star

Watchers

1 watching

Forks

Latest commit

 

History

1 Commit

Folders and files

Repository files navigation

GPU Batch Job (TensorFlow)

Tip

To deploy this project using GUI-based flow, navigate to console

  • GPU-capable batch job for image classification using TensorFlow.
  • Upload a .zip of images to the input S3 bucket — the job automatically classifies them using a pre-trained EfficientNet model and saves results to the output bucket.
  • Uses spot instances for up to 90% cost savings. Perfect as a starting point for any ML batch processing workload.
  • This project includes a pre-configured stacktape.yml configuration. The configured infrastructure is described in the stack description section

Prerequisites

  1. AWS account. If you don't have one, create new account here.

  2. Stacktape account. If you don't have one, create new account here.

  3. Stacktape installed.

Install on Windows (Powershell)
iwr https://installs.stacktape.com/windows.ps1 -useb | iex
Install on Linux
curl -L https://installs.stacktape.com/linux.sh | sh
Install on MacOS
curl -L https://installs.stacktape.com/macos.sh | sh
Install on MacOS ARM (Apple silicon)
curl -L https://installs.stacktape.com/macos-arm.sh | sh

1. Generate your project

To initialize the project, use

stacktape init --starterId tensorflow-image-classification

2. Before deploy

3. Deploy your stack

The deployment will take ~5-15 minutes. Subsequent deploys will be significantly faster.

Deploy from local machine

The deployment from local machine will build and deploy the application from your system. This means you also need to have:


To perform the deployment, use the following command:

stacktape deploy --projectName <<project-name>> --stage <<stage>> --region <<region>>

stage is an arbitrary name of your environment (for example staging, production or dev-john)

region is the AWS region, where your stack will be deployed to. All the available regions are listed below.

projectName is the name of your project. You can create it in the console or interactively using CLI.


Region name & Location code
Europe (Ireland) eu-west-1
Europe (London) eu-west-2
Europe (Frankfurt) eu-central-1
Europe (Milan) eu-south-1
Europe (Paris) eu-west-3
Europe (Stockholm) eu-north-1
US East (Ohio) us-east-2
US East (N. Virginia) us-east-1
US West (N. California) us-west-1
US West (Oregon) us-west-2
Canada (Central) ca-central-1
Africa (Cape Town) af-south-1
Asia Pacific (Hong Kong) ap-east-1
Asia Pacific (Mumbai) ap-south-1
Asia Pacific (Osaka-Local) ap-northeast-3
Asia Pacific (Seoul) ap-northeast-2
Asia Pacific (Singapore) ap-southeast-1
Asia Pacific (Sydney) ap-southeast-2
Asia Pacific (Tokyo) ap-northeast-1
China (Beijing) cn-north-1
China (Ningxia) cn-northwest-1
Middle East (Bahrain) me-south-1
South America (São Paulo) sa-east-1
Deploy using AWS CodeBuild pipeline

Deployment using AWS CodeBuild will build and deploy your application inside AWS CodeBuild pipeline. To perform the deployment, use

stacktape codebuild:deploy --stage <<stage>> --region <<region>> --projectName <<project-name>>

stage is an arbitrary name of your environment (for example staging, production or dev-john)

region is the AWS region, where your stack will be deployed to. All the available regions are listed below.

projectName is the name of your project. You can create it in the console or interactively using CLI.


Region name & Location code
Europe (Ireland) eu-west-1
Europe (London) eu-west-2
Europe (Frankfurt) eu-central-1
Europe (Milan) eu-south-1
Europe (Paris) eu-west-3
Europe (Stockholm) eu-north-1
US East (Ohio) us-east-2
US East (N. Virginia) us-east-1
US West (N. California) us-west-1
US West (Oregon) us-west-2
Canada (Central) ca-central-1
Africa (Cape Town) af-south-1
Asia Pacific (Hong Kong) ap-east-1
Asia Pacific (Mumbai) ap-south-1
Asia Pacific (Osaka-Local) ap-northeast-3
Asia Pacific (Seoul) ap-northeast-2
Asia Pacific (Singapore) ap-southeast-1
Asia Pacific (Sydney) ap-southeast-2
Asia Pacific (Tokyo) ap-northeast-1
China (Beijing) cn-north-1
China (Ningxia) cn-northwest-1
Middle East (Bahrain) me-south-1
South America (São Paulo) sa-east-1
Deploy using Github actions CI/CD pipeline
  1. If you don't have one, create a new repository at https://github.com/new
  2. Create Github repository secrets: https://docs.stacktape.com/user-guides/ci-cd/#2-create-github-repository-secrets
  3. Replace <<stage>> and <<region>> in the .github/workflows/deploy.yml file.
  4. git init --initial-branch=main
  5. git add .
  6. git commit -m "setup stacktape project"
  7. git remote add origin git@github.com:<<namespace-name>>/<<repo-name>>.git
  8. git push -u origin main
  9. To monitor the deployment progress, navigate to your github project and select the Actions tab

stage is an arbitrary name of your environment (for example staging, production or dev-john)

region is the AWS region, where your stack will be deployed to. All the available regions are listed below.

projectName is the name of your project. You can create it in the console or interactively using CLI.


Region name & Location code
Europe (Ireland) eu-west-1
Europe (London) eu-west-2
Europe (Frankfurt) eu-central-1
Europe (Milan) eu-south-1
Europe (Paris) eu-west-3
Europe (Stockholm) eu-north-1
US East (Ohio) us-east-2
US East (N. Virginia) us-east-1
US West (N. California) us-west-1
US West (Oregon) us-west-2
Canada (Central) ca-central-1
Africa (Cape Town) af-south-1
Asia Pacific (Hong Kong) ap-east-1
Asia Pacific (Mumbai) ap-south-1
Asia Pacific (Osaka-Local) ap-northeast-3
Asia Pacific (Seoul) ap-northeast-2
Asia Pacific (Singapore) ap-southeast-1
Asia Pacific (Sydney) ap-southeast-2
Asia Pacific (Tokyo) ap-northeast-1
China (Beijing) cn-north-1
China (Ningxia) cn-northwest-1
Middle East (Bahrain) me-south-1
South America (São Paulo) sa-east-1
Deploy using Gitlab CI pipeline
  1. If you don't have one, create a new repository at https://gitlab.com/projects/new
  2. Create Gitlab repository secrets: https://docs.stacktape.com/user-guides/ci-cd/#2-create-gitlab-repository-secrets
  3. replace <<stage>> and <<region>> in the .gitlab-ci.yml file.
  4. git init --initial-branch=main
  5. git add .
  6. git commit -m "setup stacktape project"
  7. git remote add origin git@gitlab.com:<<namespace-name>>/<<repo-name>>.git
  8. git push -u origin main
  9. To monitor the deployment progress, navigate to your gitlab project and select CI/CD->jobs

stage is an arbitrary name of your environment (for example staging, production or dev-john)

region is the AWS region, where your stack will be deployed to. All the available regions are listed below.

projectName is the name of your project. You can create it in the console or interactively using CLI.


Region name & Location code
Europe (Ireland) eu-west-1
Europe (London) eu-west-2
Europe (Frankfurt) eu-central-1
Europe (Milan) eu-south-1
Europe (Paris) eu-west-3
Europe (Stockholm) eu-north-1
US East (Ohio) us-east-2
US East (N. Virginia) us-east-1
US West (N. California) us-west-1
US West (Oregon) us-west-2
Canada (Central) ca-central-1
Africa (Cape Town) af-south-1
Asia Pacific (Hong Kong) ap-east-1
Asia Pacific (Mumbai) ap-south-1
Asia Pacific (Osaka-Local) ap-northeast-3
Asia Pacific (Seoul) ap-northeast-2
Asia Pacific (Singapore) ap-southeast-1
Asia Pacific (Sydney) ap-southeast-2
Asia Pacific (Tokyo) ap-northeast-1
China (Beijing) cn-north-1
China (Ningxia) cn-northwest-1
Middle East (Bahrain) me-south-1
South America (São Paulo) sa-east-1

4. Test your application

After a successful deployment, some information about the stack will be printed to the terminal (URLs of the deployed services, links to logs, metrics, etc.).

  • Upload a .zip file containing images to the input bucket (bucket name is printed after deploy). You can use the included pictures/test-pictures.zip or your own images.
    aws s3 cp pictures/test-pictures.zip s3://<INPUT_BUCKET_NAME>/test-pictures.zip
  • The batch job triggers automatically on upload. Classified images appear in the output bucket, organized into folders by predicted class.
  • Monitor job progress in the AWS Batch console or CloudWatch Logs.

5. Hotswap deploys

  • Stacktape deployments use AWS CloudFormation under the hood. It brings a lot of guarantees and convenience, but can be slow for certain use-cases.

  • To speed up the deployment, you can use the --hotSwap flag which avoids using Cloudformation.

  • Hotswap deployments work only for source code changes (for lambda function, containers and batch jobs) and for content uploads to buckets.

  • If the update deployment is not hot-swappable, Stacktape will automatically fall back to using a Cloudformation deployment.

stacktape deploy --hotSwap --stage <<stage>> --region <<region>> --projectName <<project-name>>

6. Delete your stack

  • If you no longer want to use your stack, you can delete it.
  • Stacktape will automatically delete every infrastructure resource and deployment artifact associated with your stack.
stacktape delete --stage <<stage>> --region <<region>>

Stack description

Stacktape uses a simple stacktape.yml configuration file to describe infrastructure resources, packaging, deployment pipeline and other aspects of your project.

You can deploy your project to multiple environments (stages) - for example production, staging or dev-john. A stack is a running instance of an project. It consists of your application code (if any) and the infrastructure resources required to run it.

The configuration for this project is described below.

1. Resources

  • Every resource must have an arbitrary, alphanumeric name (A-z0-9).
  • Stacktape resources consist of multiple underlying AWS or 3rd party resources.

1.1 Buckets

Two S3 buckets handle the data pipeline:

  • inputBucket — upload .zip archives of images here to kick off classification.
  • outputBucket — the batch job writes classified images here, organized into folders by predicted class.

Both use the default bucket configuration.

inputBucket:
  type: bucket

outputBucket:
  type: bucket

1.2 Batch job

The classifier runs as a GPU-capable batch job using spot instances for up to 90 % cost savings.

  • S3 event trigger — an s3:ObjectCreated event on the input bucket (filtered to .zip suffix) starts the job automatically. No polling, no cron — just upload and go.
  • Container packaging — uses a custom Dockerfile based on the TensorFlow image. Stacktape builds the image and pushes it to a managed ECR repository. You can also use other packaging types.
  • connectTo — listing inputBucket and outputBucket in connectTo makes Stacktape automatically create the required IAM permissions and inject environment variables (bucket names, ARNs) into the container. No manual policy writing needed.
  • Resources & spot — cpu and memory define the instance size AWS provisions on demand. Setting useSpotInstances: true opts into spot capacity, cutting costs significantly for fault-tolerant ML workloads.
  • Swap the model — while this starter uses TensorFlow + EfficientNet, you can replace it with any ML framework (PyTorch, JAX, etc.) by changing the Dockerfile and source code. The infrastructure stays the same.
classifierJob:
  type: batch-job
  properties:
    container:
      packaging:
        type: custom-dockerfile
        properties:
          buildContextPath: ./
    resources:
      cpu: 2
      memory: 14000
    useSpotInstances: true
    connectTo:
      - inputBucket
      - outputBucket
    events:
      - type: s3
        properties:
          bucketArn: $ResourceParam('inputBucket', 'arn')
          s3EventType: s3:ObjectCreated:*
          filterRule:
            suffix: .zip

About

Stacktape starter project

Resources

Stars

1 star

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages