This repository contains various examples for running distributed training jobs on the OICM+ Platform using PyTorch and Ray. The examples are designed to demonstrate how to effectively use distributed data parallelism, model sharding, and multi-GPU training techniques within the platform.
The repository is organized into the following directories, each containing specific examples and configurations:
presistent_volume/
├── sample/
Pytorch/
├── pytorch_ddp/
├── pytorch_fsdp/
ray/
├── fsdp/
├── ray_ddp/
└── ray_mdp/
tensorflow/
├── parameter_server/