Amigo is a python library that is designed for solving multidisciplinary analysis and optimization problems with high-performance computing resources through automatically generated c++ wrappers.
All application code is written in python and automatically compiled to c++. Automatic differentiation is used throughout to evaluate first and second derivatives using A2D. Different backend implementations are used depending on the computational environment: Serial, OpenMP, MPI and CUDA implementations can be used. The user python code and the model construction is independent of the target backend.
Integration with other MDO libraries is key for flexibility. Amigo contains interfaces to inject OpenMDAO models into Amigo models using amigo.ExternalComponent. Alternatively, Amigo can be used as a sub-optimization OpenMDAO component with accurate post-optimality derivatives.
A tutorial on how to use amigo and documentation can be found here: https://smdogroup.github.io/amigo/.
The quickest way to get a working build on Linux, macOS or Windows is pixi, which installs the compiler toolchain, MPI, BLAS/LAPACK, METIS and MUMPS into a project-local environment:
pixi install
pixi run install-amigo
pixi creates a reproducible, project-local conda environment
from pixi.toml and pixi.lock. It handles the dependencies that are otherwise
awkward to install by hand — MPI, BLAS/LAPACK, METIS and MUMPS — on Linux,
macOS (Apple Silicon) and Windows.
Nothing is installed system-wide: the environment lives in .pixi/envs/ inside
the repository, and pixi.lock pins exact package versions for every platform,
so a checkout builds the same way on every machine.
# Linux / macOS
curl -fsSL https://pixi.sh/install.sh | sh
# Windows (PowerShell)
powershell -ExecutionPolicy ByPass -c "irm -useb https://pixi.sh/install.ps1 | iex"
From the repository root:
pixi install # create the environment (first run downloads packages)
pixi run install-amigo # clone a2d if needed, then build and install amigo
pixi run install-amigo performs an editable install (pip install -e .), so
Python changes take effect immediately; re-run it after changing C++ sources.
| Platform | What you need beforehand |
|---|---|
linux-64 |
Nothing. The compiler (cxx-compiler), OpenMPI, OpenBLAS, METIS and MUMPS all come from conda-forge. |
osx-arm64 |
Nothing. Clang, llvm-openmp, OpenMPI, OpenBLAS, METIS and MUMPS come from conda-forge. |
win-64 |
Visual Studio Build Tools with the C++ workload. conda-forge cannot ship MSVC, so it must be installed system-wide. |
Amigo's Python extension has to be built with the same compiler and C runtime as CPython itself, which on Windows means MSVC. conda-forge cannot redistribute it, so it is the one dependency pixi cannot provide. You do not need the full Visual Studio IDE — the free standalone Build Tools are enough.
Run this from PowerShell:
winget install --id Microsoft.VisualStudio.2022.BuildTools -e --accept-package-agreements --accept-source-agreements --override "--quiet --wait --norestart --add Microsoft.VisualStudio.Workload.VCTools --includeRecommended"
What to expect:
- The
--overridestring must stay on one line. If your terminal wraps it across lines,--addarrives without its workload and the installer exits with code 87 (Installer failed with exit code: 87). Microsoft.VisualStudio.Workload.VCToolsis the C++ build-tools workload;--includeRecommendedpulls in the MSVC toolchain and the Windows SDK, both of which CMake needs. Without the SDK you get a compiler that cannot link.- It downloads roughly 2 GB and takes several minutes with no progress output
because of
--quiet. It is not hung —--waitkeeps winget blocked until the install finishes. - Windows may prompt for elevation. A reboot is not usually required
(
--norestart), but restart the terminal so the new tools are picked up.
Verify:
winget list --id Microsoft.VisualStudio.2022.BuildTools
You do not need a "Developer Command Prompt" or any vcvarsall.bat setup —
CMake locates MSVC on its own through the Visual Studio generator, so
pixi run install-amigo works from an ordinary shell. A successful configure
prints something like:
-- The CXX compiler identification is MSVC 19.44.35228.0
-- Check for working CXX compiler: .../VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/cl.exe
If you already have Visual Studio 2019 or 2022 Community/Professional with the "Desktop development with C++" workload, that works too — nothing extra to install.
pixi.toml defines three environments. Select one with -e; the default
environment needs no flag.
| Environment | Command | Contents |
|---|---|---|
default |
pixi run <task> |
Everything needed to build, install and run amigo: compiler toolchain, CMake, MPI, BLAS/LAPACK, METIS, MUMPS, and amigo's Python dependencies. |
dev |
pixi run -e dev <task> |
default plus pytest, smt, black, pre-commit and OpenMDAO. Use this to run the test suite or the OpenMDAO examples. |
cuda |
pixi run -e cuda <task> |
default plus the dev tools, the CUDA 12 toolkit and cuDSS. linux-64 only, and requires an NVIDIA driver supporting CUDA 12. |
Examples:
pixi run -e dev test # run the full test suite
pixi run -e cuda install-amigo-cuda # build with CUDA + cuDSS
pixi shell -e dev # drop into an interactive shell
pixi shell activates the environment in your current terminal, which is
useful for running ad-hoc scripts or pointing an IDE at
.pixi/envs/<env>/python.
| Task | What it does |
|---|---|
a2d |
Clones smdogroup/a2d into ../a2d if it is not already there. CMakeLists.txt expects the headers at that path. Every install task depends on this. |
install-amigo |
Editable install with METIS enabled, CUDA disabled, and OpenMP on (off on Windows, matching CI). |
install-amigo-cuda |
cuda environment only: editable install with CUDA, cuDSS and CMAKE_CUDA_ARCHITECTURES=native. |
test |
Runs pytest tests/ -v, installing amigo first. Use with -e dev. |
lint / format |
black --check . / black . |
clean |
Removes build directories and __pycache__. |
Because tasks declare their dependencies, pixi run test will install amigo
(and clone a2d) automatically if that has not happened yet.
- MPI — OpenMPI on Linux/macOS, MS-MPI on Windows, with
mpi4pybuilt against it. mpi4py's headers are needed at build time byamigo/amigo.cpp. - BLAS / LAPACK — OpenBLAS on Linux/macOS; MKL on Windows, which
CMakeLists.txtauto-detects at<sys.prefix>/Library/lib/mkl_rt.lib. - METIS —
METIS_ROOTis exported into the environment, socmake/Modules/FindMETIS.cmakepicks it up without any-Dflags. - MUMPS —
mumps-seqfrom conda-forge, withMUMPS_LIB_DIRpointing at the environment's library directory. This replaces the manual coin-or/ThirdParty-Mumps build described in the main README. Not available on Windows (see below). - CMake and the compiler at run time —
model.build_module()compiles generated C++ while your script runs, so CMake, Ninja, pybind11 and a compiler are part of the runtime environment, not just the build. - a2d headers — fetched by the
a2dtask.
Windows: no MUMPS. conda-forge has no reliable MUMPS build for Windows, so
the mumps solver option is unavailable — the same limitation as the Windows CI
job, which skips tests/functional. If you build coin-or/ThirdParty-Mumps
under MSYS2 as described in the main README, amigo's loader finds it in
~/mumps-coinor/bin or in %CONDA_PREFIX%\Library\bin.
Windows: OpenMP is off. install-amigo passes
-DAMIGO_ENABLE_OPENMP=OFF on win-64 to match CI. Remove that flag from the
[target.win-64.tasks.install-amigo] command if you want to try it.
Windows: scripts/pixi-activate.bat. conda-forge's msmpi package sets
MSMPI_INC and friends from %PREFIX%, a variable that only exists during a
conda build. Under pixi it expands to nothing, leaving
MSMPI_INC=\Library\include, and CMake's FindMPI then constructs a broken
MPI::MPI_CXX target that makes every try_compile fail. The activation
script re-derives those paths from %CONDA_PREFIX%. It has to be a script
rather than an activation.env entry, because pixi does not expand
$CONDA_PREFIX inside activation.env values on Windows.
Paths are passed through the environment, not the command line. Windows
paths contain backslashes, and CMake treats those as escape characters in
-D arguments, so the install tasks deliberately pass no paths.
Commit pixi.toml and pixi.lock — the lock file is what makes builds
reproducible. .pixi/ is generated and should stay out of git.
CMAKE_CXX_COMPILER not set / 'nmake' '-?' failed (Windows) — the VS
Build Tools are missing or lack the C++ workload. See
Installing MSVC on Windows, then re-run
pixi run install-amigo.
Imported target "MPI::MPI_CXX" includes non-existent path "/Library/include"
(Windows) — the activation script did not run. Confirm that
scripts/pixi-activate.bat exists and that pixi run python -c "import os; print(os.environ['MSMPI_INC'])" prints an absolute path.
METIS not found: disabling METIS support — the pixi environment exports
METIS_ROOT, so this should not happen; a configure in the pixi environment
prints METIS found: enabling AMIGO_USE_METIS. If you see it anyway, check
that pixi run python -c "import os; print(os.environ['METIS_ROOT'])" prints
an absolute path, and that a stale CMake cache is not holding empty
METIS_INCLUDE_DIR / METIS_LIBRARY values — pixi run clean clears it.
amigo builds and runs correctly without METIS; the effect is on ordering
performance for large problems.
Stale build after changing C++ sources — run pixi run clean followed by
pixi run install-amigo.
Amigo uses CMake and scikit-build to build the primary amigo module and all model modules that comprise a multidisciplinary model.
To build and install the primary amigo module and its python wrappers, you can build the module with
pip install -e .
By default the OpenMP and CUDA parallelization are turned off. You can turn on or off these modules with additional command line arguments to pip. For an OpenMP and MPI install, use the pip command
pip install -e . -v \
-Ccmake.args="-DCMAKE_CXX_COMPILER=mpicxx" \
-Ccmake.args="-DAMIGO_ENABLE_OPENMP=ON" \
-Ccmake.args="-DAMIGO_ENABLE_CUDA=OFF"
For CUDA, we recommend using the NVIDIA CUDSS library that amigo can use to solve linear systems. Locally installing CUDSS is straightforward. It can be downloaded from NVIDIA webpage. Add the $CUDSS_HOME/lib and $CUDSS_HOME/lib64 to your LD_LIBRARY_PATH.
To enable CUDA and CUDSS and disable OpenMP, use the pip command
pip install -e . -v \
-Ccmake.args="-DCMAKE_CXX_COMPILER=mpicxx" \
-Ccmake.args="-DAMIGO_ENABLE_OPENMP=OFF" \
-Ccmake.args="-DAMIGO_ENABLE_CUDA=ON" \
-Ccmake.args="-DCUDSS_HOME=/path/to/cudss" \
-Ccmake.args="-DAMIGO_ENABLE_CUDSS=ON" \
-Ccmake.args="-DCMAKE_CUDA_ARCHITECTURES=native"
CUDA and OpenMP can be enabled at the same time. All component computations will utilize the GPU and the native Amigo solver will use OpenMP.
Amigo model modules inherit the build options that are selected during the install phase.
Installing Metis is recommended for good performance. Metis can be enabled through the command line via
pip install -e . -v \
-Ccmake.args="-DCMAKE_CXX_COMPILER=mpicxx" \
-Ccmake.args="-DAMIGO_ENABLE_OPENMP=ON" \
-Ccmake.args="-DAMIGO_ENABLE_CUDA=OFF" \
-Ccmake.args="-DAMIGO_ENABLE_METIS=ON" \
-Ccmake.define.METIS_INCLUDE_DIR=/path/to/metis/include/ \
-Ccmake.define.METIS_LIBRARY=/path/to/metis/compiled/library/libmetis.a
Amigo's interior-point optimizer can use MUMPS to obtain symmetric indefinite factorization of the KKT system. MUMPS is loaded at runtime, so it is not needed at build time. We use coin-or/ThirdParty-Mumps to build from source on all platforms, installing to $HOME/mumps-coinor so the Amigo loader finds the library automatically without any extra environment variables.
If you use pixi, mumps-seq is already part of the environment and the steps below are unnecessary on Linux and macOS.
sudo apt install build-essential cmake gfortran libopenmpi-dev libopenblas-dev liblapack-dev libmetis-dev
git clone https://github.com/coin-or-tools/ThirdParty-Mumps.git ~/ThirdParty-Mumps
cd ~/ThirdParty-Mumps
./get.Mumps
./configure --prefix=$HOME/mumps-coinor
make -j$(nproc)
make install
brew install cmake gcc open-mpi openblas
git clone https://github.com/coin-or-tools/ThirdParty-Mumps.git ~/ThirdParty-Mumps
cd ~/ThirdParty-Mumps
./get.Mumps
./configure --prefix=$HOME/mumps-coinor FC=gfortran F77=gfortran
make -j$(sysctl -n hw.ncpu)
make install
Build via MSYS2 (UCRT64 shell):
pacman -S mingw-w64-ucrt-x86_64-{gcc-fortran,gcc,openblas,metis,scotch} make autoconf automake libtool
git clone https://github.com/coin-or-tools/ThirdParty-Mumps.git && cd ThirdParty-Mumps
./get.Mumps && ./configure --prefix=/ucrt64 && make -j$(nproc) && make install
Then add C:\msys64\ucrt64\bin to your Windows PATH so that libdmumps.dll is found at runtime.
Below are two short examples that illustrate some of the features of Amigo.
First, the Rosenbrock function is a frequently used example problem in optimization. In Amigo, all analysis occurs within classes that are derived from amigo.Component.
The inputs, constraints, outputs, objective function, data and class constants are defined in the constructor. Values are accessed through dictionary member data structures self.inputs, self.constraints, self.outputs, self.objective, self.data and self.constants.
The Rosenbrock component takes two inputs x1 and x2 and provides the objective value obj and a constraint con.
import amigo as am
class Rosenbrock(am.Component):
def __init__(self):
super().__init__()
self.add_input("x1", value=-1.0, lower=-2.0, upper=2.0)
self.add_input("x2", value=-1.0, lower=-2.0, upper=2.0)
self.add_objective("obj")
self.add_constraint("con", value=0.0, lower=-float("inf"), upper=0.0)
def compute(self):
x1 = self.inputs["x1"]
x2 = self.inputs["x2"]
self.objective["obj"] = (1 - x1) ** 2 + 100 * (x2 - x1**2) ** 2
self.constraints["con"] = x1**2 + x2**2 - 1.0
model = am.Model("rosenbrock")
model.add_component("rosenbrock", 1, Rosenbrock())
model.build_module()
model.initialize()
opt = am.Optimizer(model)
opt.optimize()
The Amigo model is created by initializing the amigo.Model object and adding components to it. The code is then created and compiled by model.build_module().
Note when model.build_module() is called, the module is compiled. Whenever python code within the component class is changed, the module must be re-built. This must occur before initialization.
The system dynamics are encoded within a CartComponent class that inherits from amigo.Component. In the constructor, you must specify what constants, inputs, constraints and data (not illustrated in this example) the component requires.
The only class member function that is used by Amigo is the compute function. The inputs, constraints, constants and data must be extracted from the dictionary-like member objects in the compute function. These are not numerical objects, but instead encode the mathematical operations that are reinterpreted to generate c++ code.
In some applications it can be simpler to create multiple versions of the compute function. In this case, you can set amigo.Component.set_args(). which takes a list of dictionaries of keyword arguments, which will provide the compute function the provided keyword arguments.
import amigo as am
class CartComponent(am.Component):
def __init__(self):
super().__init__()
# Set constant values (these are compiled into static constexpr values in c++)
self.add_constant("g", value=9.81)
self.add_constant("L", value=0.5)
self.add_constant("m1", value=1.0)
self.add_constant("m2", value=0.3)
# Input values specify variables that are under the control of the optimizer
self.add_input("x", lower=-50, upper=50, units="N", label="control")
self.add_input("q", shape=(4), label="state")
self.add_input("qdot", shape=(4), label="rate")
# Constraints within the optimization problem
self.add_constraint("res", shape=(4), lower=0.0, upper=0.0, label="residual")
return
def compute(self):
# The compute functions take
g = self.constants["g"]
L = self.constants["L"]
m1 = self.constants["m1"]
m2 = self.constants["m2"]
# Extract the input objects
x = self.inputs["x"]
q = self.inputs["q"]
qdot = self.inputs["qdot"]
# Compute intermediate variables
sint = am.sin(q[1])
cost = am.cos(q[1])
# Compute the residual
res = 4 * [None]
res[0] = q[2] - qdot[0]
res[1] = q[3] - qdot[1]
res[2] = (m1 + m2 * sint**2) * qdot[2] - (
L * m2 * sint * q[3] * q[3] * x + m2 * g * cost * sint
)
res[3] = L * (m1 + m2 * sint**2) * qdot[3] + (
L * m2 * cost * sint * q[3] * q[3] + x * cost + (m1 + m2) * g * sint
)
# Set the output
self.constraints["res"] = res
return
To create a model, you create the components, and specify how many instances of that component are used within the component group.
# Create instances of the component classes
cart = CartComponent()
ic = InitialConditions()
fc = FinalConditions()
# Specify the module name
module_name = "cart_pole"
model = am.Model(module_name)
# Add the component classes to the model
model.add_component("cart", num_time_steps + 1, cart)
model.add_component("ic", 1, ic)
model.add_component("fc", 1, fc)
Variables are linked between components through an explicit linking process. Indices within the model are linked with text-based linking arguments. The names provided to the linking command are scoped by component_name.variable_name.
You can also add sub-models to the model by calling model.sub_model("sub_model", sub_model). In this case the scope becomes sub_model.component_name.variable_name. Any links specified in the sub-model are added to the model.
Linking establishes that two inputs from different components are the same. Linking two constraints or two outputs together means that the sum of the two output values are used as a constraint. You cannot link between types for instance linking inputs to constraints or outputs.
# Link the initial and final conditions
model.link("cart.q[0, :]", "ic.q[0, :]")
model.link(f"cart.q[{num_time_steps}, :]", "fc.q[0, :]")
# After variables are all linked, initialize the model
model.initialize()
Source and target indices can be supplied for more general linking relationships.