Overview

Relevant source files

• .github/workflows/build-core.yml
• .github/workflows/release-apiv2-client.yml
• .python-version
• CHANGELOG.md
• README.md
• airone/lib/elasticsearch.py
• airone/settings.py
• airone/tests/test_elasticsearch.py
• entry/api_v2/serializers.py
• entry/api_v2/urls.py
• entry/api_v2/views.py
• entry/models.py
• entry/tasks.py
• entry/tests/test_api_v2.py
• entry/tests/test_api_v2_alias.py
• entry/tests/test_model.py
• entry/tests/test_view.py
• entry/views.py
• pyproject.toml

Purpose and Scope

Pagoda (formerly Airone) is a flexible Single Source of Truth (SSoT) system designed to manage structured data with dynamic schemas, complex relationships, and granular access control. The system allows organizations to define custom data models, create data instances, manage relationships between entries, and enforce fine-grained permissions at the entity, entry, and attribute levels.

This page provides a high-level introduction to the Pagoda system architecture and core concepts. For detailed information about:

• Fundamental data model concepts (Entity, Entry, Attribute, etc.), see Core Concepts
• Installation and development environment setup, see Installation and Setup
• Detailed system architecture, see System Architecture

Sources: README.md1-23 CHANGELOG.md1-50

System Identity and Positioning

Pagoda is a metadata management platform that enables:

1. Flexible Schema Definition: Define data structures (Entities) with typed attributes without database migrations
2. Dynamic Data Management: Create and manage data instances (Entries) that conform to defined schemas
3. Relationship Management: Model complex relationships between entries using referral attributes
4. Granular Access Control: Set permissions at entity, entry, and individual attribute levels
5. Automation: Configure trigger-based actions that execute when entry data changes
6. Search and Discovery: Advanced search capabilities powered by Elasticsearch

Unlike traditional CMDBs or fixed-schema databases, Pagoda allows users to dynamically create and modify data structures through a web interface or API.

Sources: README.md8-14 entry/models.py1-100

High-Level System Architecture

Architecture Overview

The system follows a layered architecture with clear separation between:

1. Presentation Layer: React-based SPA and REST APIs
2. Application Layer: Django views, DRF ViewSets, and business logic
3. Domain Layer: Core models (Entity, Entry, Attribute, ACLBase)
4. Infrastructure Layer: Database, search engine, message queue, storage

All write operations that modify entries return HTTP 202 (Accepted) and execute asynchronously via the job system, ensuring consistency and enabling complex operations like webhooks and triggers.

Sources: pyproject.toml32-75 airone/settings_common.py entry/api_v2/views.py1-100

Core Domain Model Overview

Domain Model Explanation

The domain model separates schema from data:

• Schema Layer: Entity defines the structure (like a database table), EntityAttr defines attribute schemas (like column definitions)
• Data Layer: Entry is a data instance (like a table row), Attribute is an attribute instance, AttributeValue stores versioned values
• Access Control: All schema and data objects inherit from ACLBase, enabling granular permissions
• Automation: Job handles asynchronous operations, TriggerParent configures event-driven actions

This separation allows schema changes without data migration and supports temporal queries through AttributeValue.is_latest.

Sources: entry/models.py33-567 entity/models.py acl/models.py job/models.py

Data Flow and API Request Lifecycle

Request Lifecycle Details

1. Validation: Multi-layer validation using Pydantic, DRF serializers, and custom validators
2. Job Creation: Write operations create a Job instance with JobOperation type
3. Asynchronous Execution: Celery workers execute the actual operation
4. Side Effects: Update Elasticsearch, send webhooks, invoke triggers
5. Status Tracking: Job status progresses through PREPARING → PROCESSING → DONE/ERROR

This pattern ensures:

• Fast API response times (non-blocking)
• Transactional consistency
• Ability to cancel long-running operations
• Audit trail of all operations

Sources: entry/api_v2/views.py111-124 entry/tasks.py1-100 job/models.py

Type System and Attribute Types

Pagoda supports 16 attribute types defined in AttrType enum:

Type	Enum Value	Description	Storage
STRING	2	Short text	AttributeValue.value
TEXT	4	Long text	AttributeValue.value
OBJECT	8	Reference to Entry	AttributeValue.referral
NAMED_OBJECT	16	Entry reference with label	value + referral
ARRAY_STRING	32	String array	Multiple AttributeValue children
ARRAY_OBJECT	64	Entry array	Multiple children with referral
ARRAY_NAMED_OBJECT	128	Named entry array	Multiple children with value + referral
BOOLEAN	256	Boolean flag	AttributeValue.boolean
GROUP	512	Reference to Group	AttributeValue.group
DATE	1024	Date value	AttributeValue.date
ROLE	2048	Reference to Role	AttributeValue.role
ARRAY_GROUP	4096	Group array	Multiple children with group
ARRAY_ROLE	8192	Role array	Multiple children with role
DATETIME	16384	DateTime with timezone	AttributeValue.datetime
NUMBER	32768	Numeric value	AttributeValue.value (as string)
ARRAY_NUMBER	65536	Numeric array	Multiple children

Array types use a parent-child pattern where AttributeValue.STATUS_DATA_ARRAY_PARENT marks the parent, and child values are linked via parent_attrv foreign key.

Sources: airone/lib/types.py entry/models.py33-87 entry/tests/test_model.py101-134

Search and Indexing Architecture

Search Capabilities

The search system provides:

1. Simple Search: Keyword search across entry names and attribute values

2. Advanced Search: Multi-attribute filtering with complex conditions using FilterKey enum:

   • EMPTY: Find entries with empty attribute values
   • NON_EMPTY: Find entries with non-empty values
   • TEXT_CONTAINED: Substring match
   • TEXT_NOT_CONTAINED: Negation
   • DUPLICATED: Find duplicate attribute values

3. Nested Document Structure: Each entry is indexed with nested attr[] and referrals[] arrays for efficient querying

4. Permission Filtering: Results are filtered by ACLBase.is_readable flags

5. Join Attributes: Include attributes from referenced entries in search results

Sources: airone/lib/elasticsearch.py94-400 entry/services.py entry/api_v2/views.py311-450

Job System and Asynchronous Processing

All data-modifying operations execute asynchronously through the Job system:

Job Operations

Key job types defined in JobOperation enum:

• CREATE_ENTRY: Create new entry (async to run triggers, webhooks)
• EDIT_ENTRY: Update entry attributes
• DELETE_ENTRY: Soft delete entry
• IMPORT_ENTRY: Bulk import from YAML/CSV
• EXPORT_ENTRY: Export entries to file
• EXPORT_SEARCH_RESULT: Export search results
• REGISTER_REFERRALS: Update ES for entries that reference a changed entry
• INVOKE_TRIGGER: Execute trigger actions
• NOTIFY_CREATE_ENTRY: Send webhook for creation
• NOTIFY_UPDATE_ENTRY: Send webhook for update
• NOTIFY_DELETE_ENTRY: Send webhook for deletion

Jobs support:

• Cancellation (via job.is_canceled() checks)
• Dependencies (jobs wait for parent jobs)
• Result caching (export jobs store file paths)
• Progress tracking (via job.text field)

Sources: job/models.py entry/tasks.py1-150 airone/lib/job.py

Technology Stack Summary

Layer	Technology	Purpose
Frontend	React 19, TypeScript, Material-UI 6	Single-page application
Backend	Django 4.2, Python 3.12, DRF 3.15	API server and business logic
Database	MySQL 8.0	Relational data storage
Search	Elasticsearch 7.17	Full-text search and indexing
Queue	RabbitMQ, Celery 5.5	Asynchronous task processing
Validation	Pydantic 2.10, Zod	Runtime type validation
Storage	AWS S3 / Local	File storage for exports
Package Manager	uv, npm	Python and Node.js dependencies
Monitoring	Datadog APM, Flower	Application monitoring

Deployment Configuration

• Web server: Gunicorn 23.0 with WSGI
• Static files: WhiteNoise 5.2
• Authentication: LDAP, SAML SSO, local
• Version control: Git with GitHub Actions CI/CD
• Python version: 3.12.9 (specified in .python-version1-2)
• Node version: 22.13 (from CHANGELOG)

Sources: pyproject.toml32-102 airone/settings_common.py .github/workflows/build-core.yml1-50

Key Design Patterns

Schema-Data Separation Pattern

The system maintains a strict separation between schema (Entity, EntityAttr) and data (Entry, Attribute, AttributeValue). This enables:

• Schema evolution without data migration
• Temporal queries through AttributeValue.is_latest flag
• Multiple attribute values per attribute (versioning)

ACL Inheritance Pattern

All major objects inherit from acl.models.ACLBase, which provides:

• Three permission levels: Readable (1), Writable (2), Full (4)
• is_public flag for public/private resources
• Permission associations with User, Group, and Role

Job-Based Async Pattern

All write operations follow this pattern:

1. Validate request (HTTP 400 if invalid)
2. Create Job record (HTTP 202 Accepted)
3. Execute asynchronously via Celery
4. Track status via Job model
5. Execute side effects (webhooks, triggers, ES updates)

Plugin System

The system supports backend and frontend plugins:

• Backend: Registered via @register_job_task decorator
• Frontend: Dynamic route injection and component loading
• SDK: pagoda-plugin-sdk package for plugin development

Sources: entry/models.py567-577 acl/models.py plugin/sdk airone/lib/job.py

Getting Started

To begin working with Pagoda:

1. Set up development environment: Follow instructions in Installation and Setup
2. Understand core concepts: Read Core Concepts to understand Entity, Entry, and Attribute
3. Explore the API: Review API Reference for available endpoints
4. Review the data model: Study Data Model for detailed schema information

For deploying to production, see Development and Deployment.

Sources: README.md1-23 CHANGELOG.md1-50