Multi-Tenancy

PatternOps is multi-tenant by default. Every operation is scoped by tenancy and namespace.

Hierarchy

Tenancy (organisation boundary)
└── Namespace (logical grouping)
    └── Dataset
        └── Pipeline
            └── Run (execution instance)

State Key

All state is keyed by the full hierarchy:

public record StateKey(
    String tenancy,       // e.g., "acme-corp"
    String namespace,     // e.g., "production"
    String dataset,       // e.g., "customer-orders"
    String pipeline,      // e.g., "daily-ingestion"
    String businessDate,  // e.g., "2024-01-15"
    String runId          // e.g., "run-abc123"
) { }

Isolation Guarantees

Layer	Isolation Mechanism
Pipeline definitions	Tenancy + namespace in every definition
State management	StateKey includes tenancy + namespace
Metrics	MetricsEnvelope scoped by tenancy + namespace
Provider resolution	TenancyContext passed to registry
Storage paths	Logical path: {tenancy}/{namespace}/{dataset}/{partition}
Events	CloudEvents include tenancy + namespace extensions

Provider Resolution Scoping

Different tenants can use different providers for the same capability:

// Tenant A uses Databricks for execution
registry.resolve("execution", new TenancyContext("tenant-a", "production"));
// → DatabricksExecutionProvider

// Tenant B uses Snowflake for execution
registry.resolve("execution", new TenancyContext("tenant-b", "production"));
// → SnowflakeExecutionProvider

Metrics Envelope

Every metric is tagged with tenancy context:

public record MetricsEnvelope(
    String tenancy,
    String namespace,
    String dataset,
    String pipeline,
    String runId,
    // ... metric data
) { }

Logical Storage Paths

Storage uses a standard logical path format:

{tenancy}/{namespace}/{dataset}/{partition_key}

The Storage Provider maps logical paths to physical locations without exposing physical paths to pipeline definitions.

Example

# Pipeline for Tenant A
name: daily-orders
tenancy: acme-corp
namespace: production
dataset: customer-orders
executionMode: batch
# ...

# Pipeline for Tenant B (same structure, different tenant)
name: daily-orders
tenancy: globex-inc
namespace: production
dataset: customer-orders
executionMode: batch
# ...

Both pipelines can coexist with complete isolation — different state, different metrics, potentially different providers.