Capabilities Overview

PatternOps defines 21 capability contracts organised into functional domains.

Capability Map

graph TB
    subgraph "Data Movement"
        SA[Source Acquisition]
        REP[Replication]
    end

    subgraph "Processing"
        EX[Execution]
        TR[Transformation]
        EXT[Extension]
    end

    subgraph "Orchestration & State"
        OR[Orchestration]
        SM[State Management]
        MSG[Messaging]
    end

    subgraph "Governance"
        SEC[Security]
        DQ[Data Quality]
        DC[Data Contracts]
        SR[Schema Registry]
        VC[Version Control]
    end

    subgraph "Observability"
        OBS[Observability]
        LIN[Lineage]
        MET[Metrics]
    end

    subgraph "Storage"
        ST[Storage]
    end

Contract Summary

Capability	Package	Methods
Source Acquisition	capabilities.source	connect, discover_schema, acquire, checkpoint, resume, disconnect
Execution	capabilities.execution	validate_pipeline, submit, status, cancel, metrics, logs, cleanup
Orchestration	capabilities.orchestration	submit, schedule, retry, cancel, pause, resume, observe, recover
Messaging	capabilities.messaging	publish, consume, acknowledge, dead_letter, replay, metrics
State Management	capabilities.state	write, read, query, delete, checkpoint, restore
Storage	capabilities.storage	write, read, list, commit, validate_path, apply_retention, apply_security
Replication	capabilities.replication	replicate, synchronise, verify, resume, monitor
Security: Classification	capabilities.security	classify, tag, score_risk
Security: Encryption	capabilities.security	encrypt, decrypt, rotate_keys
Security: Tokenisation	capabilities.security	tokenise, detokenise, validate
Security: Secret Mgmt	capabilities.security	resolve_secret, rotate_secret, validate_access
Data Quality	capabilities.quality	profile, validate, monitor, trend, detect_anomaly
Transformation	capabilities.transformation	validate, execute, explain, lineage, optimise
Extension	capabilities.extension	execute, validate, supported_hook_types
Observability	capabilities.observability	export_traces, export_metrics, export_logs, health_status, configure_alerts
Lineage	capabilities.lineage	emit_lineage, publish_lineage, query_lineage, export_lineage
Metrics	capabilities.metrics	emit_metric
Data Contract	capabilities.datacontract	generate_contract, validate_contract, publish_contract, check_compatibility, deprecate
Schema Registry (Messaging)	capabilities.schema	register_schema, get_schema, get_latest_schema, check_compatibility, list_subjects, list_versions, delete_subject
Schema Registry (General)	capabilities.schema	register_schema, get_schema, get_latest_schema, evolve_schema, list_schemas, compare_versions, deprecate
Version Control	capabilities.versioncontrol	commit, get_version, get_latest, list_versions, diff, rollback, tag

Base Contract

Every capability extends BaseProviderContract:

public interface BaseProviderContract {
    ValidationResult validateConfig(Map<String, Object> config);
    void initialise(Map<String, Object> config);
    HealthStatus healthCheck();
    List<MetricsEnvelope> metrics();
    void cleanup();
}

Provider Examples

Capability	Example Providers
Source Acquisition	JDBC, Kafka, S3, REST API, SFTP
Execution	Databricks, Snowflake, EMR, Synapse
Orchestration	Airflow, Prefect, Dagster, Step Functions
Messaging	Kafka, Pulsar, SQS, EventBridge
State Management	PostgreSQL, Redis, DynamoDB
Storage	S3/Delta, GCS/Iceberg, ADLS/Hudi
Replication	AWS DMS, Debezium, Striim
Security	HashiCorp Vault, AWS KMS, Protegrity
Data Quality	Great Expectations, Soda, Monte Carlo
Transformation	Spark SQL, dbt, Snowpark
Observability	OpenTelemetry, Prometheus, Datadog, Splunk
Schema Registry	Confluent, AWS Glue, Custom
Version Control	Git, Database-backed