Echo REST API¶

The Echo API module exposes the Echo Service as a Spring Boot REST API. It provides HTTP endpoints for state management, metrics, lineage, events, and quality results.

Running the API¶

cd echo-api
mvn spring-boot:run

The API starts on port 8080 by default. Health check available at /actuator/health.

Endpoints¶

State Management¶

Get Pipeline State¶

GET /api/v1/state/{tenancy}/{namespace}/{dataset}/{pipeline}/{businessDate}/{runId}

Response (200):

{
  "success": true,
  "data": {
    "key": {"tenancy": "acme", "namespace": "prod", ...},
    "currentState": "RUNNING",
    "lastTransitionAt": "2024-01-15T10:30:00Z",
    "lastEvent": "pipeline-submitted"
  },
  "timestamp": "2024-01-15T10:30:05Z"
}

Response (404 — state not found):

{
  "success": false,
  "error": "State not found for key: ...",
  "timestamp": "2024-01-15T10:30:05Z"
}

Transition State¶

POST /api/v1/state/transition
Content-Type: application/json

{
  "tenancy": "acme",
  "namespace": "production",
  "dataset": "orders",
  "pipeline": "daily-ingest",
  "businessDate": "2024-01-15",
  "runId": "run-001",
  "targetState": "RUNNING",
  "event": "pipeline-submitted"
}

Response (200):

{
  "success": true,
  "data": {
    "success": true,
    "previousState": null,
    "currentState": "RUNNING",
    "message": "Initial state set to 'RUNNING'"
  }
}

Response (409 — invalid transition):

{
  "success": false,
  "error": "Invalid state transition from 'COMPLETED' to 'RUNNING'",
  "timestamp": "..."
}

Get Recovery Metadata¶

GET /api/v1/state/{tenancy}/{namespace}/{dataset}/{pipeline}/{businessDate}/{runId}/recovery

Metrics¶

Persist Metric¶

POST /api/v1/metrics
Content-Type: application/json

{
  "tenancy": "acme",
  "namespace": "production",
  "dataset": "orders",
  "pipeline": "daily-ingest",
  "runId": "run-001",
  "executionMode": "BATCH",
  "provider": "spark-provider",
  "stage": "transform",
  "metricName": "records.processed",
  "metricValue": 150000,
  "metricUnit": "COUNT",
  "timestamp": 1705312200000,
  "dimensions": {"partition": "2024-01-15"}
}

Query Metrics¶

GET /api/v1/metrics?tenancy=acme&namespace=production&pipeline=daily-ingest&metricName=records.processed&pageSize=20

Lineage¶

Persist Lineage Event¶

POST /api/v1/lineage
Content-Type: application/json

Query Lineage¶

GET /api/v1/lineage/{dataset}?direction=BACKWARD

Direction values: FORWARD, BACKWARD, IMPACT

Events¶

Publish Event¶

POST /api/v1/events
Content-Type: application/json

Query Events¶

GET /api/v1/events?tenancy=acme&namespace=production&pipeline=daily-ingest&eventType=state.transition&pageSize=50

Quality¶

Persist Quality Result¶

POST /api/v1/quality
Content-Type: application/json

Query Quality Results¶

GET /api/v1/quality?tenancy=acme&dataset=orders&pageSize=20

Error Handling¶

HTTP Status	Cause
200	Success
400	Validation error (blank fields, invalid format)
404	State not found
409	Invalid state transition

All error responses follow the ApiResponse format:

{
  "success": false,
  "error": "descriptive error message",
  "timestamp": "2024-01-15T10:30:05Z"
}

Configuration¶

The API uses DefaultEchoService (in-memory) by default. To use a persistent store, provide a custom EchoService bean.

@Configuration
public class CustomEchoConfig {
    @Bean
    public EchoService echoService(DataSource dataSource) {
        return new JdbcEchoService(dataSource); // your implementation
    }
}