Data Scientist

Job Summary

The Data Engineer is responsible for integrating CDS source systems into the AI platform by building the connectors and ingestion pipelines that reliably move source-system data into the platform.

Key Responsibilities

• Build source-system integrations (webhooks $\rightarrow$ API Gateway/Lambda $\rightarrow$ EventBridge $\rightarrow$ SQS).
• Implement watermark/state tracking (DynamoDB) and periodic reconciliation to capture webhook delivery gaps without placing polling load on source systems.
• Onboard new CDS source systems onto the platform's ingestion pattern as the scope expands (e.g., Confluence, JIRA, and others across the CDS estate).
• Handle data migration activities where a source system needs backfilling into the platform (e.g., AWS DMS).
• Work with Knowledge Engineers to ensure ingested data meets quality and structural requirements.

Skill Requirements

(No specific skill requirements were provided in the raw input.)

Other Requirements

(No specific other requirements were provided in the raw input.)