Principal Data Engineer (PySpark)

We are seeking a Principal Data Engineer with 5+ years of hands‑on experience and a strong background in real‑time and batch data processing, containerization, and cloud‑based data orchestration. This role is ideal for someone passionate about building robust, scalable, and efficient data pipelines, and who thrives in agile, collaborative environments.

Key Responsibilities

• Design, build, and maintain real‑time data pipelines using streaming frameworks such as Kafka, Apache Flink, and Spark Structured Streaming.
• Develop batch processing workflows with Apache Spark (PySpark).
• Orchestrate and schedule data workflows using orchestration frameworks such as Apache Airflow and Azure Data Factory.
• Containerize applications using Docker, manage deployments with Helm, and run them on Kubernetes.
• Implement modern storage solutions using open formats such as Parquet, Delta Lake, and Apache Iceberg.
• Build high‑p...