Data Pipeline Engineer

Job Description

We are seeking an experienced Senior Data Pipeline Engineer to design, build, and operate secure, reliable, and cost-efficient data pipelines supporting our Android connected and infotainment experiences. You will lead Python-based ETL/ELT development, Airflow orchestration, and data platform operations across cloud environments, working closely with Android, backend, and product teams.

Responsibilities

• Build and maintain Airflow 2.x DAGs (TaskFlow, dynamic DAGs, deferrable operators, providers).
• Develop robust Python ETL/ELT ingesting from APIs, storage, message buses, and databases.
• Operate Airflow on Azure/Kubernetes; support blue/green and canary DAG releases.
• Implement data quality testing, monitoring, observability, and lineage.
• Design scalable batch and streaming pipelines with strong schema management.
• Manage SQL and blob data stores (partitioning, clustering, retention). <...