Lead data engineer

Key Responsibilities

• Design and develop scalable, high‑performance data pipelines across Hadoop ecosystem components (Hive, Impala, Spark, Kafka, Iceberg).
• Build robust data ingestion and transformation frameworks using Java, Spark, Python, and shell scripting for both batch and real-time.
• Architect and deliver modern data platforms including Lakehouse architecture, Data Mesh, Data Fabric, and domain-aligned data products
• Develop full‑stack applications and internal engineering tools using Python, shell scripting, and modern web frameworks (e.g., Flask, React).
• Design and implement secure APIs and microservices to expose data assets and machine learning models to downstream systems and user interfaces.
• Collaborate closely with data scientists to operationalize machine learning models using Cloudera Machine Learning (CML).
• Implement enterprise‑grade security and governance controls, including RBAC, LDAP, Kerberos, Apac...