Full-time
Databricks Data Platform Engineer
Posted by Epergne Solutions • Abu Dhabi, Abu Dhabi Emirate, United-Arab-Emirates
About the Role
Overview
Job Role : Databricks Data Platform Engineer
Job Location : Abu Dhabi, UAE
Experience : 5+ Years
Role Summary : Design and build scalable, high-performance, and governed data pipelines on Azure Databricks to support analytics, reporting, and enterprise data intelligence initiatives.
Responsibilities
- Key Responsibilities – Develop batch pipelines using Delta Lake with proper table design, partitioning, optimization, and lifecycle management
- Implement incremental ingestion using Autoloader with schema evolution and checkpointing
- Build Structured Streaming pipelines with watermarking, state management, and late data handling
- Develop declarative pipelines using Lakeflow standards
- Ensure pipelines support idempotency, replayability, and safe backfills
- Optimize Spark workloads (AQE, skew handling, shuffle tuning, join optimization)
Ready to Apply?
Submit your application today and take the next step in your career journey with Epergne Solutions.
Apply Now