About the Role
We are looking for a Senior Data Engineer with strong expertise in Databricks, PySpark, Delta Lake, and cloud-based data pipelines. The ideal candidate will design and build scalable ETL/ELT solutions, implement Lakehouse/Medallion architectures, and integrate data from multiple internal and external systems. This role requires strong technical leadership and hands-on architecture experience.
Key Responsibilities
Design, build, and optimize data ingestion and transformation pipelines using Databricks, PySpark, and Python.
Implement Delta Lake and Medallion architecture for scalable enterprise data platforms.
Develop ingestion frameworks for data from SFTP, REST APIs, SharePoint/Graph API, AWS, and Azure sources.
Automate workflows using Databricks Workflows, ADF, Azure Functions, and CI/CD pipelines.
Optimize Spark jobs for performance, reliability, and cost efficiency.
Implement data validation, quality checks, and monitoring with automated alerts and retri...
Key Responsibilities
Design, build, and optimize data ingestion and transformation pipelines using Databricks, PySpark, and Python.
Implement Delta Lake and Medallion architecture for scalable enterprise data platforms.
Develop ingestion frameworks for data from SFTP, REST APIs, SharePoint/Graph API, AWS, and Azure sources.
Automate workflows using Databricks Workflows, ADF, Azure Functions, and CI/CD pipelines.
Optimize Spark jobs for performance, reliability, and cost efficiency.
Implement data validation, quality checks, and monitoring with automated alerts and retri...
Ready to Apply?
Submit your application today and take the next step in your career journey with PURVIEW.
Apply Now