Data Engineer

About the Company We are a leading financial services firm dedicated to providing innovative solutions and exceptional service to our clients. Our mission is to empower businesses with data-driven insights while fostering a culture of inclusivity and collaboration. About the Role The Data Engineer – PySpark & Azure Databricks will be responsible for designing and developing scalable data pipelines, processing financial data, and ensuring data quality and compliance. Responsibilities Design and develop scalable data pipelines using PySpark on Azure Databricks Process structured and unstructured financial data (transactions, customer data, risk data) Build ETL/ELT pipelines using Azure services: Azure Data Factory (ADF) Azure Data Lake Storage (ADLS) Implement data transformation, cleansing, and aggregation for: Risk analytics Fraud detection Regulatory reporting (Basel, AML, KYC) Optimize Spark jobs for performance (partitioning, caching, tuning) Ensure data quality, governance, and sec...