Lead Databricks Engineer with Pyspark & Python Exp - US

Role: Lead Databricks Engineer with Pyspark & Python Exp Location: NYC, NY OR Iselin, NJ (Hybrid - 3 days' work from office) Duration: 6 Months Job Description: Inperson interview is must (Financial background is must) Major Responsibilities: Work on Finance data related to Collaterals, ETD, OTD, Settlement market, Cash product, Repo, Duos repo Design, develop, and deploy Databricks jobs to process and analyze large volumes of data. Collaborate with data engineers and data scientists to understand data requirements and implement appropriate data processing pipelines. Optimize Databricks jobs for performance and scalability to handle big data workloads. Monitor and troubleshoot Databricks jobs, identify and resolve issues or bottlenecks. Implement best practices for data management, security, and governance within the Databricks environment. Experience designing and developing Enterprise Data Warehouse solutions. Demonstrated proficiency with Data Analytics, Data Insights Proficient wri...