Databricks architect

Architect - Databricks (Cloud Data)
Job Title: Architect - Databricks (Cloud Data)
Primary skills: Databricks, Py Spark, and architecture solutions.
Secondary skills: Unity Catalog, Star Burst, Azure Dev Ops, CICD, and orchestration layers.
Mode of work: Work from office
Experience: 13+ Years of Experience
Responsibilities:
· Lead migration from Native Spark to Databricks.
· Architect migration to Databricks.
· Build Data Governance solutions using tools like Unity Catalog and Star Burst.
· Develop orchestration workflows using Databricks and ADF.
· Implement CICD pipelines for Databricks in Azure Dev Ops.
· Process near-real-time data through Auto Loader and DLT pipelines.
· Design secure, scalable infrastructure for Delta Lake and Spark SQL.
· Optimize Databricks environments for cost-effectiveness.
· Extract complex business logic from on-prem solutions like SSIS, Informatica, Vertica, etc., into Py Spark.
· Build analytical dashboards...