Senior Data Engineer – Azure Databricks Migration & Optimization

<p dir="auto">We’re migrating complex SQL Server logic (stored procedures, views, SSIS packages) from legacy systems straight into Azure Databricks + ADF — <strong>lift-and-shift style</strong>, no full redesign. The real work starts after migration: aggressively optimize for performance and cost, understand Spark physical plans inside-out, avoid hidden cross-joins or skew surprises, master Delta Lake retention pitfalls, and make smart compute choices that balance speed and DBUs.</p> <p dir="auto">If you love diving deep into Spark execution models, reading physical plans to fix subtle regressions, knowing exactly what VACUUM + OPTIMIZE does to time travel, and delivering production-grade cost savings without breaking business logic — this is your role.</p> <p dir="auto"><strong>Key Responsibilities</strong></p> <ul dir="auto"> <li>Migrate complex SQL Server...