About the Role
Job Description
-Design and develop scalable, reusable data ingestion frameworks using Azure Databricks and PySpark
-Build metadata-driven pipelines that decouple configuration from execution
-Implement and maintain Bronze, Silver, and Gold data layers following Medallion architecture
-Integrate data pipelines with data catalog and governance tools (e.g., DataHub, Purview)
-Implement schema enforcement, data contracts, and data quality checks within pipelines
Work with ingestion tools (e.g., Fivetran) and file-based ingestion patterns
-Enable automation for dataset onboarding using configuration-driven approaches
-Collaborate with architecture, governance, and business teams to ensure scalable and governed data access
-Contribute to evolving toward event-driven and near real-time ingestion patterns
-Support CI/CD and DevOps practices for data platform components
-Pay Rate between 50-65/hr
We are a company committed to creating diverse and...
-Design and develop scalable, reusable data ingestion frameworks using Azure Databricks and PySpark
-Build metadata-driven pipelines that decouple configuration from execution
-Implement and maintain Bronze, Silver, and Gold data layers following Medallion architecture
-Integrate data pipelines with data catalog and governance tools (e.g., DataHub, Purview)
-Implement schema enforcement, data contracts, and data quality checks within pipelines
Work with ingestion tools (e.g., Fivetran) and file-based ingestion patterns
-Enable automation for dataset onboarding using configuration-driven approaches
-Collaborate with architecture, governance, and business teams to ensure scalable and governed data access
-Contribute to evolving toward event-driven and near real-time ingestion patterns
-Support CI/CD and DevOps practices for data platform components
-Pay Rate between 50-65/hr
We are a company committed to creating diverse and...
Ready to Apply?
Submit your application today and take the next step in your career journey with Insight Global.
Apply Now