Senior Data Engineer, Big data

About Us

HG Insights is the global leader in technology intelligence, delivering actionable AI driven insights through advanced data science and scalable big data solutions. Our platform informs go-to-market decisions, and we influence how businesses spend millions of marketing and sales budgets.

What You’ll Do:

• Design, build, and optimize large-scale distributed data pipelines for processing billions of events from and to multiple sources of data.
• Architect and scale enterprise-grade big-data systems, including data lakes, ETL/ELT workflows, and syndication platforms for customer-facing products.
• Orchestrate pipelines and workflows with Airflow.
• Write optimized data into analytical databases such as ClickHouse, DuckDB, and Redshift
• Ensure data quality, consistency, and reliability across the pipeline.
• Monitor pipeline performance and troubleshoot data issues.
• Collabor...