Lead, Data Engineer

We are hiring a Lead, Data Engineer to join our team. At Kroll, we are building a strong Data practice with artificial intelligence, machine learning practice and analytics, and we’re looking for you to join our growing portfolio. You will be involved in designing, building, and integrating data from various sources and working with an advanced engineering team and professionals from the world’s largest financial institutions, law enforcement, and government agencies.

The day-to-day responsibilities include but not limited to

• Design and build organizational data infrastructure and architecture
• Identifying, designing and implementing internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes for data delivery.
• Choose the best tools/services/resources to build robust data pipelines for data ingestion, connection, transformation, and distribution
• Design, develop and manage ELT applications.
• Working with global teams to deliver fault tolerant, high quality data pipelines

Requirements:

• Advanced Experience writing ETL/ELT jobs
• Advanced Experience with Azure, AWS and Databricks Platform (Mostly data related services)
• Advanced Experience with Python, Spark ecosystem (PySpark + Spark SQL), SQL database
• Ability to develop REST APIs, Python SDKs or Libraries, Spark Jobs, etc
• Proficiency in using open-source tools, frameworks, python libraries like FastAPI, Pydantic, Polars, Pandas, PySpark, Deltalake Tables, Docker, Kubernetes, etc
• Experience in Lakehouse & Medallion architecture, Data Governance, Data Pipeline Orchestration
• Excellent communication skills
• Ability to conduct data profiling, cataloging, and mapping for technical data flows
• Ability to work with an international team

Desired Skills:

• Strong cloud architecture principles: compute, storage, networks, security, cost savings, etc.
• Advanced SQL and Saprk query/data pipeline performance tuning skills.
• Experience and knowledge of building Lakehouse using technologies including Azure Databricks, Azure Data Lake, SQL, PySpark etc.
• Programing paradigm like OOPPs, Async programming, Batch processing
• Knowledge of CI/CD, Git

About Kroll

In a world of disruption and increasingly complex business challenges, our professionals bring truth into focus with the Kroll Lens. Our sharp analytical skills, paired with the latest technology, allow us to give our clients clarity—not just answering all areas of business. We value the diverse backgrounds and perspectives that enable us to think globally. As part of One team, One Kroll, you’ll contribute to a supportive and collaborative work environment that empowers you to excel.

Kroll is the premier global valuation and corporate finance advisor with expertise in complex valuation, disputes and investigations, M&A, restructuring, and compliance and regulatory consulting. Our professionals balance analytical skills, deep market insight and independence to help our clients make sound decisions. As an organization, we think globally—and encourage our people to do the same.

Kroll is committed to equal opportunity and diversity, and recruits people based on merit.

In order to be considered for a position, you must formally apply via careers.kroll.com

Originally posted on Himalayas