Senior Data Engineer (Databricks & PySpark)
Job Summary: Looking for a developer who has hands on experience with Databricks/Pyspark , Java/ python, Webservices and SQL. Position is long term and will likely be renewed annually.
Essential Job Functions
- Understands the full picture and helps operations with monitoring, dashboards etc.
- work with architect and tech lead on design
- constantly keeps an eye on cost and performance of the system
- work with BA on the integration requirements
- Ability to trouble shoot performance issue and other functional issues
- Attention to details and documents the key decisions made
- Ability to handle fast paced projects and willing to look into production issues
- Should be a quick/self learner and be able to adopt to new technologies
- Prompt response /communication needed
Minimum Qualifications and Job Requirements
- Must be a team player.
- Must have at least 5 years of IT development experience.
- Must have strong analytical and problem-solving skills.
- Must have experience in designing solutions, performing code reviews, mentoring junior engineers.
- Must have strong SQL and backend experience, and working on data driven projects.
Must have following experience
- Python/PySpark
- SQL/ PL/SQL
- Databricks
- Java/C#
Preferred to have following experience
- Azure
- Kafka
- Node JS
- Azure Data Factory