Scala Big Data Engineer
Detailed job description
Job Description
We are looking for a very experienced and highly motivated engineer who can work independently to design, develop and support the data platform our customer is building.
To achieve success in this role you need to fully understand the data you are working with and build efficient ETL solutions together with your team and architects.
A commitment to collaborative problem-solving, sophisticated design, and quality product is essential.
This job position does not require a standard work schedule — we’re open to freelance as well. As long as you have a strong work ethic and deliver the necessary tasks, you can organise your work as you please.
The project
Our customer is focused on using data and science to help healthcare clients find better solutions for their patients.
Their data platform is integrating, managing and curating complex, clinically rich medical data sets (millions of records) from multiple data sources (spanning different countries, languages and compliance requirements).
The Perfect Candidate
- 6+ years of experience in data engineering
- Extensive experience with complex ETL design and implementation
- Experience with Batch processing, streaming and Microservices Architecture
- 4+ years of the following language: Scala
- Experience with one or more of: Apache Spark, Apache Kafka, Apache Flink, HDFS
- Good SQL knowledge (4+ years)
- Experience with data analysis, data profiling, data integration, data quality, data warehousing, data reporting (2+ years)
- Excellent analytical & troubleshooting skills
- Ability to present complex results in a clear language
- Proficient in using code versioning tools such as Git
- Nice to have: experience with Java or Python; database design and modelling; working with cloud technologies; data volume processing
- Understand complex concepts in solving problems
- Suggest workflow process improvements and sustain self-organized agile teams
- Maintaining active communication with the project team to clarify the requirements
- Empathic and working independently
- Bachelor’s Degree in Information Technology, Software Engineering, Computer Science, Mathematics, or other related field
- Excellent written and speaking English skills
Your Responsibilities
- Designing and developing ETL solutions to process big amount of medical data
- Ensuring high quality by creating comprehensive and automated tests
- Continuously improving the data platform and reducing technical debt
- Supporting introduction of new tools and migration to Azure Cloud
- Providing support to less experienced team members
- Contributing to standards definition and promoting best practices
What We Offer
- Very flexible work time and option to work from home anytime.
- Attractive salary package (including discounts for Sports activities etc.).
- The opportunity to learn from other experienced senior developers, analysts, and consultants.
- Working with international customers and projects.
- Rewards and benefits based on achievements.
- Rewards for recommending new developers and customers.
- Good coffee and cake at the office. :)