Senior Data Engineer (Scala and Spark experience) - Remote
SemanticBits
This position requires U.S. residency.
Summary
SemanticBits is looking for a talented Data Engineer who is eager to apply computer science, software engineering, databases, and distributed/parallel processing frameworks to prepare big data for the use of data analysts and data scientists. If you have experience with Scala and Spark and want your work to contribute to systems that collect healthcare data used by hundreds of thousands of daily users, we want to (virtually) meet you! You will work on projects that support the Centers for Medicare and Medicaid Services (CMS) as we develop a next-generation analytics and reporting system that directly impacts healthcare quality. You will use Spark to build data processing pipelines that derive information from large sets of government data. You will be the go-to on your team for Spark, the Spark Engine, and the Spark Dataframe API. We are a collaborative company, so we want you to use your knowledge of Spark to teach others, inform design decisions, and debug runtime problems. Tools & Technology
- Spark, Hadoop, Scala, Python, and AWS EMR
- Jupyter and Zeppelin
- Airflow, Jenkins, and AWS Step Functions
- AWS S3, AWS Redshift, and Teradata
- GSuite, Slack, Jira, Confluence, Git, and Github
Responsibilities
- Build scalable data processing pipelines in Spark
- Debug Spark jobs and do performance tuning
- Write unit and integration tests for all data processing code
- Work with DevOps engineers on CI, CD, and IaC
- Read specs and translate them into code and design documents
- Perform code reviews and develop processes for improving code quality
Requirements
- Highly Competent with Scala, Spark, the Spark Engine, and the Spark Dataframe API
- Experience with Agile methodology, using test-driven development.
- Excellent command of written and spoken English
- Candidate must reside in the United States
- Bachelor's degree required, strong preference for Computer Science field of study
- Flexible and willing to accept a change in priorities as necessary
Nice to Have
- Experience working in the healthcare industry with PHI/PII
- Federal Government contracting work experience
- Prior experience working remotely full-time
Physical and emotional requirements of the job
This position is to be performed remotely from an individual’s home office and involves sedentary work. Employees in this role can be expected to exert up to 10 pounds of force on occasion in order to lift, carry, push, pull or otherwise move standard electronic equipment. Employees are expected to make decisions in a timely manner and display emotional intelligence during occasional stressful situations.
If you are an individual with a disability and require a reasonable accommodation to complete any part of the application process, or are limited in the ability or unable to access or use this online application process and need an alternative method for applying, you may contact 703-787-9656 x257 for assistance.
Similar Jobs
