Posted to IASSIST on: 2022-10-05
Employer: ICPSR
Employer URL: https://icpsr.umich.edu/
Description
Senior Data Engineer
Job Opening ID: 220903
Working Title: Lead Data Engineer
Job Title: App Programmer/Analyst Sr
Work Location: Ann Arbor Campus, Ann Arbor, MI
Full/Part Time: Full-Time
Regular/Temporary: Regular
FLSA Status: Exempt
Organizational Group: Inst Soc Research
Department: ICPSR
Posting Begin/End Date: 11/16/2022 - 1/10/2023
Salary: $115,000.00 - $130,000.00
Career Interest: Information Technology
Summary
The Inter-university Consortium for Political and Social Research (ICPSR) is embarking on a 5 year project to build a cloud-native platform for delivering digital social science artifacts to a global research community, and we need a Lead Data Engineer to join our team. We focus on delivering value quickly in an Agile environment that fosters creativity, exploration, and learning. As a member of our team, you will help implement a microservices-based architecture using technologies such as Apache Airflow, RedHat OpenShift, NoSQL databases, and AWS services. You will be supported by a small, diverse team committed to delivering high-quality products and an exceptional user experience through modern development and delivery practices. If you love creating secure, scalable, resilient data pipelines within a test-driven development framework, and can coach others in best practices, we want to talk to you!
You will report to the Associate Director of Computing and Network Services.
Responsibilities
- Author production-quality data pipelines running in a distributed, containerized, cloud-native environment to support ingestion, transformation, and dissemination of complex social science research data from a variety of sources
- Apply best practices in software development and data management to improve curation workflows in our secure data environment by building a library of reusable scripts and tools orchestrated by an automated jobs framework
- Develop secure, scalable, and resilient APIs to expose data as a service to upstream producers and downstream consumers
- Support development teams in all aspects of data management, from the conceptual modeling stage through to the application of data governance policies
- Implement monitoring solutions for data engineering products and provide operational support to pipelines and APIs in production
- Research and identify open source and COTS tools to solve challenging problems while minimizing the need for custom code
- Collaborate with technical leadership on platform architecture and design
- Manage Data Engineering staff, providing mentoring, setting goals, and assuring the application of best practices
Required Qualifications
- Bachelor’s degree in computer science, a related field, or equivalent experience
- Seven or more years of experience in software development, data engineering, data architecture, or combination (Linux environment preferred) including:
- Expert knowledge of Python for data engineering projects
- Expert knowledge of relational database management using SQL
- Substantial experience building production-quality data pipelines using a modern ETL, data integration, or data orchestration framework.
- Substantial experience building interfaces with heterogeneous data sources and sinks including REST APIs
- Experience with cloud storage and computing (AWS preferred)
- Understanding of distributed microservices architectures and their orchestration using Kubernetes.
- One or more years of experience in a technical leadership position (management experience preferred)
Desired Qualifications
- Knowledge of data architecture, modeling, and governance fundamentals
- Experience managing data pipelines in production using monitoring tools such as Prometheus or Grafana and log indexers such as Elasticsearch or Splunk
- Knowledge of NoSQL databases, in particular graph databases, triple stores, column stores, and document databases.
- Experience with streaming and/or event-driven architectures.
- Familiarity with tools used for quality assurance (e.g. static analysis, unit testing, integration testing, etc.) of data pipelines
- Understanding of modern CI/CD tools and practices
Work Schedule
We have a DevOps team with an on-call rotation that handles most real time issues. However, this job may require occasional work outside normal working hours to analyze and correct critical problems that arise in ICPSR’s 24 hours per day operational environment.
Underfill Statement
The requirements listed reflect a lead-level expectation with an attendant salary range of $115,000 - $130,000. Candidates with lesser experience are encouraged to apply and may be considered if the primary goal is not met. Lesser-experienced candidates will be reviewed at the senior level, which requires at least 5 years of experience with a salary range of $95,000 - 115,000.
Additional Information
The Institute for Social Research at the University of Michigan wants to employ a diverse workforce as a reflection of our commitment to serve the diverse people of Michigan, to maintain the excellence of the university, and to ground our research in varied disciplines, perspectives, and ways of knowing and learning.
Benefits at the University of Michigan
In addition to a career filled with purpose and opportunity, The University of Michigan offers a comprehensive benefits package to help you stay well, protect yourself and your family and plan for a secure future. Benefits include:
- Generous time off
- A retirement plan that provides two-for-one matching contributions with immediate vesting
- Many choices for comprehensive health insurance
- Vision, dental, and life insurance
- Long-term disability coverage
- Flexible spending accounts for healthcare and dependent care expenses
- Tuition reimbursement
Application Deadline
Institute for Social Research job openings are posted for a minimum of fourteen (14) calendar days. This opening may be removed from posting boards and filled any time after the minimum posting period has ended.
U-M EEO/AA Statement
The University of Michigan is an equal opportunity/affirmative action employer.
Job Location
Ann Arbor, MI
More info/apply URL: https://careers.umich.edu/job_detail/220911/lead-data-engineer
Archived on: 2022-11-28