W2 - (2) - Sr Data Engineer (Azure Devops, Cloud infrastructure, Linux, python) - Remote

Tanson Corp (Rochester MN, USA) Follow 9 days ago

Full-time 70 USD

Duties: Position Overview: (Major Functions and Non-Essential Functions): We are seeking a Senior Software Engineer to design and build back-end services that support our portfolio of data-centric clinical and analytic applications. These applications leverage cloud computing, big data, mobile, data science, data warehousing, machine learning using state of the art software development applications and frameworks. Our Software Engineers ensures that these cloud-based micro-services adhere to uptime and accuracy targets, are resilient, and scale as data volumes and traffic increase. They work closely with the data engineering, platform, and solutions teams to develop applications as required to benefit our practice and patients.

Works closely with the Product Owners, Product Managers, Architects to translate requirements into code.
Developing services around data warehousing, big data, cloud computing, business intelligence, analytics and machine learning.
Participate in DevOps, Agile, continuous development and integration frameworks.
Programming in high-level languages such as Go, Python, Java etc.
Ensure all appropriate documentation of processes and source code is created and maintained.
Communicate effectively with peers, leaders, and customers throughout the organization.
Participate in expert level troubleshooting and resolve problems through root cause analysis, data and system investigation.
Contributes to design and architecture discussions with Principals and Architects.
Leads targeted cross-functional improvement efforts and mentors more junior software engineers.
Solves complex problems; takes a new perspective on existing solutions.
Work independently with minimal guidance. You may lead projects or project steps within a broader project or have accountability for ongoing activities or objectives.
Act as a resource for colleagues with less experience.

Skills: Additional Experience and/or Qualifications: (Has Achieved Competency in the Following Areas, Job Knowledge and Additional Considerations):

Preferred qualifications for this position include:

master's degree in computer science/Engineering or related field.
Ability to use a wide variety of open-source technologies and cloud-based services.
Experience writing software for the cloud (GCP, AWS, Azure).
Experience in databases, analytics, big data systems or business intelligence products.
Experience building high-performance, highly available and scalable distributed systems.
Experience developing software for healthcare related industries.

Education: Minimum Education and/or Experience Required: (Education Requirements and Experience):Required qualifications for this position include: Bachelor's Degree in Computer Science/Engineering or related field with 5 years of experience as noted below;

OR an Associate's degree in Computer/Science/Engineering or related field with 7 years of experience. Have in-depth knowledge of software engineering with experience coding applications or services in a high-level language (C, C++, Golang, Java, C# etc.) and a basic knowledge of related fields. Demonstrated problem solving and time management skills. Possesses strong technical aptitude for designing and implementing software solutions. Experience with modern application development frameworks Knowledge of professional software engineering practices & best practices for the full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations. Deep hands-on technical expertise, excellent verbal and written communication skills. Experience with Agile software development techniques.

Schedule Notes: Scope: The resources will be supporting an engineering team tasked with building a research data platform which will ingest and make discoverable research generated data.

Data Engineering Skills & Experience:

Create, verify, and maintain data replication scripts
Create, verify, and maintain data validation, processing, and ingestion pipelines
Deploy and automate the execution of data replication scripts and data pipelines in cloud infrastructure
Create and maintain data catalogs that describe datasets and their contents (i.e. files, file types, tables/views, columns, fields, etc.)
Create, verify, and maintain dashboards and reports that characterize ingested datasets
Create, verify, and maintain data validation scripts/APIs that verify the production dataset contains the correct number of samples/records, expects values/fields/columns are populated, and values are of the correct data type, format, and range.
Deploy and automate the execution of data validation scripts/APIs
Create and maintain user documentation (dataset descriptions, tutorials, code examples, etc.)
Define entitlements, user groups, roles, and permissions utilized to grant access to datasets

Programming Languages: Primary pipeline development language with be python. Some datatypes and formats may require the use of other languages (i.e. java, R, etc.) because the libraries/frameworks/sdks available to work with those datatypes and formats are not available in python

Operating Systems: Primary operating system for data pipeline execution will be linux, with data pipelines packaged, deployed, and run as containers. Data source systems could be windows or linux based.

Infrastructure: Primary data platform and data pipeline execution infrastructure will be hosted on Google Cloud Platform (GCP) utilizing cloud native technologies (i.e. Google Cloud Storage, BigQuery, Google Batch, Dataflow, Cloud SQL, etc.). Data will be replicated from various on-premises sources that include laboratory instruments, network shared drives, and windows desktops attached to instruments.

Development Tools: Sprints, features, and tasks will be managed in Azure DevOps. Code will be managed and versioned Azure DevOps based git repositories. Code will be compiled, packaged, and deployed utilizing Azure DevOps build pipelines. Data pipelines will be packaged, deployed, and run in docker containers. Docker containers will be stored and versioned in Google Cloud Artifact Repositories. Veracode will be utilized to scan source code for vulnerabilities and Prisma Cloud will be utilized to scan containers. The standard integrated development environment will be jetbrains (pycharm, intellij, etc.) or VSCode.

Preferred Candidates: Experience working on healthcare, life science, or scientific research projects. A degree or domain knowledge in a life science related field (biochemistry, genetics, biology, etc). Experience with Google Cloud Platform based infrastructure and services 100% remote. Client will provide equipment.

Minimum Degree Required: Bachelor's Degree

Hours Per Day: 8.00

Hours Per Week 40.00

Pay rate: $70/hr on W2.

Apply Now

Save Job