Senior Site Reliability Engineer
Location: EMBL-EBI, Hinxton near Cambridge, UK
Staff Category: Staff Member
Contract Duration: 3 years (renewable upto 9 years)
Grading: Grade 6 or 7 (Monthly Salary starting at £3,063 - £3,486 per month after tax depending on experience + other paid benefits
Closing Date: 1 June 2021
Reference Number: EBI01835
We are seeking to recruit an enthusiastic, highly motivated senior site reliability engineer to work at the European Bioinformatics Institute (EMBL-EBI) located at the Wellcome Genome Campus near Cambridge in the UK. The successful candidate will join the Software Development and Operations team (SDO) at EMBL-EBI which is responsible for bringing new technology into the EBI to improve the service provided to internal and external users and projects. The team also operates a number of key centralised services including a multi-petabyte file replication system, multi-protocol centralised file transfer services, resource useage tracking software, and an-house user account management system

The main purpose of the role is to support, maintain and extend the centralised file transfer services used both internally and by our worldwide collaborators to deliver data to and export data from many of the data services for which the EBI is world-renowned. These include the European Nucleotide Archive (ENA), European Genome Phenome Archive (EGA) and Ensembl. The postholder will also maintain, extend and operate other selected internally and externally developed software as required.

Your role

The centralised file transfer service includes Aspera, Globus, GridFTP, FTP, HTTP and rsync protocols. These services are under high volume demand from large numbers of users, averaging data ingestion in the region of 2PB per month, and over 6PB a month for downloads. The services are containerised and run in two data centres with failover capability. You will be joining the operations team which maintains the container and software infrastructure of the file transfer services and ensures that the system continues to scale to meet the growing demands. You will run the service and also actively investigate and evaluate new protocols and open software solutions to extend and update the portfolio. You will help guide other members of the team and help with day-to-day prioritisation as well as directly contributing to one or more additional key SDO operational projects.

You will participate actively in designing new systems as well as features in existing ones, contributing to the code that defines the infrastructure as well as the application. You will work closely with members of the Systems Infrastructure team; also key members of the service teams whose pipelines use the services. Main duties and responsibilities:

  • Ensure the day-to-day operation, reliability, and integrity of EMBL-EBI’s transfer services
  • Maintain and update the container infrastructure and software that underpin the transfer services and other key SDO services
  • Evaluate, recommend, design and deploy further tools and technologies to add to the portfolio of services provided
  • Personally contribute to other key SDO projects including the user accounting system.
  • Support users and investigate and resolve complex issues where in-depth evaluation of various technical and non-technical factors is required
  • Build positive relationships, communicating openly and professionally within the team and with stakeholders from the TSC and wider EMBL-EBI community 
  • Produce high quality documentation to support work done, and work actively to share knowledge within the team
  • Take an active role in team and project meetings and the daily stand-up.
  Additional to the requirements detailed here, this and all positions at EMBL-EBI are subject to our behavioural competency framework. The EMBL-EBI Behavioural Competency Framework has been developed to capture the skills, knowledge and behaviour that individuals must demonstrate to perform effectively in their role. The framework is made up of seven competencies, grouped into three clusters; core, specialist and leadership. The indicators of behaviour are not designed to be comprehensive but provide a clear and consistent sense of what is expected from individuals at EMBL-EBI. 

You have

You will have a Bachelor's degree or higher in computer science or a related discipline, or have equivalent experience. The role would be suitable for a Unix/Linux systems administrator with a software engineering background. You will have at least four years’ experience in a similar role, including successful deployment and maintenance of core services. In addition, you will have demonstrated experience of working 'full stack', integrating multiple open source tools to deploy services in an academic or commercial setting. Experience in operating and resolving problems in open-source software and contributing development fixes back to the community would be a distinct advantage. You will also have a strong working knowledge of Unix/Linux systems and systems administration, excellent Shell scripting, experience with VM and/or container deployment in public or private cloud environments, automated deployment/configuration methods (e.g. Ansible, Puppet) and a sound practical knowledge of Kubernetes. A good understanding of basic networking concepts, principles and terminology and experience with http traffic load balancing deployment and maintenance are also required.

As the role will involve liaising with different groups and users, it is essential that you have excellent communication, both written and verbal, and interpersonal skills in the English language. You will have a proven ability to work within a team and in collaboration with others, and have experience communicating and working with expert users (e.g. in a scientific community). You will also have experience mentoring less experienced colleagues. You should have the knowledge and experience to work with minimal direction when making technical decisions and be able to plan and prioritise workloads.

You might also have

Sound understanding of and practical experience of using file transfer protocols including ftp, Aspera, GridFTP, Globus. Experience with large scale computing, storage or cloud infrastructures (e.g. HPC/HTC clusters), source code managed CI/CD and automating tooling for testing and continuous deployment, and log management (syslog, rsyslog, fluentd) is also desirable.

Previous experience evaluating open-source software and its subsequent integration to a production system would also be advantageous, as would considerable experience in Python.

Familiarity with agile development practices and the full software development lifecycle in a team environment and previous experience in Service definition and ITIL/FITSM are also of interest.

Why join us

Do something meaningful
At EMBL-EBI you can apply your talent and passion to accelerate science and tackle some of humankind's greatest challenges. EMBL-EBI is a worldwide leader in the storage, analysis and dissemination of large biological datasets. We provide the global research community with access to publicly available databases and tools which are crucial for the advancement of healthcare, food security, and biodiversity.  Join a culture of innovation
We are located on the Wellcome Genome Campus, alongside other prominent research and biotech organisations, and surrounded by beautiful Cambridgeshire countryside. This is a highly collaborative and inclusive community where our employees enjoy a relaxed atmosphere. We are committed to ensuring our employees feel valued, supported and empowered to reach their professional potential.  Enjoy lots of benefits:
  • Financial incentives: Monthly family, child and non-resident allowances, annual salary review, pension scheme including 17% employer contribution, death benefit, long-term care, accident-at-work and unemployment insurances
  • Flexible working arrangements
  • Private medical insurance for you and your immediate family (including all prescriptions and generous dental & optical cover)
  • Generous time off: 30 days annual leave per year, in addition to eight bank holidays
  • Relocation package including installation grant (if applicable)
  • Campus life: Free shuttle bus to and from work, on-site library, subsidised on-site gym and cafeteria, casual dress code, extensive sports and social club activities (on campus and remotely)
  • Family benefits: On-site nursery, child sick leave, generous parental leave, holiday clubs on campus and monthly family and child allowances
  • Benefits for non-UK residents: Visa exemption, education grant for private schooling, financial support to travel back to your home country every second year and a monthly non-resident allowance.
  For more details please see our employee benefits page

What else you need to know

  • Contract duration: This position is a 3 year contract, renewable to a maximum of 9 years. 
  • International applicants: We recruit internationally and successful candidates are offered visa exemptions. Read more on our page for international applicants
  • Diversity and inclusion: At EMBL-EBI, we strongly believe that inclusive and diverse teams benefit from higher levels of innovation and creative thought. We encourage applications from women, LGBTQ+ and individuals from all nationalities. 
  • Job location: This role is based in Hinxton, UK and you will be required to relocate once it is safe to do so, if you are currently based abroad. Read more about how we are recruiting during the pandemic
  • Your application: To apply please submit a cover letter and a CV through our online system. We aim to provide a response within two weeks after the closing date.