Working with one of the most exciting products in Microsoft Azure, you will help with advancing Microsoft's cloud first strategy. The Azure Customer Experience (CXP) team is searching for a customer obsessed HPC/Supercomputing Site Reliability Engineer that can drive reliability and observability engineering excellence and embody our culture of inclusiveness, growth-mindset, and unwavering dedication to diversity

　　We are a fast-paced agile team in a start-up like culture where you are empowered to help shape the future. We apply software engineering approach to run operations. More specifically, responsible for defining, instrumenting, measuring SLO/SLI/SLAs and improving service availability, latency, scalability, performance, observability, and efficiency.

　　Our “no dead-ends”, “whatever it takes”, “biased for action”, “make it better than ever” philosophy ensures that every customer can realize their full potential through the Microsoft Cloud. We are fast growing team, but we make sure we are committed to remain agile. Customer first, nurturing trust, high responsiveness, automation, SLO/SLI/SLA, blameless post-mortem, observability, monitoring, alerting, and toil reduction form the foundations of our code and we work with teams across Microsoft and external customers to ensure success. We work on exciting engineering challenges in a fun and supporting environment, with access to cutting edge technology surrounded by world-class engineers.

　　Responsibilities

　　Distributed systems architecture – understand and manage the most complex systems

　　Continual reliability and performance optimisation – enhancing observability stack to improve proactive detection and resolution of issues

　　Working at bleeding edge - adopting new approaches and technologies, iterating on existing tooling to drive improvements

　　Problem solving capabilities – troubleshooting complex issues and proactively reducing toil through automation

　　Collaboration skills – working across teams to drive change and provide guidance

　　Technical expertise – depth skills and ability to act as subject matter expert in High Performance Computing

　　Capacity planning – effectively forecasting demand and react to changes

　　Incident response – rapidly detecting and resolving critical incidents. Minimising customer impact through effective collaboration, escalation (including periodic on-call shifts) and post incident reviews.

　　Regular travel to the customer site in the Southwest of England should be expected on at least a monthly basis.

　　Candidates must be eligible for Security Clearance

　　Qualifications

　　Required qualifications/experience:

　　Proven build and operational experience in HPC/Supercomputer environments, preferably labs with 100+ users.

　　Deploying and configure large-scale HPC clusters for parallel jobs and AI workloads.

　　Orchestration tools e.g. Azure CycleCloud, Bright Cluster Manager, AWS Parallel Cluster

　　Provisioning Linux-based compute nodes, drivers, software, application licensing and networking

　　Workload scheduling, PBS, Slurm, LSF, Kubernetes or similar

　　Parallel filesystems; Lustre, network filesystems NFSv4, and other cloud storage services

　　Linux/OSS: Administration and scripting languages (e.g., Bash, Python, Perl)

　　Monitoring and Performance Tuning, setting up monitoring tools, analyzing system performance metrics, and implementing improvements.

　　Effective diagnosis of complex technical issues. Experience with hardware/software system interrupts, node interrupts and application job failures · Reliability Engineering Knowledge; designing and implementing systems for fault tolerance, scalability, and resilience.

　　Preferred qualifications/experience:

　　Familiarity with tools such as Terraform, Bicep, Ansible, Spack

　　Public Cloud Infrastructure, preferably Azure. Including compute, networking, security, identity, governance and storage

　　Managing and utilizing version control systems such as Git, GitHub, or Azure DevOps.

　　Knowledge of continuous integration and continuous deployment (CI/CD) practices, including pipeline configuration and automation.

　　Experience of tools like Prometheus, Grafana, or Azure Monitor.

　　Microsoft is an equal opportunity employer. Consistent with applicable law, all qualified applicants will receive consideration for employment without regard to age, ancestry, citizenship, color, family or medical care leave, gender identity or expression, genetic information, immigration status, marital status, medical condition, national origin, physical or mental disability, political affiliation, protected veteran or military status, race, ethnicity, religion, sex (including pregnancy), sexual orientation, or any other characteristic protected by applicable local laws, regulations and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application process, read more about requesting accommodations (https://careers.microsoft.com/v2/global/en/accessibility.html) .