Site Reliability Engineer – Cloud Platform (ECS)

Help Vanguard reach 99.999% availability for our mission critical applications in support of our Agility pillar for the Enterprise Technology Strategy!! You will be joining a high performing enterprise cloud container service (ECS) platform which hosts the majority of the web and batch applications developed by application teams in Vanguard, with many of them providing business\mission critical functionality for our external sites and services and internal applications. IF YOU ARE A GREAT DEVELOPER, AND ARE PASSIONATE ABOUT INCREASING THE RESILIENCY OF OUR SYSTEMS, THEN THIS IS PERFECT FOR YOU! We can train on AWS container technology, ECS and partner with CTO SRE team on core SRE functions.

As a Cloud Compute SRE Lead within the ECS platform team expanding to a global multi region presence, you'll proactively seek out points of pain and opportunities for wide-reaching improvement by analyzing enterprise-wide telemetry data. Also, support reliability-centric tasks for cross-cutting concerns and applications spanning more than one sub-division. You will be expected to be a hands on developer to implement the found opportunities to enhance resiliency and harden the platform. You will partner with other shared services teams (performance, chaos, security and fraud and various ops teams) to bring a broad approach to hardening the platform’s security and resiliency posture.

In this role you will,

• Proactively seek out operational anomalies using Honeycomb, Splunk, CloudWatch, and other telemetry tools

• Execute chaos experiments and other resilience tests for spinal services and applications with cross-cutting impacts or high criticality

• Define SLIs and aligned SLOs for platform services. Implement automation via synthetic monitors and formulas to capture platform availability.

• Build\Deploy – Determine efficiencies to reduce build\deploy times and failures or application workloads. Assess build\deploy metrics to capture for further refinement and reporting

• Update application code based on findings to improve resilience and assist in automating workloads to be stood up in a Multi Region\Out of Region environment

• Improve the platform’s security posture by easing integration with modernized authorization\authentication protocols (OIDC, Auth0, Kong, Envoy) and identifying any potential vulnerabilities

• Help product and platform teams and their SRE representatives diagnose complex technical problems, including performance issues and intermittent errors

• Listen in and participate on high severity major incident calls (SEV1s, some cross-cutting SEV2s) to assist with triage and recovery –also participate in post-incident reviews for these incidents

• Review critical and complex architectures, including facilitation of FMEA exercises

• Maintains product-level runbooks for incident response to document the step-by-step process to recover from specific components within a system.

What it takes

• Undergraduate degree in a related field or the equivalent combination of training and experience

• Adept user of telemetry tools, including CloudWatch, Splunk, and Honeycomb

• Ability to read and understand application code written in NodeJS, Java, and Python

• Ability to write and update application code confidently in at least one of the following languages: NodeJS, Java, Python

• Deep familiarity with Siteminder, MFA and OIDC (Kong, envoy, OPA, etc. ) protocols and implementation

• Strong conceptual thinking to quickly understand new and complex architectures, and ongoing incidents

• Experience debugging production incidents using a combination of logs, metrics, and traces

• Familiarity with executing performance and chaos tests and analyzing results

• Experience working within the constraints of regulated workloads, including security restrictions

• Experience building cloud-native applications/platforms*

• Ability to create, interpret, and update technical architecture diagrams

Specializations that will make an impact

• Experience with Distributed Tracing implementation eg OpenTelemetry
• Experience with circuit breakers or developing circuit breaker logic and bulkhead patterns

• Experience with structured and unstructured logging frameworks
• Experience with resiliency patterns toggling, redundancy, autoscaling, idempotent methods etc.

Core Responsibilities

1. Collaborates with internal teams to evaluate the health, stability and reliability of systems/platforms. Provides subject matter expertise on architecture and programming design decisions related to availability and resilience.

2. Leads localized failure modes when new features and architecture patterns are introduced. Facilitates post-incident reviews for any client-impacting events local to the product family.

3. Leads the planning and execution of chaos experiments to meet the development and maintenance requirements of systems/platforms for the product family. Coordinates performance tests for the product family. 

4. Leads product teams in triage and troubleshooting during client impacting incidents. 

5. Ensures alignment between service level indicators and objectives within the product family. 

6. Maintains product-level runbooks for incident response, in collaboration with SRE Practitioners on each product team, to document the step-by-step process to recover from specific components within a system. Makes final decisions regarding usage of tools, libraries, and standards for SRE in situations where multiple options have been provided by SRE.

7. Participates in special projects and performs other duties as assigned.

Qualifications

• Minimum of eight years related experience, with at least two years of development experience.
• Undergraduate degree or equivalent combination of training and experience. Graduate degree preferred.

Special Factor

Vanguard is not offering visa sponsorship for this position.

About Vanguard

We are Vanguard. Together, we’re changing the way the world invests.

For us, investing doesn’t just end in value. It starts with values. Because when you invest with courage, when you invest with clarity, and when you invest with care, you can get so much more in return. We invest with purpose – and that’s how we’ve become a global market leader. Here, we grow by doing the right thing for the people we serve. And so can you.

We want to make success accessible to everyone. This is our opportunity. Let’s make it count.

Inclusion Statement

Vanguard’s continued commitment to diversity and inclusion is firmly rooted in our culture. Every decision we make to best serve our clients, crew (internally employees are referred to as crew), and communities is guided by one simple statement: “Do the right thing.”

We believe that a critical aspect of doing the right thing requires building diverse, inclusive, and highly effective teams of individuals who are as unique as the clients they serve. We empower our crew to contribute their distinct strengths to achieving Vanguard’s core purpose through our values.

When all crew members feel valued and included, our ability to collaborate and innovate is amplified, and we are united in delivering on Vanguard's core purpose.

Our core purpose: To take a stand for all investors, to treat them fairly, and to give them the best chance for investment success.

Future of Work

During the pandemic, we transitioned to a work from home model for the majority of our crew and we continue to interview, hire, and on-board future crew remotely.

As we have developed the path forward, we have taken a thoughtful approach that both maximizes the advantages of working remotely and the many benefits of coming together and collaborating in a shared workspace. We believe that in-person interactions among our crew are important for preserving our unique culture and advantageous for the personal development of our crew.

When our Crew return to the office, many will work in our hybrid model. A smaller proportion of our crew will operate in the Work from Home work model (for example, field sales crew); or in the Work from Office model (for example, portfolio managers).

The working model that your role falls into will be communicated to you in the interview process – please do ask if you are unsure. We encourage you to make the decision regarding your job interview and offer knowing which model your role will fall into. We will test and learn as our ways of working evolve and will continue to evaluate working models along the way.