Experimentation Analytics Lead

Malvern, PA, US

January 22, 2019
To define, gather and analyze measures of Vanguard’s client experience across multiple channels (with the primary focus being the web). To be an advocate for measuring client experience. Supports the Client Experience (CX) journey labs by informing and educating Vanguard about clients and prospects through descriptive, diagnostic and predictive analytic insights.

Duties and Responsibilities:

  • Serves as the lead measurement analyst across multiple client experience journey labs, collaborating with a cross-functional team from the business, design and development disciplines. Represents the analytics team, providing guidance on measurement and experiment design, ensuring correct methodology is followed and translating data insights into actionable insights that the team can utilize to help improve the client experience. Attends daily stand-ups, provides analysts guidance, tracks all key metrics and experiment results and participates in the decision making process in the lab.
  • Develops queries and performs extensive programming to access, transform and prepare data for statistical analysis. Identifies and diagnoses data inconsistencies and errors, forages to fill data gaps and blazes new data trails. Creates derived characteristics.
  • Performs deep dive descriptive, diagnostic, predictive and prescriptive analytics for customer behavioral segmentation, look-alike targeting, growth opportunity estimation and next best action identification.
  • Prepares and delivers insight presentations and action recommendations. Communicates complex material in layman’s terms. Educates leaders and crew on analytical findings and implications using storytelling, data visualization and graphical artisanship.
  • Designs analytic plans, translating business problems/opportunities into analytic approaches. Engages with project sponsors and stakeholders to understand each business question. Probes for hidden questions and goals. Brings structure to each request and translates requirements into an analytic approach.
  • Analyzes data to describe client behavior and preferences, understand channel usage, diagnose client experience problems, size marketing opportunity, target clients for treatment and measure the impact of marketing and servicing initiatives. Identifies and utilizes appropriate analytic methods and specialized tools to generate insights, answer the business questions and fulfill project objectives. Seeks out innovative approaches and techniques.
  • Leads test design, research design and experimentation validation across multiple journey lab teams. Provides statistical consultation services. Serves as the analytics expert on cross functional teams for large strategic initiatives.
  • Develops and applies domain knowledge of Vanguard’s businesses and data to shape the analytic road map.
  • Contributes to the growth of the analytic community and provides senior level guidance across multiple lab teams. Mentors, coaches and develops junior analysts.
  • Participates in special projects and performs other duties as required.


  • Undergraduate degree (BA/BS) in an analytic/quantitative field such as: Applied Mathematics, Statistics, Economics, Finance, Science/Engineering, Computer Science or related analytical field of study or equivalent combination of training and experience. A graduate level degree in an analytics focused field is strongly preferred.
  • A minimum of 8 years general experience and 8+ years of relevant professional experience in a data analysis role.
  • Expert level skills in the following (or related) suite of data analysis tools: Data Extraction languages such as SQL and experience with relational databases or Big Data platforms, Data Analysis packages (Python, R, SAS, MATLAB, etc.) and Data Visualization packages (Tableau, etc.)
  • Experience working with MS Office suite tools (PowerPoint, Excel, etc.)
  • Exceptional analytical skills coupled with the ability to form hypotheses and draw meaningful insights from disparate data sources – ability to understand data, process it, extract value from it, visualize it and communicate insights to audiences of various backgrounds in data, including senior management.
  • Demonstrated expert data-wrangling programming skills to access, transform and prepare large scale data for statistical analysis or modeling.
  • Significant and strong experience communicating complex concepts and analysis, able to handle a variety of audience knowledge levels. Highly advanced data visualization and storytelling skills.
  • Experience utilizing advanced consulting skills to understand business problems/opportunities, translating needs into analytic projects and applying results. Exceptional ability to quickly develop business acumen and understand business processes.
  • Strong experience developing creative analytical solutions to marketing and client service problems and opportunities and experience leading implementation of those solutions.
  • Significant and expert-level experience in most of the following marketing analytics areas: customer acquisition, digital marketing, response modeling, identifying customer needs, predicting customer behavior, next best action, diagnosing customer struggle, attrition/churn, customer lifetime value, sales territory analytics.
  • Significant experience leading, designing and implementing marketing or digital experimentation (A/B or MVT testing) to improve marketing performance or improve the client experience.
  • Experience leading or consulting on the process of creating metrics and KPIs and creating dashboards to track and help visualize performance metrics.
  • Significant experience formulating and evaluating hypotheses using statistical hypothesis testing. Applied experience developing confidence intervals, comparing differences in two or multiple groups, and an understanding of the assumptions underpinning respective tests.
  • Ability to understand and explain implementation of advanced statistical and machine-learning methods such as: regression models, propensity models, segmentation, optimization, time series analysis, cohort analysis, Bayesian methods, decision trees, text analysis, etc.

Special Factors:

Vanguard is not offering visa sponsorship for this position.