A/B Testing & Experimentation EngineerRestoration of America

Position Overview: Restoration of America is seeking an A/B Testing & Experimentation Engineer to lead its testing and optimization program across digital and traditional media channels. This role operates at the intersection of data engineering, statistical analysis, and content strategy, focusing on designing experiments, building measurement infrastructure, and translating results into actionable recommendations that enhance performance.

• Design, implement, and manage A/B and multivariate testing programs across web properties, social media platforms, paid advertising, SMS/MMS campaigns, and print distribution.

• Build and maintain experimentation infrastructure including data pipelines, tracking systems, and reporting dashboards.

• Define testing methodologies, sample size requirements, and statistical frameworks to ensure valid, reliable results across different content types and distribution channels.

• Analyze experimental outcomes using appropriate statistical techniques—frequentist and Bayesian approaches, significance testing, confidence intervals, and effect size estimation.

• Identify confounding variables and design controls for cross-channel attribution challenges.

• Develop predictive models to forecast content performance and prioritize testing opportunities.

• Partner with content creators, marketing teams, and leadership to translate business questions into testable hypotheses.

• Present findings in clear, non-technical terms and provide specific, actionable recommendations.

• Build a culture of data-driven decision-making by educating stakeholders on experimentation best practices.

Required Qualifications:

• Minimum 5 years of experience in experimentation, data engineering, or applied data science

• Strong foundation in statistics including hypothesis testing, regression analysis, and experimental design

• Proficiency in Python or R for statistical analysis and data manipulation

• Experience with SQL and data pipeline tools

• Track record of designing and analyzing experiments at scale

• Excellent communication skills with ability to translate complex findings for non-technical audiences

Preferred Qualifications:

• Experience with Meta Ads Manager and Meta Business Suite analytics

• Familiarity with YouTube Studio analytics and YouTube advertising experiments

• Google Ads (AdWords) campaign testing and optimization experience

• SMS/MMS platform experience (Twilio, Attentive, or similar) and mobile messaging analytics

• Background in print media measurement methodologies including response tracking, circulation analysis, and direct mail testing

• PostgreSQL experience for data warehousing and analytical queries

• Experience with causal inference methods beyond standard A/B testing

• Knowledge of multi-armed bandit or Bayesian optimization approaches