Experimentation Manager

Meet Upside:

We created Upside to help communities thrive! Our retail technology uses the sophistication of online retail—profit measurement, attribution, and incrementality—to provide users with more value on their everyday purchases and brick and mortar businesses with new, profitable customers. We’ve helped millions of users earn 2 to 3 times more cash back than any other product, and tens of thousands of brick and mortar businesses earn measurable profit. Billions of dollars in commerce run through the Upside platform every year, and that value goes directly back to our local retailers, the consumers they serve, and towards important sustainability initiatives. Our mission, values, and commitment to inclusivity guide our team of more than 300 people worldwide, and the quality of our culture is reflected in the impact we’ve had on communities nationwide. You’ll join a Team whose experience hails from places such as Google, Uber, Amazon, Instacart, Capital One, and MasterCard with proven startup executives and venture capitalists.

Upside was named #2 on Deloitte's 2021 list of Fastest Growing 500 Tech Companies and  #308 in Inc.’s America’s fastest growing private companies of 2022. Upside’s Series D funding was raised in March 2022 and led by General Catalyst at a $1.5 billion valuation. Other investors include Bessemer Ventures and Formation8.

Meet the Research team:

The Research team uses a combination of qualitative and quantitative methods to better understand the mindset of our customers (consumers and merchants). We conduct studies supporting product, marketing, and sales. We live-and-breathe a customer-centric philosophy to help GetUpside reach its business goals.

About the job:

As the Research Experiment Manager you will be the core team member who facilitates A/B testing of new features and functionality supporting Product and Marketing initiatives; and collaborates with the Data team on design methodology and key success metrics. Your work will help us improve our conversion and app experience. Your deep knowledge of interventions and time socializing results with business leaders will result in sharper experimentation models over time and help build a collaborative culture of testing-and-learning. 

What you'll do:

• Partner closely with product management, engineering, data science, marketing, and content teams to develop experimentation best practices to ensure product features and tests are implemented consistently across all platforms
• Prioritize and schedule experiments according to the organization’s roadmap; manage timelines from inception to launch 
• Create experiment designs and learning agendas based on data, goals and hypothesis-driven measurement plans/frameworks
• Manage the suite of analytics and experimentation products (Optimizely and future tools)
• Write concise documents communicating results to stakeholders and visualize data to drive decision making
• Conduct product analytics related to user behavior and transactions
• Evolve automated tools and dashboards into use cases, data relevance and data visualizations 
• Develop long-term consulting relationships with internal stakeholders to identify and address business issues pertaining to our customers and product categories

What you need:

• Bachelor’s degree in a technical field (e.g. statistics, data science, computer science, engineering) 
• Extensive knowledge of experimental design
• 8+ years of running experiments across mobile/app and desktop platforms
• Have a strong understanding of statistical technique and methodologies as well as an understanding of distributions, confidence-intervals, hypothesis testing, A/B Testing, and interpretation of statistical significance
• Experience with digital experimentation platforms such as Optimizely, VMO or others as well as an understanding of business intelligence and statistical software (Google Analytics, SQL, Mixpanel, Looker and Snowflake or similar tools)
• Ability to formulate measurable hypotheses and act as a controller to avoid test collisions and ensure validity of results
• Ability to QA and validate event tracking (instrumentation)
• Strong communicator with the ability to partner with a range of cross-functional stakeholders
• Working knowledge of SQL and an advanced analytical tool (Python, R, etc.)
• PMP or Six sigma certification preferred
• Understanding of the end-to-end process of a marketing campaign and tactics to drive growth

The fine print:

• Notice to recruiters and placement agencies: This is an in-house search with a dedicated resource. Please do not submit resumes to any person or email address at Upside. Upside is not liable for, and will not pay, placement fees for candidates submitted by any party or agency other than its approved recruitment partners.

At Upside, we believe that diversity drives innovation. Our differences are what makes us stronger. We‘re passionate about building a workplace that represents a variety of backgrounds, skills, and perspectives and do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. Everyone is welcome here. Come join us!