Product Manager II, AI Evaluations

WHAT IS BOX?   Box (NYSE:BOX) is the leader in Intelligent Content Management. Our platform enables organizations to fuel collaboration, manage the entire content lifecycle, secure critical content, and transform business workflows with enterprise AI. We help companies thrive in the new AI-first era of business. Founded in 2005, Box simplifies work for leading global organizations, including JLL, Morgan Stanley, and Nationwide. Box is headquartered in Redwood City, CA, with offices across the United States, Europe, and Asia. By joining Box, you will have the unique opportunity to continue driving our platform forward. Content powers how we work. It’s the billions of files and information flowing across teams, departments, and key business processes every single day: contracts, invoices, employee records, financials, product specs, marketing assets, and more. Our mission is to bring intelligence to the world of content management and empower our customers to completely transform workflows across their organizations. With the combination of AI and enterprise content, the opportunity has never been greater to transform how the world works together and at Box you will be on the front lines of this massive shift.   WHY BOX NEEDS YOU We're looking for a high-agency, deeply AI-fluent Product Manager to own AI Evaluations — a highly cross-functional, highly visible role at the front lines of how Box assesses, launches, and manages the AI models that power our product. You'll run rigorous evaluations of frontier models, own the decision of which models power Box Agents, shape how the world's leading model providers understand enterprise performance, and own the full lifecycle of every model we support. This is a hands-on role for someone who deeply knows the AI landscape and can do their own work: statistically literate, comfortable in code, and able to build the evals, run the analyses, and prototype the pipelines themselves rather than spec them out for someone else. It sits at the intersection of technical depth, go-to-market execution, and external partnerships — connective tissue across Product, Marketing, Partnerships, and Engineering — and it's a fit for someone who thrives in ambiguity, moves fast on tight timelines, and brings order to messy, fast-evolving problem spaces.   WHAT YOU'LL DO  Run rigorous evaluations across frontier models (OpenAI, Anthropic, Google/Gemini) and continuously mature the stack, methodology, and process behind them. Define what makes a high-quality evaluation dataset and robust, cost-appropriate graders, and translate what end users care about into measurable criteria. Drive synthetic data generation and data-generation pipelines to ensure they are high-quality, reliable, and scalable. Own online evaluations, turning real production usage into rich insight on how our product is actually used. Lead each model launch end to end, managing the process, the stakeholders, and the timeline. Own the external narrative for each model launch — partnering with Product Marketing on blog, social, and PR content, and creating video demos yourself. Build and manage deep relationships with model providers and AI companies — securing early access to frontier models, delivering technical deep-dives back to providers, and identifying collaboration opportunities on evaluations and benchmarking. Identify, pitch, and structure opportunities to collaborate with providers and other AI companies specifically on evaluations and benchmarking. Run evaluations to decide whether a model is production-ready, and own the decision of which models power agentic products at Box. Assess the compliance posture of production models, partner on model pricing decisions, and define how and when we deprecate models.   WHO YOU ARE We are an AI-first company. This means you approach your work with a growth mindset and find ways to leverage AI to help make faster, smarter decisions that will 10X your impact at Box. You have a minimum of 2 years of product management experience. You know the AI landscape deeply, follow frontier models closely, and can hold your own in a technical conversation with any model provider. You're technical and hands-on : statistically literate and comfortable enough in code (Python, SQL, notebooks) to build your own evals, run your own analyses, and prototype your own data pipelines. You've driven cross-functional launches across Product Marketing, Communications, Developer Relations, and Engineering. You're an excellent written and verbal communicator who can represent Box externally, presenting complex, technical topics simply to senior stakeholders. You operate with extremely high agency, taking ambiguous, undefined problems and driving them to resolution on tight timelines. You're fluent across many stakeholders—from Product Marketing and Sales to Security, Legal, and Compliance—adapting your communication to each audience. You're a na