Advancing AI research at NAACL 2025

Capital One researchers and technologists are participating in the upcoming NAACL 2025 conference, where we’ll showcase our latest research and engage with the broader natural language processing (NLP) academic community. Capital One is dedicated to pushing the boundaries of NLP as we work to advance the state of the art in the areas of multilingual understanding, model efficiency, and AI safety. Expect paper presentations, poster sessions, hands‑on tutorials on efficient NLP methods, the TrustNLP workshop and opportunities to connect with our team at Booth 4.

Research led directly by our Enterprise AI teams examines new methods for AI safety and real‑world NLP applications and includes end‑to‑end red teaming frameworks, robust evaluation techniques and deployment insights. 

Building Safe GenAI Applications: An End-to-End Overview of Red Teaming for Large Language Models

The rapid growth of Large Language Models (LLMs) presents significant privacy, security, and ethical considerations and potential risks. This paper provides a concise and practical overview of the LLM red teaming literature, structured so as to describe a multi-component system end-to-end. Our survey will be useful for any reader who wants to gain a clearer understanding of the major red teaming concepts for their own use in practical applications. 

• Capital One authors: Alberto Purpura, Sahil Wadhwa, Jesse Zymet, Akshay Gupta, Andy Luo, Melissa Kazemi Rad, Swapnil Shinde, Shahed Sorower
• Details: Workshop session on Saturday, May 3rd, 9:00 AM - 5:30 PM GMT

Capital One is also committed to fostering collaboration with the broader AI research community. We are excited to share additional NLP research conducted in partnership with leading institutions as we work to advance the field through shared knowledge and innovation. Our joint efforts focus on creating open source benchmarks and tools that address challenges in low‑resource and culturally diverse NLP settings. 

WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines

Vision Language Models (VLMs) often struggle to understand culture-specific knowledge, particularly in languages other than English and in underrepresented cultural contexts. To address this, we introduce WorldCuisines, a massive-scale benchmark for multilingual and multicultural, visually grounded language understanding. This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points, making it the largest multicultural VQA benchmark to date. 

• Capital One authors: Genta Winata, Anirban Das, Austin Zhang
• Details: In Person Poster Presentation Session C, Poster Session 2, Wednesday, April 30, 2:00 - 3:30 GMT. Check the reader board on site for Poster Board Number location.

ProxyLM: Predicting Language Model Performance on Multilingual Tasks via Proxy Models 

This paper introduces ProxyLM, a scalable framework for predicting Language Model (LM) performance using proxy models in multilingual tasks. These proxy models act as surrogates, approximating the performance of fine-tuned LMs on specific downstream natural language processing (NLP) tasks. By leveraging proxy models, ProxyLM significantly reduces computational overhead on task evaluations, achieving up to a 37.08x speedup compared to traditional methods. This research demonstrates the power of efficient model evaluation and selection. 

• Capital One author: Genta Winata
• Details: In Person Poster Presentation Session J, Poster Session 7; Friday, May 2, 9:00 - 10:30 AM GMT

We are also proud to sponsor and organize the TrustNLP workshop at NAACL 2025. This NLP workshop focuses on the critical topic of building trustworthy and reliable NLP systems which will foster important discussions and research that will be essential for the responsible development and deployment of AI technologies. The workshop will unite leading researchers and practitioners to exchange best practices and pioneer solutions that ensure NLP models are fair, transparent and secure.

We invite you to visit our booth at NAACL 2025 to learn more about our current NLP research, career opportunities and how we are leveraging AI to transform the financial industry. Our team will be available to discuss our papers, answer your questions and share insights into our work. 

• Visit our booth: Booth 4 in the Expo Hall 

• Explore our research: Dive deep into our latest advancements in AI and machine learning.

• Discover career opportunities: Learn about exciting applied research career paths at Capital One for researchers and engineers passionate about AI and join our world-class team.

• Learn about our student and grad internships: Put your knowledge and skills to work in our 10-week to two year graduate programs innovating new products and creatively solving the problems that impact our customers and our business.

• Engage with our team: Meet our researchers and AI experts, ask questions and discuss the future of AI in finance.