Highlights from ICML 2024

The 41st International Conference on Machine Learning (ICML) was held this year in Vienna, Austria with thousands of attendees - including members of our Capital One AI teams - coming from all over the globe. ICML is a top-tier conference in artificial intelligence (AI) and machine learning (ML) where attendees, including Capital One associates, can learn about cutting edge research in generative AI, natural language processing, computer vision and reinforcement learning, as well as many other topics. 

ICML provided a number of opportunities for Capital One, such as bringing our team together, attending conference sessions presented by experts in the field and collaborating with others as we work to advance the state of the art in AI. We were also able to share some of the great work we’re doing in the AI space (not to mention, we had a little fun touring the city, too!).

Once the conference began, the clock was ticking: there were tutorials, keynotes, lightning talks, recruiting events, company expos, team dinners, poster sessions, along with many serendipitous encounters with former colleagues and new faces, all sparking inspiration for new paths ahead. 

There were many papers and presentations that caught our interest and highlighted exciting advances in the field throughout the week. Below is our selection from that list.

Training data plays a crucial role in the development of state-of-the-art large language models. “Understanding Finetuning for Factual Knowledge Extraction” and “Memorization through the lens of curvature of loss function around samples” helped us think critically about how models learn from data and ways to identify and measure problematic overfitting. Furthermore, several of our associates found a tutorial on the “Physics of Language Models” highly enlightening both from the perspective of understanding the inner workings of LLMs as well as in terms of recipes for effective data preparation.

We were also inspired by the methods for supervising and evaluating increasingly sophisticated systems shared in “Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision” and “Debating with More Persuasive LLMs Leads to More Truthful Answers”. A tutorial based on the “Lessons from the Trenches on Reproducible Evaluation of Language Models” nicely laid out, end to end, some challenges and best approaches to evaluating LLM outputs.

Capital One convened leading academics and industry thought leaders for an intimate dinner conversation focused on the latest advancements in Generative AI and the importance of building systems that are fair, safe and reliable. The intimate setting provided the cross-pollination of ideas from industry and academia. There were thought-provoking discussions on topics such as the future of agentic systems and the challenges we face in a world where the lines between human-generated and machine-generated content are blurring fast. These unique convenings have become signature conversations to help foster the open exchange of ideas and advance AI, always leading to new insights and perspectives. 

By day, Capital One associates worked shifts at the Capital One booth in the Expo Center. The booth enabled us to talk with Masters students, PhD students, Postdocs, as well as fellow industry professionals about the problems they are trying to solve (i.e. modeling with time series, graphs, sequences, etc), the challenges they are facing (i.e., need for more data and compute) and where they want to go next with their careers. Fortunately, we were able to extend many of these conversations at a networking event at a local restaurant mid-week. For those attendees evaluating new career choices, Capital One associates described the array of complex problems we are solving, as well as the foundational teams we are building to support both research and business-critical applied work.

Overall, ICML 2024 was an excellent opportunity for Capital One associates to gather important context and share experiences with leading scientists from around the world. We're bringing these exciting experiences, important research and moments of inspiration back home as we continue to think about how we can apply AI/ML to transform financial services. 

Interested in joining a world-class team that is accelerating state-of-the-art AI research in finance to change banking for good? Explore Applied Research jobs at Capital One.