Optimizing Snowflake with Capital One Slingshot

Companies moving their data workloads to the cloud enjoy the benefits of scalability, flexibility and near infinite storage capacity. But without proper monitoring and oversight, cloud costs can quickly add up and inefficiencies, such as poorly written queries, can compound across an organization. 

In Snowflake, one of the most popular cloud-based data platforms, where resource usage is directly tied to costs, managing cloud costs and consumption is paramount to getting maximum value from the platform. Capital One Slingshot is a cloud-based data management solution that optimizes Snowflake usage and helps businesses get the most out of their Snowflake investments. Organizations gain greater visibility and control over their cloud costs, equipping businesses to better adopt, manage and scale Snowflake. Slingshot’s distinctive capabilities include dynamically provisioning warehouses with set schedules based on need, custom tagging that enables chargebacks and more transparency into costs and performance overall.

Understanding costs are critical aspects of managing a Snowflake environment. When organizations can see where and how they are spending in the cloud, they can make data-driven decisions about resource allocation, identify areas for potential cost optimization, and forecast cloud spending confidently. 

Snowflake provides views of individual account performance and costs, but businesses also need a holistic understanding of how the company is incurring costs across the Snowflake environment. With Slingshot’s preconfigured dashboards, organizations can get that comprehensive view of cost and performance metrics across all Snowflake accounts, along with a detailed breakdown of key metrics around cost, performance and usage of Snowflake. The dashboards pull together metrics from accounts to show, for example, the total monthly spend across Snowflake accounts, the costliest queries, or the projected runway on credits remaining. Slingshot also notifies customers of their credit usage and notable cost fluctuations such as unexpected cost spikes.

As organizations grow, the complexity around cost centers and assigning chargebacks to individuals and groups also increases. Along with cost visibility, Slingshot helps companies get even more granular with the Cost Breakdown Report, which is created from custom data object tagging. Using these tags, businesses can categorize spend data in a way that makes sense for their teams, such as by user, account, warehouse, query or line of business (LOB). The Cost Breakdown Report then visualizes and filters the results in custom reports. They can view, for example, the costliest warehouses in the last month by LOB such as marketing, sales, finance or the usage across accounts for a group of users on a new project. The greater cost visibility allows businesses to allocate costs to specific departments or functions, which enables chargeback reporting and strengthens accountability for teams.

Ensuring data workloads in the cloud are running efficiently is a time-consuming, and often manual endeavor for data teams, especially when the work to fine tune multiplies across many warehouses and teams. Slingshot helps Snowflake customers balance cost and performance with tools to fine tune warehouse schedules and by providing recommendations for rightsizing warehouses. In these ways, Slingshot takes the guesswork out of Snowflake warehouse optimization.

Custom schedules in Slingshot allow warehouses to dynamically adjust parameters to meet real-time needs in usage, environment, day or time. The schedules, which are automatically applied once they are set, allow customers to dynamically tune warehouse size and cluster settings in a predictable way. For example, a company usually needs more compute power during working hours on weekdays than off hours on the weekend. Slingshot automatically adjusts the warehouse size based on preset configurations, significantly reducing the time and effort involved in tuning warehouses manually. Dynamic scheduling ensures your business is optimizing for cost and performance and paying only for what is needed.

Slingshot empowers users to further optimize Snowflake with data-driven warehouse recommendations. Snowflake administrators historically have relied on the time-consuming task of manual trial and error to optimize warehouse configurations, a process that can lead to suboptimal results and is difficult to scale as the organization grows. Slingshot removes the guesswork and manual oversight usually involved in determining the most efficient configurations for warehouses. Slingshot continuously monitors the Snowflake environment to make recommendations for right-sizing warehouses based on historical usage patterns. These recommendations may be for adjustments in warehouse size, scaling policies or auto-suspend time. A recommendation engine monitors warehouses for issues such as high idleness, long query completion times and inactive warehouses, helping teams identify inefficiencies and warehouses that are under- or overutilized. The recommendation is applied to Snowflake automatically when accepted with full control to reverse the change. Warehouse recommendations bring even greater efficiency into dynamic scheduling, providing businesses the opportunity to set up their warehouses in the optimal configurations based on Slingshot’s analysis. Slingshot additionally provides a projection of cost savings if the recommendation is applied. 

Using Slingshot’s dynamic schedules and warehouse recommendations, organizations minimize inefficiencies while optimizing Snowflake performance over time.

Warehouse templates and streamlined approval processes in Slingshot add automation and standardization into warehouse provisioning. Role-based management allows businesses to delegate warehouse management to specific teams or users without sacrificing oversight and governance. Teams can quickly create their own warehouses with preconfigured templates that contain custom rules and warehouse parameters that have been set by administrators. Applying a schedule to the newly created warehouse adds parameters that ensure the warehouse runs as efficiently as possible. The Slingshot customer can then submit the warehouse for approval, saving time while adhering to governance requirements.

Query performance is an important factor that impacts Snowflake costs and efficiency. Poorly written queries can cause slowdowns in performance and consume more resources than necessary. At Capital One, we collected our best practices for writing efficient queries and created Query Advisor, a tool that analyzes queries to improve query text and quality for greater efficiency. In Slingshot, customers can view the costliest queries or most frequently run queries and select a query for analysis. Query Advisor then assesses the query text for inefficiencies and provides a recommendation to improve performance.

With Slingshot, companies using Snowflake can feel confident they are maximizing their investment, balancing both cost and performance in the cloud. With greater visibility, proactive recommendations and streamlined workflows, organizations can allocate resources efficiently, reduce manual dependencies and scale responsibly.