Optimize your data warehouse using your existing queries

Snowflake Integration

This guide will help you integrate Snowflake with Twing Data to optimize your data warehouse performance, reduce costs, and gain valuable insights into your Snowflake usage patterns.

What Data We Collect

Twing Data analyzes your Snowflake metadata to provide actionable insights. We collect information from two key account usage views:

QUERY_HISTORY - Contains detailed information about SQL operations, including execution time, bytes scanned, and warehouse utilization. This helps identify performance bottlenecks and optimization opportunities.
WAREHOUSE_METERING_HISTORY - Provides detailed metrics about warehouse usage, including credits consumed and computation time. This enables us to calculate the cost per query and powers our recommendations and insights.

How We Use Your Data

Twing Data processes your Snowflake metadata to provide:

Cost optimization recommendations based on warehouse sizing and auto-suspend settings
Query performance insights to identify slow-running queries and optimization opportunities
Resource utilization patterns to help right-size your warehouses and control costs
Data access patterns to improve security and resource allocation
Warehouse usage analytics to optimize credit consumption and performance

Live Integration

The live integration creates a dedicated environment with minimal permissions for secure, continuous monitoring. This includes a custom role, user, and warehouse specifically configured for Twing Data's analysis needs.

-- Create user and role
use role useradmin;
create user if not exists twing_user
  password = 'RANDOM_PASSWORD_HERE'
  default_role = twing_user_role
  default_warehouse = twing_user_dw
  comment = 'Used by Twing Data'
;
create role if not exists twing_user_role
  comment = 'Used by Twing Data';
grant role twing_user_role to role sysadmin;
grant role twing_user_role to user twing_user;

-- Create warehouse
use role sysadmin;
create warehouse if not exists twing_user_dw
  warehouse_size=xsmall
  auto_suspend=60
  initially_suspended=true
  comment = 'Used by Twing Data';
grant monitor, operate, usage, modify
  on warehouse twing_user_dw to role twing_user_role;

-- Grant table permissions
grant imported privileges on database snowflake
  to role twing_user_role;
use role accountadmin;
grant monitor usage on account to role twing_user_role;

Note: The created resources have minimal permissions focused on monitoring and analysis. The warehouse is configured for cost-efficiency with auto-suspend enabled.

One-time Integration

For teams preferring a one-time analysis, this option extracts historical query and warehouse usage data for a comprehensive performance audit. The analysis provides insights into query patterns, resource utilization, and cost optimization opportunities.

-- The query history
select *
from snowflake.account_usage.query_history
where query_type in (
    'ALTER_TABLE', 'ALTER_TABLE_ADD_COLUMN', 'ALTER_TABLE_DROP_COLUMN',
    'COPY',
    'CREATE_TABLE', 'CREATE_TABLE_AS_SELECT', 'CREATE_VIEW',
    'DELETE', 'DROP', 'INSERT', 'MERGE',
    'RENAME_TABLE', 'SELECT', 'TRUNCATE_TABLE', 'UPDATE'
)
and query_text not in (
  'SELECT 1;',
  'SELECT CURRENT_SESSION()'
)
order by start_time desc
limit 10000; -- Specify limit to match desired rows and file size

-- The warehouse usage
-- Note that this table is likely small and can be exported
-- without any filters
select *
from snowflake.account_usage.warehouse_metering_history;

The extracted data focuses on meaningful operations while filtering out system queries. This provides a comprehensive view of your Snowflake usage patterns while maintaining data privacy and security.

← Back to Docs