Improving Superset dashboard performance

Our embedded Superset dashboards took about 20 s to load. After four changes, a cold load takes about 5.5 s and a cached reload 1–2 s.

TL;DR

• More Gunicorn worker processes on two instances, plus HTTP/2
• A shared Redis cache in front of Snowflake
• A repeated translation join moved into a dynamic table
• Each layer traced before we changed it

The setup: Superset on Snowflake, embedded in a Vue and Express.js app. There was no single setting to blame, as the delay was spread across the request path, so we measured first and changed one thing at a time.

Where we started

We started as simple as it gets: one Superset instance with the Docker image's default Gunicorn setup (a single worker process with 20 threads) and no shared cache. Every request went straight to Snowflake.

The diagram shows where we ended up. The sections below add each piece in order.

Figure 1: The request path after the changes — from a dashboard with many charts and filters to either a Redis cache hit or a Snowflake query.

Step one: giving Superset more room to work

A dashboard fires many requests at once. One of ours sends about 18 chart requests plus filters. The default's 20 threads can accept them all, but they share one Python process. Python's GIL lets only one thread run Python code at a time, so query building and result serialization take turns.

We moved to two instances behind a load balancer, each running 5 Gunicorn worker processes × 2 threads. That's fewer threads than the default, but five processes can actually use both vCPUs, and one slow request no longer holds up the rest. Five follows 2 × cores + 1 for 2 vCPUs. Two threads per worker cover the time spent waiting on Snowflake.

gunicorn --bind 0.0.0.0:8088 \
  --workers 5 \
  --worker-class gthread \
  --threads 2 \
  --timeout 120 \
  --max-requests 1000 \
  --max-requests-jitter 50 \
  'superset.app:create_app()'

max-requests is for stability, not speed: workers restart every ~1,000 requests, so memory leaks can't build up.

We started on t4g.large instances and later moved to c7g.large. t4g instances are burstable and throttle once their CPU credits run out, whereas c7g gives full CPU all the time.

HTTP/2 mattered too. Over HTTP/1.1, a browser opens only six connections per host, so requests queue in the browser no matter how many workers you have. HTTP/2 sends them all over one connection.

Step two: adding Redis caching where it matters

We put Redis (AWS ElastiCache) behind Superset's data and filter-state caches. A hit returns in about 5 ms instead of a Snowflake round-trip.

• Shared: one cache for all 10 workers, not a copy per process.
• Free refreshes: Snowflake no longer recomputes what we just served.
• Filters too: dropdown values are data queries, so up to 11 per dashboard come from cache.
• Off the app servers: Superset's memory stays free for the workers.

Each client's RLS clause travels in its embedded-SDK guest token, and Superset makes it part of the cache key. One client's cached results are never served to another.

We kept the default 1-hour TTL. Our data only changes nightly, so that's safe.

New Relic tracing then showed the split: cache hits were fast, and the remaining time sat in the uncached Superset → Snowflake path.

Step three: finding the expensive Snowflake work

We pulled the 50 slowest uncached queries from a week of query history: execution and compilation time, partitions scanned, pruning, spilling, and result-cache hits. Two patterns stood out:

• Most hit our largest table and scanned much of it every time.
• Nearly all carried the same translation join, rebuilt for every chart.

A join repeated that often is a perfect candidate to precompute.

Step four: moving the translation join upstream

We moved the join into a Snowflake dynamic table, defined directly in Snowflake, and tested it on one chart first.

Why a dynamic table? Our models arrive nightly through a data share from another Snowflake account. The dynamic table joins the translations onto them and refreshes itself as part of the data pipeline, so Superset always reads a ready-made dataset.

Figure 2: How the data reaches Superset — analytical models arrive from the source account via a data share, get the translation join applied, and land in dynamic tables that Superset reads.

Measurement	Before	After
Single chart, average load	5.375 s	~2.3 s (−57%)
Whole dashboard, average of 3 runs	10.66 s	2.46 s (−77%)
Whole dashboard, worst cold run	21.04 s	~6 s (−71%)
Snowflake execution, single run	0.328 s	0.672 s

Execution time didn't drop, yet the chart got faster. Execution time excludes compilation and queueing, so the gain most likely sits there: the join-heavy query simply had more to compile.

Lesson: a join that rides on every chart query belongs in the data model.

Checking the warehouse itself

Was our SMALL, first-generation warehouse simply undersized? No: execution times were short and spilling was low.

We did raise MAX_CONCURRENCY_LEVEL from 8 to 16. That wasn't for speed but for headroom as concurrent users grow. Spilling and queuing stayed healthy afterwards.

What we deliberately did not enable

• Celery async queries: built for queries that run for minutes; ours take seconds.
• Snowflake clustering: our largest table prunes poorly, but at ~50M rows the scan cost is small and reclustering isn't free. We'll revisit it as the table grows.

The results

Measurement, all dashboards	Before	After
Average cold load	~20 s	~5.5 s (one 12 s outlier)
Cached reload	~5 s	1–2 s

Charts load in 2–3 s; filters roughly double that.

What we learned

• Trace the whole request path, not just the warehouse query. APM is what shows which layer the time sits in.
• Fix concurrency and caching before blaming the warehouse, and confirm it is the bottleneck before resizing it.
• Use query history to find work that repeats across queries, then materialize it upstream.
• Count the browser as part of the system: HTTP/1.1's connection limit can cap a dashboard before your servers do.
• Measure cold and warm loads separately, and keep application time apart from warehouse time.

Further reading

• HTTP/2 vs HTTP/1.1 (Cloudflare)
• Gunicorn — adding workers
• Superset caching configuration
• Snowflake dynamic tables

Thanks for reading. I hope this helps you make your dashboards a little faster!