> ## Content Index
> Fetch the complete content index at: https://shakeride.com/guides/llms.txt
> Use this file to discover other available public pages before exploring further.

# Object storage egress and CDN caching limits: the deep dive
- URL: https://shakeride.com/guides/object-storage-egress-cdn-caching-limits-deep-dive/
- Published: 2026-10-24T08:00:00.000Z
- Updated: 2026-11-09T20:15:20.000Z
- Description: Every measurement, setting and trap behind a CDN that does not cut bucket egress, on S3 with CloudFront, Blob Storage with Front Door and Cloud Storage with Cloud CDN, with the commands to run.
- Author: Shakeride
- Tags: Cloud costs, AWS, Azure, Google Cloud, S3, CloudFront, Blob Storage, Front Door, Cloud Storage, Cloud CDN, #deep-dive

The two-minute version is [Object storage egress and CDN caching limits](https://shakeride.com/guides/object-storage-egress-cdn-caching-limits/). This page keeps every detail.

Putting a CDN in front of a bucket is the standard answer to a growing data transfer line on the bill, and it is often the right one. It only works under two conditions that nobody checks after the first deployment: requests must hit the cache, and clients must not be able to reach the bucket without going through the CDN. When either condition fails, the bucket keeps serving bytes to the internet, the CDN adds its own requests on top, and the dashboards show a CDN that is in place and doing very little.

This guide covers Amazon S3 with CloudFront, Azure Blob Storage with Azure Front Door, and Google Cloud Storage with Cloud CDN. The mechanics are the same on all three; the setting names, the metrics and the default behaviour differ.

## Recognise the pattern

Storage egress that keeps climbing after a CDN went live is the main symptom. Others are a cache hit ratio that never rises above a low plateau, origin request counts that track client request counts almost one for one, a bucket hostname that appears in access logs, page source or a mobile app build, and a sudden rise after a deployment that changed headers or query strings rather than content.

Two separate paths produce it, and they need different fixes:

- **Cache misses.** The CDN forwards most requests to the origin because the cache key is too wide, the response is marked uncacheable, or the TTL is so short that objects expire before the next request. Every miss is an origin read, so the bucket still serves the bytes.
- **Cache bypass.** The bucket is publicly readable and some clients use its own hostname. Those requests never touch the CDN, so no CDN setting can help. Crawlers and hotlinkers find bucket URLs quickly once one is published anywhere.

![Three lanes, one per provider, showing clients reaching the bucket through the CDN, where a miss with a wide cache key or no-store header becomes an origin read, and a dashed path where clients skip the CDN and read the public bucket hostname directly.](https://shakeride.com/guides/content/images/2026/11/object-storage-egress-cdn-caching-limits-deep-dive-egress-paths.svg)

## Measure before changing anything

Get three numbers for the same time window before touching configuration: requests served by the CDN, requests and bytes served by the bucket, and the split of CDN responses between hits and misses. If bucket bytes are close to CDN bytes, the cache is not absorbing anything. If bucket requests exceed CDN misses, something reaches the bucket without the CDN.

Measure over at least a full day so batch jobs and traffic peaks are included, and compare with a period before the last deployment. Billing data lags by hours, so use service metrics for diagnosis and billing data to confirm the result afterwards.

## Amazon S3 with CloudFront

Data transfer from S3 to CloudFront is not charged as internet egress, but every CloudFront miss is still an S3 GET request, and every direct request to the bucket is internet egress from S3\. In Cost Explorer, group S3 usage by usage type and look at the `DataTransfer-Out-Bytes` line for the Region; it should fall once the distribution takes the traffic.

On the distribution, enable the additional metrics and read `CacheHitRate`. Then check which cache policy each behaviour uses. `CachingOptimized` keeps query strings, cookies and most headers out of the cache key and honours origin `Cache-Control`; `CachingDisabled` sends every request to the origin and is often left on a static path after it was copied from an API behaviour. A custom policy that includes all query strings or all cookies creates one cache entry per unique value, so a tracking parameter alone can drop the hit rate close to zero. Forward what the origin needs with an origin request policy instead, which does not widen the key.

```bash
aws cloudfront get-distribution-config --id E1A2B3C4D5E6F7 \
  --query 'DistributionConfig.{default:DefaultCacheBehavior.CachePolicyId, paths:CacheBehaviors.Items[].{path:PathPattern, policy:CachePolicyId}}'

aws cloudfront get-cache-policy --id 658327ea-f89d-4fab-a63d-7e88639e58f6 \
  --query 'CachePolicy.CachePolicyConfig.{name:Name, minTtl:MinTTL, defaultTtl:DefaultTTL, key:ParametersInCacheKeyAndForwardedToOrigin}'

```

Inspect responses as well: `x-cache` and the `x-edge-result-type` field in standard logs report `Hit`, `RefreshHit`, `Miss` or `Error`. Objects uploaded with `Cache-Control: no-store` or `private` are never cached, whatever the policy says, unless the policy's minimum TTL overrides the header. For a global audience, Origin Shield adds one regional cache layer and collapses misses from many edge locations into one origin request.

For the bypass path, check the bucket. With Origin Access Control on the distribution, the bucket policy grants `s3:GetObject` to the `cloudfront.amazonaws.com` service principal conditioned on the distribution ARN, and Block Public Access stays on at both the account and the bucket level. A bucket policy with `"Principal": "*"`, or a static website endpoint used as the origin, means the bucket answers anyone.

```bash
aws s3api get-public-access-block --bucket assets-prod
aws s3api get-bucket-policy-status --bucket assets-prod
aws s3api get-bucket-policy --bucket assets-prod --query Policy --output text

```

To measure what still reaches the bucket, enable S3 request metrics on a filter for the assets prefix and read `GetRequests` and `BytesDownloaded`. Server access logs or CloudTrail data events then show the user agents and referrers of requests that did not come from CloudFront.

## Azure Blob Storage with Azure Front Door

Front Door caches only on routes where caching is enabled; a route created without it proxies every request to the origin. On each route that serves blobs, check that caching is on, then check the query string caching behaviour. "Use query string" caches one entry per unique query string, which is correct for content that varies by query and wrong for assets whose URLs carry tracking or cache-busting parameters. "Ignore query string" or "Ignore specified query strings" restores the hit ratio. Rule sets can override cache expiration per path; a rule set that bypasses cache for a whole route is a common leftover from debugging.

```bash
az afd route show --resource-group rg-web --profile-name fd-web \
  --endpoint-name assets --route-name blobs \
  --query "{caching:cacheConfiguration, ruleSets:ruleSets[].id, patterns:patternsToMatch}"

```

Front Door reports `Byte Hit Ratio` and `Origin Request Count` as metrics on the profile. For a per-request view, send `FrontDoorAccessLog` to a Log Analytics workspace and group by `cacheStatus_s`. `HIT` and `REMOTE_HIT` are served from cache; `MISS`, `PARTIAL_HIT`, `CONFIG_NOCACHE` and `UNCACHEABLE` reach the origin, and the last two say the configuration or the response headers forbid caching. Large blobs are fetched from the origin in 8 MB chunks, so partial hits are expected on large downloads.

```kusto
AzureDiagnostics
| where Category == "FrontDoorAccessLog"
| summarize requests = count(), bytes = sum(toreal(responseBytes_s)) by cacheStatus_s
| order by bytes desc

```

For the bypass path, read the storage account. `allowBlobPublicAccess` set to true together with a container whose public access level is `blob` or `container` lets anyone read the blobs at the account hostname. With Front Door Premium, a Private Link origin lets you set the storage account's public network access to disabled, so only Front Door reaches it. On Standard, at least turn off anonymous access and authorise Front Door with a SAS on the origin path, rotated like any secret.

```bash
az storage account show --resource-group rg-web --name stassetsprod \
  --query "{anonymous:allowBlobPublicAccess, network:publicNetworkAccess}"
az storage container list --account-name stassetsprod --auth-mode login \
  --query "[].{name:name, access:properties.publicAccess}"

```

Then measure the account directly. The storage `Egress` and `Transactions` metrics split by the `ApiName` and `Authentication` dimensions; `Anonymous` transactions on `GetBlob` that are not explained by Front Door misses are clients using the account hostname.

## Google Cloud Storage with Cloud CDN

Cloud CDN serves a bucket through a backend bucket on an external Application Load Balancer, and caching applies only when `enableCdn` is true on that backend bucket. Check the cache mode next. `CACHE_ALL_STATIC`, the default, caches static content types and honours origin headers; `USE_ORIGIN_HEADERS` caches only what the object metadata allows, so objects without `Cache-Control` are never cached; `FORCE_CACHE_ALL` caches everything, including objects that should stay private. The default TTL is 3600 seconds and the maximum TTL 86400 seconds unless you change them.

```bash
gcloud compute backend-buckets describe assets-backend \
  --format="yaml(bucketName,enableCdn,cdnPolicy.cacheMode,cdnPolicy.defaultTtl,cdnPolicy.maxTtl,cdnPolicy.cacheKeyPolicy)"

```

The cache key includes the full query string by default. If asset URLs carry parameters that do not change the content, set a cache key policy with an allowlist of the query parameters that matter, or none. Check object metadata too: Cloud Storage serves publicly readable objects with `Cache-Control: public, max-age=3600` by default, but an object uploaded with `no-store` or `private` is never cached in the default mode.

```bash
gcloud compute backend-buckets update assets-backend \
  --cache-key-query-string-whitelist=v

gcloud storage objects describe gs://assets-prod/app/main.js \
  --format="value(cache_control,content_type)"

```

Load balancer request logs carry `httpRequest.cacheLookup`, `httpRequest.cacheHit` and `httpRequest.cacheFillBytes`. In Cloud Monitoring, the load balancer `request_count` metric has a `cache_result` label (`HIT`, `MISS`, `DISABLED`), which gives the hit ratio without parsing logs.

For the bypass path, look at the bucket IAM policy. A binding of `allUsers` to `roles/storage.objectViewer` makes every object readable at `storage.googleapis.com`, whatever the CDN does. To keep the bucket private behind Cloud CDN, grant read access to the Cloud CDN fill service account, `service-PROJECT_NUMBER@cloud-cdn-fill.iam.gserviceaccount.com`, and serve content through signed URLs or signed cookies. Enforce public access prevention on the bucket once the binding is gone.

```bash
gcloud storage buckets get-iam-policy gs://assets-prod --format=json
gcloud storage buckets update gs://assets-prod --public-access-prevention

```

The bucket's `storage.googleapis.com/network/sent_bytes_count` metric, split by method, shows what the bucket still serves; compare it with the CDN's cache fill bytes over the same window.

## Apply the smallest fix

Close the bypass first, because it is a security issue as well as a cost one and no cache tuning helps it. Then fix the cache key, then the TTLs, and change one at a time so the effect of each is visible in the hit ratio.

![Three lanes showing the repaired path on each provider: clients reach only the CDN, which uses a static cache policy and a locked origin (Origin Access Control with Block Public Access on S3, a Private Link origin with anonymous access off on Blob Storage, the Cloud CDN fill account with signed URLs and no allUsers binding on Cloud Storage), each with the metric to watch.](https://shakeride.com/guides/content/images/2026/11/object-storage-egress-cdn-caching-limits-deep-dive-egress-fixed.svg)

- Lock the origin so it answers only the CDN: Origin Access Control and Block Public Access on S3, a Private Link origin or SAS-authorised origin with anonymous access off on Blob Storage, the fill service account and public access prevention on Cloud Storage.
- Search code, templates and app builds for the bucket hostname and replace it with the CDN hostname before locking the origin, or those clients break.
- Narrow the cache key to what changes the content: drop tracking parameters and cookies from it.
- Set `Cache-Control` on uploads in the deployment pipeline, with long lifetimes for fingerprinted assets and short ones for HTML, rather than relying on CDN defaults.
- Add an alert on the hit ratio and on bucket egress, so a deployment that widens the key again is visible within hours.

## Verify the result

Request an asset twice through the CDN and read the response headers: `x-cache: Hit from cloudfront` on CloudFront, `x-cache: TCP_HIT` on Front Door, and an `Age` header greater than zero on Cloud CDN. Request the same object at the bucket hostname and expect a refusal: `AccessDenied` from S3, `PublicAccessNotPermitted` or an authorisation error from Blob Storage, `AccessDenied` from Cloud Storage.

```bash
curl -sI https://cdn.example.com/app/main.js | grep -iE '^(x-cache|age|cache-control):'
curl -sI https://cdn.example.com/app/main.js | grep -iE '^(x-cache|age|cache-control):'
curl -sI https://storage.googleapis.com/assets-prod/app/main.js | head -n 1

```

Then compare the three measurements from the start over the same length of window. The hit ratio should rise, bucket requests should fall towards the CDN miss count, and bucket egress to the internet should approach zero. Confirm on the bill a few days later, since billing data trails usage.

## Avoid the traps

- Do not invalidate or purge the whole cache to fix a stale object; it turns the next minutes into a full origin read. Purge the path, or version the file name.
- Do not raise TTLs on HTML or API responses to lift the hit ratio; cache what is static and version it.
- Do not use `FORCE_CACHE_ALL` or a minimum TTL override on a bucket that also holds private objects.
- Do not treat a high request hit ratio as success; large objects dominate egress, so watch the byte hit ratio.
- Do not forget log and backup buckets: they are rarely behind a CDN and their egress comes from tools that download them.

## Practise on a live incident

These Shakeride challenges reproduce the failures above in a sandbox account:

- [Open to the whole internet](https://shakeride.com/challenges/open-to-the-whole-internet?ref=shakeride.com): a bucket readable by anyone, on S3, Blob Storage and Cloud Storage.
- [Storefront slows under a traffic surge](https://shakeride.com/challenges/storefront-slows-under-a-traffic-surge?ref=shakeride.com): an edge layer that passes the load straight to the origin, on Front Door and Cloud Load Balancing.
- [Log storage costs keep climbing](https://shakeride.com/challenges/log-storage-costs-keep-climbing?ref=shakeride.com): storage that grows without anyone reading it.