> ## Content Index
> Fetch the complete content index at: https://shakeride.com/guides/llms.txt
> Use this file to discover other available public pages before exploring further.

# Log ingestion spikes
- URL: https://shakeride.com/guides/log-ingestion-spikes/
- Published: 2026-11-05T08:00:00.000Z
- Updated: 2026-11-09T20:15:20.000Z
- Description: Logs are billed when they are written, not when they are read or kept. How to find the source that floods ingestion and cut it at the source on CloudWatch Logs, Azure Monitor Logs and Cloud Logging.
- Author: Shakeride
- Tags: Cloud costs, AWS, Azure, Google Cloud, CloudWatch, Azure Monitor, Cloud Logging

**In one sentence:** log ingestion is billed per byte on the way in, so a debug level left on or a chatty new source costs the same whether anyone reads it or not, and shorter retention does not undo it.

## Symptom

The logging line of the bill jumps while traffic stays flat. Ingested bytes per day step up overnight, usually after a deploy, an incident or a new diagnostic setting, and one log group, table or resource type carries most of the increase.

## How it happens

![One application writing to three log stores. A debug level left on, a retry loop logging every attempt and a newly enabled diagnostic category all add bytes at the ingestion meter, which bills before retention. Retention only trims storage later.](https://shakeride.com/guides/content/images/2026/11/log-ingestion-spikes-ingestion-meter.svg)

- **The meter sits at the door:** every byte is billed when it lands. Retention, archive tiers and queries come later and do not refund it.
- **Typical sources:** `DEBUG` left on after an incident, a retry loop that logs each attempt, health checks logged per request, a diagnostic category enabled with "all logs".
- Find the top source by bytes first, then cut it where it is produced: log level, sampling, exclusion. Cut retention only for storage.

## Act

### AWS

Rank log groups by the `IncomingBytes` metric of the `AWS/Logs` namespace with the metric math below. For a Lambda function, set the level in its logging config instead of redeploying code. Log groups that are only kept for audit can be created with the `INFREQUENT_ACCESS` class, which ingests at a lower rate.

```text
SORT(SEARCH('{AWS/Logs,LogGroupName} MetricName="IncomingBytes"', 'Sum', 3600), SUM, DESC)

```

```bash
aws lambda update-function-configuration --function-name checkout \
  --logging-config LogFormat=JSON,ApplicationLogLevel=WARN,SystemLogLevel=WARN

```

### Azure

Query the workspace's `Usage` table to rank billable tables. Trim the diagnostic setting that feeds the top table to the categories you use, move verbose tables to the Basic plan, and set a daily cap as a backstop, knowing it drops data once reached.

```kusto
Usage | where TimeGenerated > ago(1d) and IsBillable == true
| summarize GB = sum(Quantity) / 1000 by DataType | sort by GB desc

```

```bash
az monitor log-analytics workspace update -g rg-obs -n law-prod --quota 20

```

### Google Cloud

Chart `logging.googleapis.com/billing/bytes_ingested` grouped by resource type to find the source. Drop what nobody needs with an exclusion on the `_Default` sink: excluded entries are not billed. For load balancers, lower the logging sample rate rather than logging every request.

```bash
gcloud logging sinks update _Default \
  --add-exclusion=name=drop-debug,filter='severity<INFO'

```

## Keep watch

### AWS

Alarm with anomaly detection on `IncomingBytes` for the account and for the top log groups. Prove the fix: the hourly sum for the noisy group falls back to its baseline after the change.

### Azure

A log search alert on the `Usage` query above fires when a table exceeds its usual daily volume. Prove the fix: the top table's billable volume falls the next day, with no daily cap hit.

### Google Cloud

Alert on `billing/bytes_ingested` per resource type. Prove the fix: `gcloud logging read 'severity<INFO' --freshness=1h --limit=5` returns nothing, because excluded entries are never stored.

## Exercise

Bring a log bill back in line, with the clock running: [Log storage costs keep climbing](https://shakeride.com/challenges/log-storage-costs-keep-climbing?ref=shakeride.com).

Related: [Audit logging of data access](https://shakeride.com/guides/data-access-audit-logging-costs/).