> ## Content Index
> Fetch the complete content index at: https://shakeride.com/guides/llms.txt
> Use this file to discover other available public pages before exploring further.

# GCP troubleshooting practice: drills for projects, IAM and Pub/Sub
- URL: https://shakeride.com/guides/gcp-troubleshooting-practice/
- Published: 2026-10-20T08:00:00.000Z
- Updated: 2026-11-09T20:15:20.000Z
- Description: GCP troubleshooting practice for the first ten minutes of an incident: confirm the project, read bounded logs, ask Admin Activity audit logs what changed, and follow the service agent that does the failing step.
- Author: Shakeride
- Tags: Practice, Google Cloud, Cloud Logging, Cloud Audit Logs, Pub/Sub

**In one sentence:** on Google Cloud, rehearse the first ten minutes until they are automatic: confirm the project, read bounded logs, ask Admin Activity audit logs what changed, then name the identity that performs the failing step and read the IAM policy on that one resource.

## Symptom

A Pub/Sub subscription never dead-letters, or a Cloud Run service cannot read a secret it read yesterday, and nothing in the application says why. Or every list command comes back empty, because the active project is not the one in the brief.

## How it happens

![Five steps from left to right: confirm the account and project with gcloud config list, read errors in Logs Explorer bounded in time, read Admin Activity audit logs for the window, name the identity doing the failing step and read the IAM policy on that resource, then replay the user action. If the symptom remains, an arrow returns to step two with a new hypothesis.](https://shakeride.com/guides/content/images/2026/11/gcp-troubleshooting-practice-evidence-ladder.svg)

- Resources, IAM, logs and quotas are scoped to a project: the wrong project returns empty results that look like evidence.
- Managed features act as Google-managed service agents, such as `service-PROJECT_NUMBER@gcp-sa-pubsub.iam.gserviceaccount.com`. A cleanup that removes unfamiliar accounts removes their grants, and nothing errors.
- Admin Activity audit logs are always on; Data Access logs mostly are not, so an empty result there proves nothing about reads.

## Act

Thirty minutes in a sandbox project, on a failure you did not build. Say what each command should return before you run it, and write the answer with its UTC time.

```bash
gcloud config list --format="value(core.account,core.project)"
gcloud logging read 'resource.type="cloud_run_revision" AND severity>=ERROR' --freshness=30m --limit=20
gcloud logging read 'logName:"cloudaudit.googleapis.com%2Factivity"' --freshness=1h --limit=20 \
  --format="table(timestamp,protoPayload.methodName,protoPayload.authenticationInfo.principalEmail)"
gcloud pubsub topics get-iam-policy orders-dead-letter

```

Grant the narrowest role on the resource the feature touches, never a project-wide role that hides the symptom. Verify by replaying the user's action.

## Keep watch

Log each drill: did the project check come first, minutes to name the failing identity, and how many grants you made at project level rather than on a resource. Both of the last two should fall over the weeks.

## Exercise

Follow the service agent in [Bad messages with nowhere to go](https://shakeride.com/challenges/bad-messages-with-nowhere-to-go?ref=shakeride.com), then restore one narrow grant in [A service can no longer read its secret](https://shakeride.com/challenges/a-service-can-no-longer-read-its-secret?ref=shakeride.com).

Related: [AWS troubleshooting practice](https://shakeride.com/guides/aws-troubleshooting-practice/), [Azure troubleshooting practice](https://shakeride.com/guides/azure-troubleshooting-practice/).