Google Cloud OAuth: From Follow-the-Steps to Production Operator

A recurring invalid_grant is not a reason to click faster. It is an invitation to identify the identity, permission, resource, and policy that actually control the system.

A Google integration can fail in a strangely punctual way.

On Monday, an application connects to Gmail, Calendar, or Drive. A user signs in, approves access, and the scheduled job works. The next week it stops with:

google.auth.exceptions.RefreshError: ('invalid_grant: Bad Request', ...)

Nothing in the application changed. Reauthorizing makes it work again—until it fails again.

The tempting response is to search for a new checklist of Google Cloud Console screens. That may repair the immediate symptom. It does not necessarily teach you why the failure occurred, which control plane owns it, or whether the change granted more authority than the application needed.

The production skill is not memorizing a sequence of buttons. It is being able to answer:

Which principal is trying to perform which operation on which resource, and which policy controls the result?

That question works for Google Cloud IAM. A related version works for OAuth. Together they turn a confusing rights problem into something diagnosable.

First, separate the control planes

Many Google authorization problems become difficult because several independent systems are treated as one giant permissions screen.

These planes interact, but they are not interchangeable.

Giving someone a powerful project role does not authorize an application to read that person's Gmail. Adding a user to an OAuth test-user list does not let that user change a project's IAM policy. A Workspace administrator can block an app that the user would otherwise consent to. An organization policy can prohibit an operation even when an IAM allow role appears sufficient.

The first diagnostic move is therefore classification, not configuration.

Why the seven-day failure happens

OAuth uses two credentials with different jobs.

An access token is short-lived and is sent to a Google API. A refresh token lets the application obtain new access tokens while the user is absent. Background Gmail, Calendar, and Drive integrations depend on that second capability, usually requested as offline access.

For a Google Cloud project configured with an External OAuth audience and a publishing status of Testing, refresh tokens normally expire after seven days. Google documents an important exception: the seven-day limit does not apply when the only requested scopes are a subset of openid, email, and profile identity scopes.

The typical failure is therefore:

User authorizes external test app
             │
             ▼
Access token + refresh token issued
             │
             ▼
Application refreshes access in the background
             │
             ▼
Testing-mode refresh token reaches its seven-day limit
             │
             ▼
Token exchange fails with invalid_grant

That is not a random API outage. It can be the configured token lifecycle doing exactly what it was designed to do.

But invalid_grant is a symptom, not a complete finding. Refresh tokens can also stop working because the user revoked access, the token went unused for six months, a Gmail-scoped grant was affected by a password change, the account exceeded token limits, the user granted time-limited access, or an administrator enforced a policy. Production code must expect refresh failure and provide a controlled reauthorization path.

Google's current OAuth 2.0 overview lists the expiration cases and the exact External/Testing exception.

“In production” removes one limit, not every risk

Moving an appropriate External application out of Testing removes the Testing-mode seven-day refresh-token behavior. It does not make refresh tokens permanent, and it does not override verification or Workspace policy.

Before changing the app's state, record:

Audience:              Internal or External?
Publishing status:     Testing or In production?
Users:                 Same Workspace organization or external?
Scopes:                Non-sensitive, sensitive, or restricted?
Data handling:         Local only, server-side storage, or transmission?
Workspace app policy:  Trusted, limited, blocked, or unknown?
Current grant:         New, revoked, time-limited, or stale?

The claim “private app means no verification” is too broad. An Internal app limited to users in its associated Google Workspace or Cloud Identity organization generally does not require OAuth verification. External apps requesting sensitive or restricted scopes can be subject to Google's verification requirements, and restricted-scope server-side data handling can trigger additional assessment requirements. Workspace administrators still control access to organizational data.

Use Google's OAuth app-state overview and verification requirements for the current rules. Do not treat an “unverified app” bypass screen as a universal production architecture.

IAM: principal, permission, resource, policy

Google Cloud IAM is easiest to reason about as a four-part record:

Principal:   Who is asking?
Permission:  What atomic capability is required?
Resource:    Where is the action being attempted?
Policy:      Which effective allow, deny, boundary, or organization rule applies?

For example:

Principal:   [email protected]
Operation:   Change a project's IAM policy
Permission:  resourcemanager.projects.setIamPolicy
Resource:    projects/example-production
Error:       PERMISSION_DENIED

A role is a bundle of permissions. The goal is not to make the error disappear by granting Owner. The goal is to choose the smallest justified predefined role at the smallest useful resource scope, then repeat the original failing operation.

Google Cloud resources also form a hierarchy:

Organization
    └── Folder
         └── Project
              ├── Cloud Run service
              ├── Storage bucket
              ├── Vertex AI
              └── Service account

Applicable grants can be inherited downward. Authority on one project does not imply authority on its parent organization or sibling projects. Corporate authority does not automatically create cloud authority: a company owner can still lack the Google identity and role required to change organization policy.

When the answer is unclear, Google Cloud's Policy Troubleshooter evaluates a principal, resource, and permission against applicable allow, deny, and Principal Access Boundary policies. Its result is only as complete as the policies the investigator is allowed to view.

Human deployers and runtime identities are different actors

Consider a small AI worker on Cloud Run:

Lee, the human deployer
          │
          │ deploys and attaches identity
          ▼
Cloud Run service
          │
          │ runs as
          ▼
factory-lab-runtime@PROJECT_ID.iam.gserviceaccount.com
          │
          ├── invokes Vertex AI
          └── accesses one bounded Storage location

There are two separate authorization questions:

1. May Lee deploy the service using that service account? The deployer needs the relevant deployment capabilities and permission to attach the service identity. For Cloud Run, that includes iam.serviceAccounts.actAs on the service account.
2. What may the running service do? The service account receives its own narrowly scoped permissions for Vertex AI, Storage, or other runtime dependencies.

A service account is therefore both a principal and a resource. It acts as a principal when it calls Vertex AI. It is a resource when another principal is allowed to attach or impersonate it.

Google's Cloud Run service-identity guidance documents the deployer and runtime sides of this relationship.

Do not begin with a downloaded service-account key

The old tutorial pattern is familiar:

Create service account → download JSON key → copy key to server

That should not be the default. A long-lived key is a possession-based credential that must be stored, restricted, rotated, monitored, and eventually removed. Prefer an attached service identity inside Google Cloud, service-account impersonation for approved operator workflows, Application Default Credentials for interactive development, or Workload Identity Federation for external workloads when those approaches fit.

Avoid confusing this with a desktop OAuth client's credentials.json. Both are JSON files, but they represent different identity systems and risks. A service-account private key authorizes software as a workload identity. An OAuth client configuration identifies an application that still needs a user consent grant.

Google recommends avoiding service-account keys whenever possible.

A troubleshooting method for the next failure

Use this sequence before changing permissions:

1. Reproduce the exact failure. Capture the complete error, time, command or API operation, active project, and identity.
2. Classify the control plane. Is this Cloud IAM, OAuth consent or token lifecycle, Workspace administration, organization policy, service-account use, or service-specific authorization?
3. Identify the actor. A human account, OAuth user-and-client grant, service account, or federated workload?
4. Identify the resource and operation. Name the exact organization, project, service, account, bucket, API, or user data involved.
5. Identify the controlling capability. For IAM, find the permission. For OAuth, record audience, publishing status, scopes, user, grant state, and Workspace policy.
6. Inspect effective policy. Remember inherited allow grants, deny policies, boundaries, organization constraints, and service-specific controls.
7. Make the smallest justified change. Avoid broad permanent privilege as a diagnostic shortcut.
8. Retest the original operation. A screen that looks correct is not proof; the previously failing operation succeeding is proof.
9. Record the result. Preserve the cause, change, scope, evidence, and whether any elevation must be removed.

Use an incident note like this:

Problem:
Control plane:
Principal:
Operation and resource:
Missing permission or invalid grant state:
Controlling policy:
Smallest change:
Verification:
Temporary or permanent:
Rollback or reauthorization path:

Lab: build and break a governed cloud worker

The fastest way to move beyond checklist-following is to create a boundary, break it intentionally, diagnose the denial, and repair only what was actually missing.

Build a small Cloud Run worker that accepts a bounded public-business event, invokes Gemini through Vertex AI, and returns a structured research artifact. The worker is prepare-only: no CRM mutation, no outreach, no private enrichment, and no messaging credentials.

Example response:

{
  "observed_fact": "Publicly reported ERP consolidation",
  "possible_operational_trigger": "systems integration",
  "target_function": "enterprise technology",
  "confidence": 0.81,
  "limitations": ["No buyer intent inferred"],
  "action": "prepare_only"
}

Lab boundary

Human operator
      │
      │ deploys
      ▼
┌──────────────────────────────┐
│ Cloud Run                    │
│ governed-worker-lab          │
│ dedicated runtime identity   │
└──────────────┬───────────────┘
               │
               ├── Vertex AI / Gemini
               ├── Cloud Logging
               └── optional bounded Storage buckets

Explicitly absent:
CRM credentials · messaging credentials · broad default identity · JSON key

Phase 1: isolate cost and blast radius

Create a sandbox project, attach the intended billing account, and configure a deliberately small budget with alert thresholds. Budget alerts are notifications, not proof of a hard spending cap, so document the actual cost control in use.

Before creating resources, be able to state which human principal is active and which project every command will affect.

Phase 2: create a dedicated runtime identity

Create a service account such as:

factory-lab-runtime@PROJECT_ID.iam.gserviceaccount.com

Do not generate a key. Attach the account to the Cloud Run service. Keep the human operator's deploy permissions separate from the runtime account's API permissions.

Phase 3: cause one useful failure

Deliberately omit a required runtime permission, then deploy or invoke the worker. Capture the denial:

Principal:
Permission:
Resource:
Operation:
Full error:

Use the error and Policy Troubleshooter or policy inspection to determine the smallest relevant role. Grant it at the narrowest practical scope and repeat the same operation.

The learning loop is:

denial → classification → evidence → minimal repair → retest

Phase 4: add bounded storage and a negative test

Optionally add separate input and result locations. Grant only the object operations the worker needs on those locations. Then ask the runtime identity to access a different bucket where it has no grant.

Expected result:

PERMISSION_DENIED

That denial is a passing governance test. It proves the workload's blast radius is bounded.

Phase 5: produce the consulting artifact

Finish the lab with a short evidence packet containing:

• the architecture and identities;
• an IAM matrix of principal, resource, role, and justification;
• the original failure and minimal repair;
• a negative test the platform correctly refused;
• the cost boundary and observed spend;
• confirmation that no static service-account key was created;
• cleanup steps for the lab resources.

Translate the cloud design into governance language:

The operator test

You are ready to steer the next Google Cloud troubleshooting session when you can explain, without relying on a console walkthrough:

1. why IAM and OAuth solve different problems;
2. when the External/Testing seven-day rule applies and when it does not;
3. why In production does not make a refresh token immortal;
4. how Workspace policy and OAuth verification differ;
5. why project administration does not imply organization administration;
6. what iam.serviceAccounts.actAs controls;
7. why the deployer and runtime service account need separate permissions;
8. why a downloaded service-account key is usually the wrong starting point;
9. why a deliberate PERMISSION_DENIED can be evidence of a correct boundary; and
10. what exact retest proves the repair worked.

The goal is not to know every Google role or remember every console label. Those will change.

The durable skill is to slow the failure down, identify the controlling layer, make the smallest justified change, and demand evidence from the operation that originally failed.