Allow teams to build in GCP without central governance for ten years and this is what you get. A project list with names referencing products no one ships anymore. Service accounts whose keys predate the current CTO. Multiple VPCs all claiming to be "production." The outcome is predictable and well-documented at this point: without an Organization node, project-creation controls, tagging standards, and lifecycle policies from day one, the environment accretes. What looks like a security failure or an engineering failure is almost always an inventory failure first.
Governance programs typically start with policy. Write the rules first, enforce them second. In a greenfield environment, that order is fine. In a brownfield environment, policy-first governance produces two outcomes: the policy breaks things you did not know existed, or you carve so many exceptions that the policy means nothing. Neither is progress.
The right starting point is inventory. Not a spreadsheet someone maintains manually - a live, queryable record of what your organization actually owns, who owns it, and what it is doing. Without that record, every governance decision is a guess.
Flexera's 2026 State of the Cloud Report places wasted IaaS and PaaS spend at 29%. [1] Cost is the visible part of the problem. The harder part is security.
An orphaned GCP project with a service account configured for Domain-Wide Delegation is more than a cost leak; it is a path to full Workspace domain compromise. Hunters demonstrated the DeleFriend attack in 2023: a principal with project-level Editor access on a project that holds a DWD-enabled service account can mint a new private key for that account, sign a JWT, and impersonate any Workspace user, with no Super Admin involvement. [3] Google noted that large organizations commonly accumulate "thousands of unattended projects." [2] Each one is a potential attack surface.
Getting visibility required three tools, three scripts, and several hours of runtime. Cloud Asset Inventory exports a full org-wide snapshot to BigQuery. An IAM policy export loops through every project and captures who has access to what. An activity check pulls the last log entry from each project to identify which ones are dormant. Together, they answer the questions that matter: which projects lack an owner, which service account keys exceed 90 days, which projects show no activity.
The inventory work produced an uncomfortable finding: roughly 70% of the organization's GCP projects were auto-generated stubs - created as side effects of Apps Script edits, Gemini activations, and console logins - with no business purpose and no owner. About 82% of all assets lived in a single project. Drilling into that project revealed a second layer of sprawl: ~830 BigQuery datasets where over 90% were non-production (developer scratch, stale PR staging, and sandboxes), ~1,900 secrets where 69% were auto-generated by a managed data-integration platform (one secret per connector configuration), and ~3,400 KMS key versions accumulating from unmanaged rotation. The same pattern of organic accumulation without lifecycle management repeated at every level of the hierarchy.
The inventory work is not glamorous. Label every project. Identify every owner. Classify every environment tier. Then drill into the projects that remain and do it again at the resource level. But this is the map you need before you can build any road. Part 2 covers what to do with the map: the project factory, golden path governance, and how to measure compliance by adoption instead of audit.
The Anatomy of Ten Years of Organic GCP Growth
Organic GCP growth follows predictable patterns. Recognizing them is the first step toward counting them.
The most common sprawl drivers are PoC projects that survived their evaluation, developer sandboxes abandoned when engineers changed teams or left the organization, and shadow IT projects spun up by business units that needed to move faster than the ticket queue allowed. Google's own resource hierarchy documentation notes that without an Organization node, projects belong to the individual who created them. Many organizations accumulated hundreds of projects before formal Organization setup was in place.
But the largest single category is one most administrators do not expect: auto-generated projects. Google Workspace creates GCP projects automatically when a user writes an Apps Script, enables Gemini, or accesses certain APIs for the first time. These projects carry names like "Untitled project," "Default Gemini Project," or "My First Project" followed by a numeric suffix. They have no compute resources, no identified owner, and no business function. In the inventory analyzed here, auto-generated projects outnumbered intentionally created ones by roughly two to one.
Table 1 summarizes the symptom patterns and their risk profiles.
| Symptom | Description | Primary Risk |
|---|---|---|
| Auto-generated projects | Apps Script, Gemini, API console side effects | Governance noise; unmonitored attack surface |
| Orphaned projects | No active owner; bottom 10% of usage | Unpatched vulnerabilities; ongoing billing |
| Zombie VMs | Continuous runtime with near-zero CPU/network | Direct cost waste; expanded attack surface |
| Ungoverned service accounts | Keys years old; no rotation; no Workload Identity | Credential exposure; lateral movement |
| Billing leaks | Idle disks, unattached IPs, forgotten Cloud SQL | Pure financial waste |
| ClickOps resources | Console-created; bypasses IaC pipelines | Drift; no audit trail; governance blind spot |
| DWD-enabled orphaned SAs | Service accounts with domain-wide delegation in unmanaged projects | Full Workspace domain compromise |
Industry data on the cost dimension comes from two independent sources. Flexera's 2026 State of the Cloud Report (N=753, winter 2025 survey) puts wasted IaaS and PaaS spend at 29% and ranks cost management as the top cloud challenge for the fourth consecutive year. [1] Harness projects $44.5 billion in enterprise cloud waste for 2025, with 52% of engineering leaders citing the FinOps-developer gap as the primary driver. [6]
The security case is harder to quantify but more important. The DeleFriend vulnerability, documented by Hunters Security in 2023, demonstrated a practical attack chain from an orphaned GCP project to a full Workspace domain compromise. [3] The attack requires only project-level editor access to a project containing a DWD-enabled service account. No Super Admin privileges. The delegation is tied to the service account's OAuth client ID, not to specific private keys - creating a new key does not revoke existing delegation. Hunters' research found that Google Workspace's domain-level delegation model and GCP's project-level permission model create a gap that is difficult to close without isolating DWD-enabled service accounts in dedicated, tightly controlled projects.
Hunters Security demonstrated that a principal with project-level Editor access (which includes iam.serviceAccountKeys.create) on a project that holds a DWD-enabled service account can create a new private key for that account, sign a JWT for any Workspace user, and exchange it for an OAuth token, with no Super Admin involvement. [3] Delegation is bound to the service account's OAuth client ID, so creating a new key does not revoke existing delegation.
"It was not uncommon for us to come across organizations with thousands of unattended projects." - Google Cloud Product Team, Unattended Project Recommender launch, 2021 [2]
Building the Inventory Before Writing a Single Policy
The inventory phase uses three tools for different purposes: Cloud Asset Inventory for org-wide snapshots, Steampipe for real-time ad-hoc queries, and the Unattended Project Recommender for ML-based prioritization.
Cloud Asset Inventory: The Org-Wide Snapshot
Cloud Asset Inventory is a global metadata service that indexes every GCP resource, IAM policy, and org policy across an organization. [4] It supports search, export to BigQuery or Cloud Storage, real-time change feeds via Pub/Sub, and IAM policy analysis.
Table 2 lists the available content types and what each returns.
| Content Type | CLI Flag | What It Returns |
|---|---|---|
RESOURCE |
resource |
Full resource metadata (machine type, IPs, disk config, etc.) |
IAM_POLICY |
iam-policy |
IAM bindings on the resource |
ORG_POLICY |
org-policy |
Organization policies applied |
OS_INVENTORY |
os-inventory |
OS runtime inventory (packages, versions) |
RELATIONSHIP |
relationship |
Cross-resource relationships (requires SCC Premium) |
ACCESS_POLICY |
access-policy |
VPC Service Controls access policies |
Key limitation: Cloud Asset Inventory retains asset history for 35 days only. [4] For longer retention, export snapshots to BigQuery on a scheduled basis and preserve them there.
The following CLI commands cover the core inventory operations:
# Search for service account keys created before a specific date.
# createTime expects ISO 8601; quote the value inside the query string.
gcloud asset search-all-resources \
--scope="organizations/123456789012" \
--asset-types="iam.googleapis.com/ServiceAccountKey" \
--query='createTime < "2023-03-10T00:00:00Z"' \
--order-by="createTime"
# Export org-wide asset snapshot to BigQuery
gcloud asset export \
--organization=123456789 \
--content-type=resource \
--bigquery-table=projects/my-project/datasets/asset_inventory/tables/resources \
--per-asset-type
# Enable a real-time asset change feed (feeds deliver to Pub/Sub only)
gcloud asset feeds create my-asset-feed \
--asset-types="compute.googleapis.com/Instance","storage.googleapis.com/Bucket" \
--content-type=RESOURCE \
--project=<your-project-id> \
--pubsub-topic=projects/<your-project-id>/topics/<your-pubsub-topic>
When using --per-asset-type, the export creates separate BigQuery tables per asset type (e.g., resources_compute_googleapis_com_Instance). With --per-asset-type, each table includes typed RECORD columns mapped from the nested fields of Resource.data, up to 15 levels deep. The resource.data JSON-string column appears only in the unified single-table export.
Building a Project Registry in BigQuery
The practical registry combines three data sources: the Cloud Asset Inventory export, the billing export, and project labels. The SQL query below identifies projects missing required labels - the starting point for ownership cleanup:
SELECT
name AS project_id,
display_name,
labels,
create_time
FROM `asset_inventory.cloudresourcemanager_googleapis_com_Project`
WHERE JSON_VALUE(labels, '$.owner') IS NULL
OR JSON_VALUE(labels, '$.environment') IS NULL
ORDER BY create_time ASC
The label standard that supports this query requires five keys across all projects. Table 3 defines the minimum label set.
| Label Key | Values | Purpose |
|---|---|---|
environment |
prod, staging, dev, sandbox |
Tier-based policy inheritance |
owner |
Team or individual email | Accountability; orphan detection |
cost-center |
Business unit code | Billing allocation |
managed-by |
terraform, manual, pulumi |
IaC coverage tracking |
data-classification |
public, internal, confidential, restricted |
Data handling requirements |
Note: GCP does not enforce required labels natively without a custom Organization Policy constraint or external policy-as-code tooling. The label standard requires enforcement through a custom CEL constraint (see Part 2).
Steampipe: Real-Time Ad-Hoc Queries
Steampipe queries live GCP APIs using standard SQL via PostgreSQL Foreign Data Wrappers - no ETL pipeline, no BigQuery export, no waiting for a snapshot. [8] Targeted queries return in seconds; org-wide sweeps still hit live API rate limits, so Steampipe is complementary to BigQuery-backed exports rather than a replacement. Install the GCP plugin with steampipe plugin install gcp and query directly:
SELECT name, labels, create_time
FROM gcp_project
WHERE labels ->> 'environment' IS NULL
ORDER BY create_time;
Steampipe works well for ad-hoc investigations and real-time compliance checks. It does not store historical data.
CloudQuery: Persistent Historical Inventory
CloudQuery syncs cloud APIs to a destination database (PostgreSQL, BigQuery) on a schedule, enabling historical trend analysis and multi-cloud inventory. [9] Use CloudQuery when you need to answer questions over time: "How many projects existed 90 days ago? How many service account keys have been added since the last audit?"
Cartography: Graph-Based Relationship Mapping
Cartography, accepted into CNCF Sandbox in August 2024 (announced December 2024), loads cloud asset data into Neo4j and enables graph queries across resource relationships. [5] The key questions Cartography answers are relational: "Which identities have access to which datastores?" and "What are the lateral movement paths from this compromised service account?"
GitHub: https://github.com/cartography-cncf/cartography
The Unattended Project Recommender
The Unattended Project Recommender applies ML classification across API activity, networking activity, billing activity, user activity, and service usage. For organizations with more than 50 projects, it classifies a project as unattended when it falls in the bottom 10% of usage activity org-wide. [10] It generates CLEANUP_PROJECT or RECLAIM_PROJECT recommendations.
# List unattended project recommendations for the org
gcloud recommender recommendations list \
--organization=ORG_ID \
--location=global \
--recommender=google.resourcemanager.projectUtilization.Recommender
The recommender does not shut down or delete anything - it only surfaces candidates. The operator decides. This is the right model for brownfield cleanup where the blast radius of an incorrect deletion is high.
The Full Inventory Pipeline
The tools above provide the building blocks. Running them at org scale against nearly 1,000 projects requires a pipeline: export the asset inventory, export IAM policies per project, and check activity logs per project to identify dormant projects. This section documents the scripts, their runtime characteristics, and the practical gotchas encountered running them in Cloud Shell.
Step 1: Asset Inventory Export
The baseline inventory export uses Cloud Asset Inventory's search API at organization scope:
gcloud asset search-all-resources \
--scope="organizations/<ORG_ID>" \
--format=json \
> gcp_asset_inventory.json
This single command exports every resource the API can enumerate across every project in the organization. No filtering by project, no asset type restriction - a full org-wide sweep. Runtime was under 10 minutes for ~60,000 resources. The resulting JSON file is the starting point for all subsequent analysis.
Step 2: Project List Export
Before running per-project scripts, you need the project list. The obvious command is gcloud projects list, but this requires resourcemanager.projects.list at the organization level - a permission that Cloud Shell's default credentials may not have, depending on your IAM setup. If the export comes back empty, the workaround is to extract project IDs from the asset inventory export itself:
# Extract project IDs from the asset inventory.
# Count goes to stderr so project_list.txt contains only IDs;
# downstream loops would otherwise read the count line as a project ID.
cat gcp_asset_inventory.json | python3 -c "
import json, sys
data = json.load(sys.stdin)
projects = set()
for r in data:
p = r.get('project','')
if p:
projects.add(p.split('/')[-1])
print(f'{len(projects)} projects found', file=sys.stderr)
for p in sorted(projects):
print(p)
" > project_list.txt
Step 3: IAM Policy Export
The IAM export loops through every project and captures the full IAM policy - every role binding, every member, every condition. The output format is JSONL (one JSON object per line), which is both streamable and loadable into BigQuery.
#!/bin/bash
# Export IAM policies for all projects as JSONL
# Runtime: ~15-30 minutes for ~1,000 projects
# Output: ~5-15 MB depending on policy complexity
OUTPUT="iam_policies.jsonl"
> "$OUTPUT"
while IFS= read -r PROJECT_ID; do
POLICY=$(gcloud projects get-iam-policy "$PROJECT_ID" \
--format=json 2>/dev/null)
if [ $? -eq 0 ] && [ -n "$POLICY" ]; then
echo "{\"project_id\": \"$PROJECT_ID\", \"policy\": $POLICY}" >> "$OUTPUT"
else
echo "{\"project_id\": \"$PROJECT_ID\", \"error\": \"access_denied_or_not_found\"}" >> "$OUTPUT"
fi
done < project_list.txt
echo "Done. $(wc -l < "$OUTPUT") projects exported."
The JSONL format matters. An earlier version used echo "=== $PROJECT_ID ===" separators between projects, producing a file that looked readable but was not valid JSON and could not be loaded into BigQuery or parsed with jq. One JSON object per line eliminates that problem.
Query the output with jq to find overbroad bindings:
# Find projects where roles/editor is granted to a user (not a group or SA)
cat iam_policies.jsonl | jq -c '
select(.policy.bindings[]? |
select(.role == "roles/editor") |
.members[]? | select(startswith("user:"))
) | .project_id
'
Step 4: Activity Check
The activity check identifies dormant projects by pulling the most recent log entry timestamp from each project. This is the slowest step - the Cloud Logging API has stricter rate limits than the Resource Manager API.
#!/bin/bash
# Check last activity timestamp for each project
# Runtime: 1-2+ hours for ~1,000 projects (Logging API rate limits)
# Output: CSV with project_id,last_activity_timestamp
OUTPUT="project_activity.csv"
echo "project_id,last_activity" > "$OUTPUT"
while IFS= read -r PROJECT_ID; do
TIMESTAMP=$(gcloud logging read \
"logName:projects/$PROJECT_ID" \
--project="$PROJECT_ID" \
--limit=1 \
--format="value(timestamp)" \
2>/dev/null)
if [ -z "$TIMESTAMP" ]; then
TIMESTAMP="NO_LOGS"
fi
echo "$PROJECT_ID,$TIMESTAMP" >> "$OUTPUT"
done < project_list.txt
# Subtract the CSV header line from the count.
echo "Done. $(($(wc -l < "$OUTPUT") - 1)) projects checked."
Run the activity check with nohup if using Cloud Shell - the session will disconnect if the browser tab closes, and a 1-2 hour script is long enough to lose:
nohup bash activity_check.sh > activity_check.log 2>&1 &
Cloud Shell Limitations
Three Cloud Shell constraints affected this pipeline:
- Permission gaps.
gcloud projects listreturned empty despite having org-level viewer roles. The workaround (extracting project IDs from the asset inventory export) worked, but added an unexpected step. - Session timeouts. Cloud Shell disconnects after 20 minutes of browser inactivity. For the activity check (1-2+ hours),
nohupis mandatory. - Disk persistence. Cloud Shell's $HOME directory persists, but
/tmpdoes not survive session resets. Write all output to$HOMEor a GCS bucket.
What the Inventory Actually Showed
Volume
The asset inventory export produced a 43 MB JSON file containing over 60,000 resources spanning 165 distinct asset types across nearly 1,000 projects. The resource distribution follows a power law: BigQuery tables and container registry images account for more than half the total, while the long tail includes hundreds of resource types with fewer than 50 instances each.
| Asset Type | Count | % of Total |
|---|---|---|
| BigQuery tables | ~21,500 | 35.6% |
| Container registry images | ~12,900 | 21.3% |
| Enabled services | ~3,400 | 5.6% |
| KMS crypto key versions | ~3,400 | 5.6% |
| Secret versions | ~3,300 | 5.4% |
| Log sinks | ~1,900 | 3.2% |
| Log buckets | ~1,900 | 3.2% |
| Secrets | ~1,900 | 3.1% |
| Projects | ~960 | 1.6% |
| BigQuery datasets | ~880 | 1.5% |
| BigQuery routines | ~730 | 1.2% |
| Dataproc sessions | ~590 | 1.0% |
| Artifact Registry images | ~580 | 1.0% |
| Compute routes | ~480 | 0.8% |
| Subnetworks | ~450 | 0.7% |
The remaining 150 asset types combined account for less than 10% of total resources.
The Analysis Scripts
Two scripts extract the patterns that matter for governance. The first produces a total count and a ranked breakdown by asset type:
# Total resource count
cat gcp_asset_inventory.json | python3 -c "
import json, sys
data = json.load(sys.stdin)
print(f'Total resources: {len(data)}')
"
# Breakdown by asset type
cat gcp_asset_inventory.json | python3 -c "
import json, sys
from collections import Counter
data = json.load(sys.stdin)
types = Counter(r.get('assetType','unknown') for r in data)
for t, count in types.most_common(40):
print(f'{count:>6} {t}')
"
The second script targets the security-relevant questions - service account key distribution, project ownership gaps, and resource state:
import json
from collections import Counter
with open('gcp_asset_inventory.json') as f:
data = json.load(f)
# Service account key concentration
sa_keys = [r for r in data if r.get('assetType') == 'iam.googleapis.com/ServiceAccountKey']
sa_key_projects = Counter(r.get('project','').split('/')[-1] for r in sa_keys)
print(f"SA keys: {len(sa_keys)} across {len(sa_key_projects)} projects")
print(f"Top project: {sa_key_projects.most_common(1)[0][1]} keys")
# Destroyed but still-enumerated resources
destroyed = [r for r in data if r.get('state') == 'DESTROYED']
print(f"Destroyed resources still in inventory: {len(destroyed)}")
# Project state summary
projects = [r for r in data if r.get('assetType') == 'cloudresourcemanager.googleapis.com/Project']
states = Counter(p.get('state','unknown') for p in projects)
for s, c in states.most_common():
print(f" {c:>4} {s}")
What the Numbers Revealed
Five patterns stood out.
Roughly 70% of projects are sunset candidates. Classifying every project by business purpose revealed that the majority were auto-generated stubs: projects named "Untitled project" created when someone opened an Apps Script editor, default Gemini projects provisioned automatically by Google Workspace, and "My First Project" placeholders from initial console logins. These projects have near-zero compute activity, no identified owner, and no business function. They exist because Google's platform creates projects as a side effect of unrelated user actions, and nothing in the default configuration cleans them up. "Sunset candidate" is not the same as "safe to delete" - Apps Script projects can back live Google Sheets and Docs macros that the original user has long since forgotten. Each one needs a per-project check that confirms the underlying script is not bound to an active document before deletion. The remaining projects split roughly evenly between those that needed owner review (~15%) and those with a confirmed business purpose (~15%).
About 82% of all assets lived in a single project. One project - the organization's primary application platform running Cloud Run, GKE, Cloud SQL, and BigQuery - contained ~49,000 of the ~60,000 total assets. This concentration is not surprising for an org that consolidated application workloads onto a shared platform, but it means that asset-count-based sprawl metrics are misleading. The "nearly 1,000 projects" headline obscures the fact that the bulk of actual compute and data infrastructure lives in a handful of projects. The sprawl problem is not resource sprawl - it is project sprawl. Hundreds of empty or near-empty projects that create governance surface area without delivering value.
Service account key concentration. Over 300 service account keys existed across roughly 50 projects, with nearly half concentrated in a single project. The mean density (~6 keys per project) obscures the real shape of the risk: the distribution is heavily skewed, and the median across the other ~49 projects is closer to 3. Concentration alone does not prove credential exposure, but it does mean that a single compromised project would yield disproportionately more usable keys than a uniform distribution would suggest.
The destroyed resource tail. More than 1,300 resources still appeared in the inventory with a DESTROYED state. These are resources the platform has marked as deleted but that Cloud Asset Inventory continues to enumerate for audit purposes. They are not a runtime risk, but they pollute inventory analysis and must be filtered in any governance query. Any script that counts "total resources" without filtering state will overcount.
Project sprawl at scale. Nearly 1,000 active projects across just 3 organizational folders, with the bulk concentrated in one of the three. Either way, that ratio - hundreds of projects per folder - indicates flat hierarchy, not tiered governance. Without subfolder structure by environment (production, staging, sandbox), org policies cannot cascade differently per tier. The folder structure is the prerequisite for tiered governance, and this org did not have it.
The Classification
The classification exercise changed the remediation plan. Instead of "govern 1,000 projects," the actual scope split into three tiers:
| Tier | Count | % | Description |
|---|---|---|---|
| Sunset | ~670 | ~70% | Auto-generated projects (Untitled, Apps Script, Gemini, My First Project). No compute, no owner, no purpose. Safe to bulk-delete after confirming no active integrations. |
| Review | ~140 | ~15% | Named projects with unclear ownership. Includes automation tool integrations, test sandboxes, team sandboxes, and uncategorized projects needing someone to claim them. |
| Keep | ~150 | ~15% | Identified business purpose. IT infrastructure, internal applications, finance tooling, investment operations, and data infrastructure. |
A finer-grained category breakdown reveals where the projects actually came from and who should own cleanup.
| Category | Projects | Sunset | Review | Keep |
|---|---|---|---|---|
| Apps Script Auto-Generated | ~400 | ~395 | ~5 | 0 |
| Untitled Project | ~245 | ~245 | 0 | 0 |
| Uncategorized | ~50 | 0 | ~50 | 0 |
| Comms / Email Automation | ~40 | 0 | ~35 | ~5 |
| Finance / Ops | ~35 | 0 | 0 | ~35 |
| Internal App (Business) | ~35 | 0 | 0 | ~35 |
| AI / ML | ~25 | 0 | ~25 | 0 |
| Internal App (IT-managed) | ~25 | 0 | 0 | ~25 |
| Investment / Portfolio | ~20 | 0 | 0 | ~20 |
| Infrastructure / IT | ~20 | 0 | 0 | ~20 |
| Default Gemini Project | ~15 | ~15 | 0 | 0 |
| Automation Platform | ~15 | 0 | ~15 | 0 |
| Default / Placeholder | ~15 | ~15 | 0 | 0 |
| Test / Sandbox | ~7 | 0 | ~7 | 0 |
| Team Sandbox | ~6 | 0 | ~6 | 0 |
| Data / Analytics | ~6 | 0 | 0 | ~6 |
| SSO / Auth Integration | ~2 | 0 | 0 | ~2 |
Two patterns stand out in Table 6. First, the great majority of projects (the ~810 in the Sunset and Review tiers) hold no compute resources - they are metadata stubs the platform created and no one cleaned up. Second, more than 80% of projects have no identifiable owner. The Keep tier's ~150 projects are the only ones where team assignment is clear, split across finance, investment operations, business operations, and IT. Note that Tables 5 and 6 round independently and may differ by a handful of projects against the ~960 total.
The first useful action from the classification was separating projects with a sys-* prefix (auto-generated by Google) from named projects. The sys-* projects are almost universally safe to sunset - they are created by the platform, not by users making intentional infrastructure decisions. Splitting them out reduced the "review" pile by half immediately.
The IAM export then answered the follow-up question: do any of these sunset candidates have IAM bindings that suggest active use? A project with no compute resources but active IAM bindings to a service account may be serving as a credential store for an integration that runs elsewhere.
The activity check provided the final signal: when was the last logged event in each project? A project with no resources, no meaningful IAM bindings, and no log activity in 90+ days is a confident sunset candidate.
These three signals together - asset inventory, IAM policy, and activity timestamp - form the minimum data set for a defensible cleanup decision. Any single signal alone is insufficient.
Drilling Into the 82% Project
The org-wide inventory showed that one project held about 82% of all assets: ~49,000 of the ~60,000 total. The classification exercise flagged it as Keep (it runs the organization's primary data platform), but the asset count demanded a deeper look. What was actually in there?
The breakdown by resource type tells the story.
| Resource Type | Count | Notes |
|---|---|---|
| BigQuery tables | ~21,300 | 92% non-production (developer scratch, stale PR staging, sandboxes) |
| Container registry images | ~12,900 | Legacy CR, AR migration incomplete |
| Other resources (long tail) | ~5,000 | ~150 asset types each below 50 instances |
| KMS key versions | ~3,400 | Auto-rotation accumulation |
| Secret versions | ~3,200 | Average 1.7 versions per secret |
| Secrets | ~1,900 | 69% auto-generated by a managed data-integration platform |
| BigQuery datasets | ~830 | The organizing layer for all those tables |
| Artifact Registry images | ~420 | Migration target (partially populated) |
| Storage buckets | ~50 | Well-named, clear environment separation |
| Pub/Sub topics | ~40 | Data pipeline topics (prod + sandbox duplicated) |
| GKE nodes | 8 | Single cluster |
| Cloud Functions | 6 | Data ingest functions |
| Cloud SQL instances | 5 | Prod, dev, UAT + tooling |
| Redis instances | 5 | Caching layer |
| AI Platform notebooks | 5 | Personal developer notebooks |
Four findings from the deep-dive changed the remediation plan.
BigQuery dataset sprawl is the dominant issue. ~830 datasets containing ~21,000 tables, but only ~70 datasets (~8%) are production. The rest breaks down as:
- ~470 developer personal datasets (prefix pattern
xx_*) across ~50 developers, holding ~14,800 tables. Each developer maintains their own copies of production schemas for local iteration. No automated cleanup, no TTL, no enforcement. - ~195 stale PR staging datasets from ~33 already-merged branches, holding ~3,800 tables. The CI/CD pipeline creates per-PR datasets for integration testing but has no post-merge cleanup step. Every merged PR leaves a dataset behind permanently.
- ~80 sandbox environment datasets. Some actively used, others stale.
The developer dataset pattern deserves attention. Cross-referencing the ~50 developer names against the HRIS would identify which developers have left the organization. Their personal datasets are immediately deletable. For active developers, a two-week claim-or-delete window recovers the rest. The PR staging datasets are the easiest win: they reference merged branches that no longer exist. Deleting them is low-risk and recovers ~3,800 tables.
Secret sprawl is largely machine-generated. ~1,900 secrets with ~3,200 versions, but 69% (~1,300) are auto-generated by a data integration platform (one secret per connector configuration, across four workspace IDs). Another ~250 are orchestration tool variables. Only ~340 are application or service secrets. Version sprawl is minimal (average 1.7 versions per secret), so the problem is secret count, not version accumulation. The remediation path is auditing which integration workspace IDs are still active - decommissioned workspaces mean their secrets are immediately deletable.
KMS key version accumulation. ~3,400 key versions across 21 keys in 8 key rings. The GKE key rings account for ~1,800 enabled versions (expected: automatic rotation creates new versions). The storage encryption keys hold ~1,500 versions with ~1,300 in destroyed or disabled state. The destroyed versions are already inert but pollute the inventory. The disabled versions need confirmation before destruction. Several key rings reference decommissioned local cluster environments and are candidates for full key ring deletion.
Container registry migration is incomplete. ~12,900 images in legacy Container Registry versus ~420 in Artifact Registry. The migration was started (10 AR repositories exist) but never finished. Every image in legacy CR sits outside Artifact Registry's lifecycle and cleanup-policy controls and predates the org's current scanning configuration.
| Priority | Action | Impact | Risk |
|---|---|---|---|
| P0 | Delete ~195 stale PR staging datasets (~3,800 tables) | Immediate table cleanup | Low: stale CI artifacts from merged branches |
| P0 | Add CI/CD post-merge hook to auto-delete PR staging datasets | Prevent future sprawl | Requires pipeline change |
| P1 | Cross-reference ~50 developer names against HRIS; delete departed devs' datasets | ~5,000-11,000 table cleanup | Needs HRIS data |
| P1 | Notify active developers: 2-week claim-or-delete on personal datasets | Recover remaining dev tables | Developers may have dependencies |
| P1 | Audit ~1,300 integration platform secrets by workspace ID | ~500-1,000 secret cleanup | Need platform admin to confirm active IDs |
| P2 | Complete Container Registry to Artifact Registry migration; purge legacy CR | ~12,500 image cleanup | Must verify all deployments reference AR |
| P2 | Destroy non-enabled KMS key versions for storage keys (~1,300 versions) | KMS hygiene | Disabled versions need confirmation |
| P3 | Review AI Platform notebook runtimes for active usage | Cost savings | Personal dev notebooks may be billing |
| P3 | Evaluate sandbox data pipeline topics and functions for decommission | Reduce duplicate infrastructure | Confirm sandbox pipeline still used |
Part 2 covers what to do with this inventory: the identity chain that governance depends on, the project factory, org policies in brownfield (dry-run before enforcement), Terraform import at scale, and measuring governance by adoption instead of audit.
References
[1] Flexera. "2026 State of the Cloud Report." 15th annual. https://info.flexera.com/CM-REPORT-State-of-the-Cloud
[2] Google Cloud Blog. "Google Cloud launches Unattended Project Recommender." August 6, 2021. https://cloud.google.com/blog/products/identity-security/google-cloud-launches-unattended-project-recommender
[3] Hunters Security. "DeleFriend: Severe design flaw in Domain Wide Delegation could leave Google Workspace vulnerable for takeover." 2023. https://www.hunters.security/en/blog/delefriend-a-newly-discovered-design-flaw-in-domain-wide-delegation-could-leave-google-workspace-vulnerable-for-takeover
[4] Google Cloud. "Cloud Asset Inventory overview." Google Cloud Documentation. https://cloud.google.com/asset-inventory/docs/overview
[5] Alex Chantavy. "Cartography joins the CNCF." Lyft Engineering Blog. December 18, 2024. https://eng.lyft.com/cartography-joins-the-cncf-6f6b7be099a7 CNCF project page: https://www.cncf.io/projects/cartography/ ("Accepted to CNCF on August 23, 2024 at the Sandbox maturity level.")
[6] Harness. "FinOps in Focus 2025." 2025. https://www.harness.io/press-and-news/finops-in-focus-report-2025
[8] Turbot. "Steampipe documentation: GCP plugin." https://hub.steampipe.io/plugins/turbot/gcp
[9] CloudQuery. "GCP source plugin documentation." https://hub.cloudquery.io/plugins/source/cloudquery/gcp
[10] Google Cloud. "Unattended Project Recommender." Google Cloud Documentation. https://cloud.google.com/recommender/docs/unattended-project-recommender