Securing Background Processes and Data Pipelines on Google Cloud
Gagan Kataria
Every security review we've run starts the same way: firewall rules, load balancer configs, auth flows on the public API. Then it stops, right at the edge of the request-response cycle. Nobody asks what happens after the API call returns 200 and hands the job to a queue.
That gap is not an accident. It's structural. Background workers, scheduled jobs, and ETL pipelines don't have a browser tab or a Postman collection pointed at them, so they rarely get pentested the way a login form does. Attackers have noticed the gap faster than most security teams have. Third-party and pipeline-adjacent breaches doubled from 15% to 30% of all incidents in a single year — the largest year-over-year shift ever recorded in the Verizon Data Breach Investigations Report (Verizon, 2025).
This is Part 3 of our Cloud Security series. Part 1 covered defense-in-depth at the edge, transport, network, and identity layers. Part 2 applied the OWASP API Top 10 to APIs running on Cloud Run. This part covers the surface both of those posts deliberately left alone: the asynchronous one.
Key Takeaways
- Third-party and pipeline-related breaches doubled to 30% of all incidents in 2025, and now take 267 days on average to contain — 26 days longer than the overall breach average (Verizon/IBM, 2025).
- Machine identities now outnumber human identities 109:1 in the enterprise, and most of that growth sits in service accounts nobody reviews (Palo Alto Networks, 2026).
- Eventarc triggers created without an explicit service account default to the Compute Engine identity, which typically holds the broad Editor role.
- Pin every third-party GitHub Action to a commit SHA, not a tag — a compromised tag pointer is how 23,000+ repositories were hit in March 2025.
Why Background Processes Are the Blind Spot in Security Reviews
A supply-chain or pipeline-adjacent breach now costs $4.91 million on average and takes 267 days to identify and contain, the longest lifecycle of any breach category IBM tracks (IBM Cost of a Data Breach Report, 2025). Compare that to 241 days for the overall average, and the gap is not noise. It's what happens when nobody is watching a surface closely.
Think about what actually runs in the background of a typical enterprise app on Google Cloud: a Pub/Sub topic that fans out order events, a Cloud Run job that reconciles payments overnight, a Dataflow pipeline that moves data into BigQuery for reporting, a Cloud Scheduler cron that syncs inventory. None of these have a user sitting in front of them. All of them have a service account, a payload format, and — usually — far more IAM permission than they need.
Why does the gap persist? Part of it is tooling. Static analysis and API scanners crawl endpoints; they don't subscribe to a Pub/Sub topic and fuzz the message schema. Part of it is ownership. A background job usually belongs to a data or platform team, not the AppSec team that owns the API gateway. Nobody's watching it, exactly, but that's precisely why it deserves your attention now, not after an incident forces the conversation.
The Identity Problem: Service Accounts Running Your Background Workloads
Every Pub/Sub push subscription, Eventarc trigger, and Cloud Run job authenticates as something. On Google Cloud that something is a service account, and the default one is far more dangerous than most teams realize.
When you create an Eventarc trigger without specifying an identity, GCP falls back to the Compute Engine default service account. That account typically carries the broad, project-wide Editor role — enough to read Cloud Storage buckets, modify IAM policies, and spin up new resources, none of which your worker actually needs. The permissions your job doesn't use don't sit idle; they sit exposed, waiting for the one time your worker gets compromised.
When an automation trigger runs on the default identity, a single compromised job doesn't just leak data — it inherits every permission that identity holds, whether the job uses them or not. Least privilege for background workloads starts with never accepting the default.
This isn't a hypothetical edge case. Machine identities — service accounts, API keys, OAuth tokens, and now AI agents — outnumber human identities by 109 to 1 in the enterprise, up from 82 to 1 just a year earlier (Palo Alto Networks 2026 Identity Security Landscape report, via Axis Intelligence). Almost none of that growth goes through the same access review your employee accounts get. When we scope service accounts for client workloads at Enerasoft, we treat every Pub/Sub subscriber, Cloud Run job, and Cloud Function trigger as its own identity — never a shared one, and never the default.
The fix is a handful of gcloud commands, not a redesign:
# 1. Create a dedicated identity for this one worker
gcloud iam service-accounts create order-pipeline-worker \
--display-name="Order pipeline worker (Cloud Run)"
# 2. Grant it only what it needs — nothing else
gcloud projects add-iam-policy-binding PROJECT_ID \
--member="serviceAccount:order-pipeline-worker@PROJECT_ID.iam.gserviceaccount.com" \
--role="roles/pubsub.subscriber"
# 3. Point the Eventarc trigger at that identity explicitly
gcloud eventarc triggers create order-created-trigger \
--location=asia-south1 \
--destination-run-service=order-worker \
--destination-run-region=asia-south1 \
--event-filters="type=google.cloud.pubsub.topic.v1.messagePublished" \
--transport-topic=orders-created \
--service-account="order-pipeline-worker@PROJECT_ID.iam.gserviceaccount.com"
For anything touching multiple resources — a Dataflow job reading from Pub/Sub and writing to BigQuery, say — use the Service Account Credentials API to mint short-lived tokens instead of attaching one broad account to everything. If a piece of the pipeline is compromised, the blast radius stays small. We covered the mechanics of service account authentication in more depth in our precursor post on why service accounts matter for cloud authentication — worth a read before you touch IAM bindings on a production pipeline.
When the Payload Is the Attack: Poisoned Messages and Insecure Deserialization
A queue message looks harmless. It's JSON, or maybe a serialized object, sitting in a topic waiting for a worker to pick it up. That trust is exactly the problem OWASP's newest Top 10 update was built to address.
OWASP Top 10:2025, finalized in January 2026, added Software and Data Integrity Failures as a standing category and introduced Software Supply Chain Failures as a brand-new entry at #3 — the first structural change to the list since 2021. Both categories describe the same underlying failure: trusting data or code because of where it came from, not because you validated it.
Insecure deserialization is the sharpest version of this in a background-job context. One documented case involved a job queue worker deserializing JSON payloads through Ruby's Oj library without validating the object graph first — a flaw that let an attacker inject arbitrary objects and achieve remote code execution on the worker itself. The worker never touched a browser. It just trusted its queue.
Three defenses matter more than any others here, and none of them require a new vendor contract:
- Validate against a schema before you deserialize anything. Pub/Sub supports schema enforcement natively — reject a message that doesn't match the Avro or Protobuf definition before your worker code ever sees it.
- Configure a dead-letter topic, and actually monitor it. A poisoned message that fails five times should land somewhere your team watches, not retry forever or vanish silently.
- Treat every queue payload as attacker-controlled input, the same way you'd treat a form field. Internal doesn't mean trusted; it means the validation moved one hop downstream.
gcloud pubsub topics create orders-created \
--message-encoding=JSON \
--schema=orders-created-schema
gcloud pubsub subscriptions create orders-created-sub \
--topic=orders-created \
--dead-letter-topic=orders-created-dlq \
--max-delivery-attempts=5
If your API layer rejects malformed input by default, why does your queue consumer still assume every message is well-formed? It shouldn't. Build the same skepticism into both.
The Pipeline Behind the Pipeline: CI/CD as a Data-Pipeline Attack Surface
Data pipelines don't just move data — they get deployed, and the deployment path is now a bigger target than the pipeline itself. In March 2025, attackers compromised tj-actions/changed-files, a GitHub Action used by more than 23,000 repositories, by retroactively rewriting its version tags to point at a malicious commit. Workflows that pinned the action by tag pulled the payload automatically; the change dumped CI runner memory and leaked secrets straight into public workflow logs (tracked as CVE-2025-30066).
It's one concrete example of a broader shift: third-party involvement in breaches jumped from 15% to 30% between 2024 and 2025, the sharpest single-year change the Verizon DBIR has ever recorded. Separately, 40% of organizations have GitHub Actions workflows configured to auto-approve pull requests, which means malicious code can merge into a pipeline's deploy path with zero human review (Orca Security, 2026).
For any pipeline that deploys via Cloud Build, Cloud Deploy, or GitHub Actions into GCP, three habits close most of that gap:
- Pin third-party Actions to a commit SHA, never a tag or branch. A tag can be moved after the fact; a commit hash cannot.
- Use short-lived, workload-identity-federated credentials for CI-to-GCP auth instead of a long-lived service account key sitting in a GitHub secret.
- Sign build artifacts and enforce provenance with Binary Authorization before a Cloud Run job or GKE deployment can pull an unsigned image.
# Before: pinned to a mutable tag
- uses: tj-actions/changed-files@v45
# After: pinned to an immutable commit SHA
- uses: tj-actions/changed-files@a1b2c3d4e5f6789012345678901234567890abcd
None of this is exotic. It's the same "verify, don't trust" posture Part 2 applied to API requests, moved one layer earlier — to the pipeline that ships the code your workers run.
Data Exfiltration Paths Unique to Pipelines: VPC Service Controls and Audit Logging
Here's a detail that catches even experienced GCP teams off guard: Data Access audit logs are disabled by default for most services except BigQuery. If you assumed "audit logging is on" once you enabled Cloud Audit Logs, you assumed wrong for everything outside BigQuery — including Pub/Sub, Cloud Storage, and Dataflow (Cloudaware, referencing Google Cloud Threat Horizons H1 2026).
That matters because a data pipeline's biggest exfiltration risk usually isn't a broken firewall rule — it's a compromised or over-permissioned identity quietly reading from BigQuery or Cloud Storage and no audit trail catching it. In our own client engagements, the single most common finding during a GCP posture review isn't a public bucket. It's a Dataflow or Cloud Function service account that can read datasets three projects away from anything it processes.
Two controls close most of that gap:
- VPC Service Controls (VPC-SC) wrap BigQuery, Cloud Storage, Dataflow, and Pub/Sub in a security perimeter that blocks data from leaving even if an IAM policy is misconfigured. It's designed specifically to stop exfiltration from Google-managed services, not to replace your network firewall.
- Explicit Data Access audit logs, turned on per service rather than assumed. Enable them for every service in the pipeline's blast radius, not just the ones with sensitive data today — pipelines change what they touch faster than access reviews keep up.
Closing the Loop: What to Check This Week
You don't need a quarter-long initiative to make progress here. Pick one pipeline — the one moving your most sensitive data — and run four checks: does its Eventarc trigger use a dedicated service account, does its Pub/Sub subscription have a monitored dead-letter topic, is every third-party GitHub Action in its deploy path pinned to a commit SHA, and are Data Access audit logs actually turned on for every service it touches?
Most teams find at least one gap on the first pass. That's normal — this is precisely the surface Part 1 and Part 2 didn't cover, and precisely why it accumulates risk quietly instead of loudly. The next post in this series moves further downstream, into the piece background pipelines feed most often: the databases and storage layers holding what those pipelines produce.
If you're evaluating your own background-processing surface and want a second set of eyes on the IAM bindings, reach out to our team at Enerasoft — this is the exact kind of review we run for clients building on Cloud Run, Pub/Sub, and BigQuery.
Frequently Asked Questions
Why are background processes harder to secure than a public API?
APIs get scanned, fuzzed, and pentested because they have a clear entry point. Background workers consume from queues and triggers instead, so standard scanners miss them entirely. That blind spot is part of why third-party and pipeline-related breaches take 267 days to contain, 26 days longer than the overall breach average (IBM, 2025).
Do I need a separate service account for every Cloud Function or Cloud Run job?
Yes, for anything handling distinct data or permissions. Machine identities already outnumber humans 109:1 in the enterprise (Palo Alto Networks, 2026), and sharing one broad account across jobs is exactly how that number keeps climbing unchecked. A dedicated, narrowly-scoped account per workload keeps a single compromise from cascading.
What's actually wrong with the default Compute Engine service account?
It typically carries the project-wide Editor role, granting far more access than almost any single background job needs. If that job is compromised, the attacker inherits every one of those unused permissions instantly, turning a small breach into a project-wide one.
How do I stop a malformed or malicious Pub/Sub message from taking down a worker?
Enforce a schema on the topic so malformed messages get rejected before your code runs, and configure a dead-letter topic with a capped retry count. Insecure deserialization of unvalidated queue payloads has already produced real remote-code-execution cases in production job processors.
Does VPC Service Controls replace the need for strict IAM on data pipelines?
No — VPC-SC wraps services like BigQuery and Dataflow in a perimeter to block exfiltration even when IAM is misconfigured, but it doesn't substitute for least-privilege access. Note that Data Access audit logs are disabled by default for most GCP services except BigQuery, so confirm logging explicitly rather than assuming it's on.
Sources (retrieved September 12, 2026):
- Verizon, 2025 Data Breach Investigations Report — third-party breach share doubling from 15% to 30%
- IBM, Cost of a Data Breach Report 2025 — supply-chain/third-party breach cost and lifecycle figures
- Palo Alto Networks, 2026 Identity Security Landscape report (via Axis Intelligence analysis) — machine-to-human identity ratio
- OWASP, Top 10:2025 (finalized January 2026) — Software Supply Chain Failures and Software and Data Integrity Failures categories
- Unit 42 / GitHub Advisory Database —
tj-actions/changed-filescompromise, CVE-2025-30066 - Orca Security, Top Cloud Security Risks (2026) — GitHub Actions auto-approval misconfiguration rate
- Google Cloud IAM documentation — best practices for service accounts and workload identity
- Cloudaware, referencing Google Cloud Threat Horizons H1 2026 — Data Access audit log defaults