Enterprise Data Intelligence: Governed on AWS Without Moving Customer Data

Trusted Enterprise Data, Without Moving It

Our enterprise customer runs a multi-tenant data intelligence platform that helps organizations catalog, profile, quality-check, and govern their data. The platform's users keep data in many places: Snowflake, Databricks, Oracle, PostgreSQL, cloud object storage, SaaS applications, BI tools, and on-premises systems.

The Challenge:

No single, trusted view showed what data existed, what it meant, who owned it, how good it was, or who could see it. Data teams spent much of their time finding and validating data before any analysis began. Business users needed a specialist to write queries for them. Leaders could not tell whether a number was reliable.

Two constraints made the problem harder:

• Data could not move. Many organizations, especially in regulated industries, cannot copy business data into a new central platform because of residency, security, and cost rules.

• AI needed governed data. AI initiatives need data that is quality-checked and access-controlled before they can give trustworthy answers.

Without a solution, organizations faced lost analyst time, decisions made on stale or duplicate data, inconsistent protection of sensitive fields, and AI pilots that stalled on ungoverned data. Over time, regulated organizations under HIPAA, SOC 2, or ISO 27001 expectations would struggle to show where their data lived and who accessed it.

A Serverless, Data-in-Place Platform on AWS

Emumba partnered with the customer to engineer and run a serverless-first, multi-tenant SaaS platform on AWS. The core design decision: end-user data stays where it is.

A lightweight Gateway agent runs inside each end user's own network. It holds the database drivers, connection routing, and scan scheduler, and it connects outbound only, over AWS PrivateLink or an encrypted Site-to-Site VPN. End users open no inbound firewall ports, and nothing is exposed to the public internet. Only metadata, such as schemas, column names, and profiling statistics, reaches the platform's cloud. Sample data, query results, and business records stay on the Gateway.

The platform uses these AWS services:

• Amazon Bedrock for natural-language querying and AI reasoning, with model calls kept inside the AWS account boundary.

• AWS Lambda and Amazon ECS on AWS Fargate for compute, with no servers or clusters to patch.

• Amazon Aurora Serverless v2 (PostgreSQL) for catalog, governance, quality, and access-control data.

• Amazon DynamoDB for chat history, audit events, and user activity at high write volume.

• Amazon S3 and AWS Step Functions to stage and orchestrate metadata scans.

• AWS DMS, Amazon Kinesis, and Amazon EventBridge to capture every data change as an audit trail.

• Amazon Cognito and Amazon API Gateway for authentication, MFA, SSO, and request throttling.

The team chose this design over four alternatives: centralizing all data in a warehouse first, an on-premises appliance per end user, a self-hosted open-source catalog, and exposing end-user databases to the internet for scanning. Each alternative either moved sensitive data, added operational burden, or failed regulated organizations' security rules.

Built for Scale, Resilience, and Cost Control

The platform runs in a dedicated VPC across three Availability Zones. Each zone has its own NAT Gateway, so no single zone failure stops outbound traffic.

• Automatic scaling: API Gateway, Lambda, S3, and DynamoDB scale as managed services. Aurora Serverless v2 scales within a set capacity range. Fargate services scale on CPU and memory, with tasks spread across zones.

• Secure request flow: Each request carries a token from Amazon Cognito. A Lambda authorizer checks it against an Amazon ElastiCache cache and passes tenant identity downstream.

• Governed AI answers: User questions reach a Fargate service that pulls governed metadata from Aurora and DynamoDB, calls Amazon Bedrock, and streams results back over WebSocket.

• Disaster recovery: AWS Backup takes daily snapshots of Aurora, DynamoDB, RDS, and S3 with 30-day retention, copied to a second US region. On failover, AWS CDK redeploys the full infrastructure. This Backup-and-Restore strategy meets a 24-hour RTO and RPO without paying for idle standby resources.

• Observability: Composite Amazon CloudWatch alarms, defined as code, send exactly one alert per service per incident through Amazon SNS to OpsGenie. External synthetic checks every minute catch DNS, CDN, and certificate failures that CloudWatch cannot see.

• Cost governance: A monthly review cycle uses AWS Cost Explorer, AWS Trusted Advisor, and AWS Instance Scheduler to find idle resources, auto-suspend eligible services, and track savings per environment.

Key Takeaways from the Data Intelligence Platform:

• Prove the DR plan by running it. The DR plan existed on paper but had never been executed. A full drill measured real restore times per service and confirmed that every restore met the 24-hour target.

• Let incidents shape monitoring. Aurora Serverless v2 was the root cause of two earlier critical incidents. The team added Aurora-specific composite alarms for deadlocks, capacity ceiling, and connection saturation to catch these failures earlier.

• One incident, one alert. Composite alarms defined as code stop alert storms, so on-call engineers act on a single, clear signal.

• Monitor from the outside, too. Synthetic checks catch DNS, CDN, and certificate failures that internal metrics cannot see.

• Treat cost as a monthly practice. A fixed review cadence, with at least three documented optimization actions each month, turned cost control into a habit and cut spend by a third in one month.

The customer now gives organizations one governed view of their data without asking them to move it. Regulated organizations keep records inside their own networks, while the platform delivers cataloging, quality checks, access control, and AI-driven answers from AWS.

The serverless, multi-AZ design scales with real usage, recovers from a regional failure inside 24 hours, and costs less each month through a steady optimization cycle. The platform holds SOC 2 Type II and ISO 27001 certification and is HIPAA compliant.