Case studies

Complex projects. Concrete results.

From database migrations to document AI, these case studies show the decisions I owned and what they delivered.

Retail · Data engineering

Automating a 7-billion-row partitioning job

Oracle · Aurora PostgreSQL · Python · SQL

A global retailer migrating from Oracle to Aurora PostgreSQL needed to partition eight tables across four environments. The largest held 7 billion rows and occupied more than 6 TB.

I took ownership of all eight tables, wrote a parameterized Python tool to generate the SQL, and executed partition loads and index creation in parallel. For the largest table, I broke the generated SQL into batches so I could parallelize inserts and then index creation. I watched database CPU usage to set a safe concurrency limit.

Result The automated process completed in about 12 hours per environment, down from an estimated two weeks of work per environment.

Edtech · Cloud engineering

Automating a 100+ machine migration

AWS · CloudEndure · EC2 · Python

An edtech provider was evaluating AWS for disaster recovery through a short migration pilot. Configuring each target machine by hand limited how much the team could validate.

I used the CloudEndure API to automate EC2 target configuration and kick off migrations in batches. I also helped design the migration workflow and presented the solution to the client.

Result The engagement migrated over 100 machines, with more than 50 launched in a single run. The client moved ahead with AWS and extended the engagement by four weeks.

Healthcare · Solutions architecture

Designing a controlled path to Aurora

AWS Batch · Fargate · AWS KMS · Aurora PostgreSQL

A healthcare platform needed to migrate from a document database to Aurora PostgreSQL. The migration had to protect the source system under load, while some tenants required encryption with their own keys.

I designed and built an event-driven migration workflow with AWS Batch and Fargate, including a safeguard that stopped extraction when source load became too high. In a separate proof of concept, I validated cross-account KMS encryption and tested what happened when key access was revoked.

Result The client selected the orchestration design. The encryption tests surfaced a delay between key revocation and loss of database access, giving the team a concrete tradeoff to account for.

Life sciences · Data quality

Verifying rebuilt tables in minutes

Python · SQL · Dataiku · Amazon Redshift

A life sciences team was replacing vendor-generated tables with internal pipelines. Complex column mappings spanned 89 source tables and 15 targets. Developers often had to wait for QA to check the rebuilt data.

I built a Python tool that generated transformation SQL from the mappings. I also automated checks for record counts, missing study IDs, and schema differences, with an emailed summary when checks flagged discrepancies.

Result Table verification went from days to minutes, giving developers feedback before the next round of changes.

Logistics · Database reliability

Bringing replica lag down to three seconds

Amazon RDS · PostgreSQL · CloudWatch

A logistics platform faced query timeouts and rising replica lag during a traffic surge. Several large tables needed more targeted vacuum settings, and expensive queries were adding pressure.

I extracted timed-out queries from replica logs, deduplicated them, and used execution plans to identify expensive scans and missed partitions. I tuned queries and table-level autovacuum settings and documented the changes so the team could track performance afterward.

Result Replica lag fell from about 500 seconds to about three. I also enabled performance monitoring to help the team spot expensive queries earlier.

Life sciences · Observability

Making 200,000 daily API calls observable

Python · SQL · Database indexing

A data integration made about 200,000 API calls per day across 60 threads. The team needed per-call execution times to see where time was being spent.

I designed and built a Python monitoring framework that captured execution time per call. I also indexed a frequently run pipeline-log query to reduce the overhead of retrieving those records.

Result The team gained per-call timing for observability reporting. The indexed query went from about 18 seconds to under a second.

Life sciences · Platform reliability

Removing operational bottlenecks

Apache Airflow · Apache NiFi · GCS · Amazon S3

Airflow runtime

Diagnosed a development notification workflow that routinely ran for over 24 hours and brought its runtime down to 13 minutes.

NiFi maintenance

Used the NiFi API to automate Python configuration updates for up to 400 processors per environment and shared the script for reuse.

Cross-cloud transfer

Worked with operations to secure an approved execution environment and moved 1.3 TB from Google Cloud Storage to Amazon S3 in under four hours.

Aviation · Applied AI

Scaling ingestion for document-grounded AI

Amazon S3 · SQS · Lambda · CloudFormation

An aviation team wanted users to ask questions across its technical manuals and get answers grounded in those documents. The prototype handled about 25 PDFs at a time, and the deployment templates needed work before the client rollout.

I put an SQS queue between S3 uploads and Lambda processing so a large upload could be processed asynchronously. I led design and code reviews, introduced a shared repository structure, and updated the CloudFormation templates to help the team ship changes consistently.

Result The new design supported thousands of PDFs per upload batch. The team also had a more repeatable way to deploy changes.