Graduate Researcher, SSAIL at UIUC. Open to 2027 software engineering roles.

Making distributed systems agree with each other.

Software engineer working on distributed AI systems and ML infrastructure: multi-GPU training and inference at UIUC's Supercomputing Systems & AI Lab. Previously built production backend systems at Oracle handling 10M+ transactions a day.

View selected work Résumé (PDF) GitHubLinkedIn
Shreyash Bhatkar at the Alma Mater statue, UIUC
10M+transactions/day, Oracle billing services
+37% / −12%gRPC throughput / P95 latency after tuning
2.09×fewer decode steps, adaptive inference runtime
4× BlackwellRTX PRO 6000 GPUs, FSDP2 + vLLM stack

Selected work

Three pieces, each with the part that was actually hard. Diagrams are drawn from the real systems, not stock art.

Distributed RL infrastructure across an HPC cluster and the cloud

SSAIL, UIUC · Jan 2026 – present · advisor Prof. Minjia Zhang
Problem
Run and benchmark agentic RL (TRL AsyncGRPO, TorchForge) on official SWE-Bench Verified tasks, with training on NCSA DeltaAI and the Dockerized evaluation environments on AWS EC2. Two clusters, two schedulers, one experiment.
Hard part
An early result showed one framework clearly outrunning the other. It was an artifact: the two runs had been given asymmetric GPU placement. Left uncaught, that number would have steered the project.
What I did
Profiled both frameworks on step latency, memory, utilization, and placement traces; normalized placement and re-ran matched experiments. Deployed Qwen3.5-4B with FSDP2 (2-GPU BF16 training) and vLLM on 4× RTX PRO 6000 Blackwell, then traced an NCCL policy-weight sync hang at the first update to the source, ruling out inference DP and the 1 GB packed-transfer limit with controlled A/B topology tests.
Result
A corrected comparison, a reproducible pinned stack (PyTorch/CUDA/vLLM/TRL with state moved off ephemeral NVMe), and a working DeltaAI→EC2 execution path for SWE-Bench workloads.
Benchmark scripts on GitHub lab infrastructure is otherwise private
NCSA DeltaAI · Slurm trainer FSDP2 · BF16 · 2 GPU TRL AsyncGRPO vLLM w0 vLLM w1 vLLM w2 · stall vLLM w3 NCCL policy weights 4× RTX PRO 6000 Blackwell AWS EC2 · persistent EBS Docker sandboxes SWE-Bench Verified patch · test · resolve reward + trajectory rollouts rewards back to trainer profiled: step latency · GPU mem/util · rollout time · sync time · placement
The dashed worker is where the first-update sync hang reproduced. The placement artifact came from comparing runs that did not share this topology.

Adaptive LLM inference runtime

independent project · Python, PyTorch, vLLM, CUDA · Llama models, ShareGPT prompts
Problem
Autoregressive decoding pays a full forward pass per token even when the model is nearly certain about the next several. Speculative approaches help, but unsafe multi-token transitions degrade output quality.
Hard part
Deciding when a multi-token step is safe. Single confidence signals were unreliable; the runtime combines entropy, margin, energy, and agreement signals, and profiles verification and KV-cache behavior to characterize the transitions that go wrong.
What I did
Built a decoding runtime that routes between single- and multi-token generation per step, plus the profiling to find where it broke. Batched inference and LLM-as-judge evaluation across 5,000 generations.
Result
2.09× fewer decoding steps and 19% lower long-form latency against the single-token baseline. Evaluation time went from 90 s to 3.3 s per sample (27×) by batching, which is what made the 5,000-sample sweep feasible at all.
Related: KV-cache tiering for multimodal RAG serving (GitHub) runtime code not yet public
hidden statestep t confidence gate entropy · margin energy · agreement 1 token k tokens verify reject → 1 token relative to single-token baseline (lower is better) decode steps 0.48× (2.09× fewer) long-form latency 0.81× (−19%) eval time / sample 3.3 s vs 90 s (27×, batched) 5,000 ShareGPT generations · LLM-as-judge quality held against baseline unsafe transitions found by profiling verification + KV-cache behavior
Bars are the measured ratios from the project; the flow is the actual routing decision made once per step.

Production billing and subscription services at telecom scale

Oracle · Batelco Telecom, PPC Retail India · Jul 2023 – Jan 2026
Problem
Subscription, billing, and account workflows for a telecom operator: 10M+ transactions a day across 8 Java/Kotlin Spring Boot services, 20+ REST APIs, and Kafka fan-out to 12+ downstream integrations. The failure that matters is charging someone twice.
Hard part
Throughput collapsed under load and nobody could say why. Prometheus traces showed gRPC HTTP/2 stream exhaustion and connection-pool saturation on the path to the 5G core, not the application code everyone was staring at.
What I did
Tuned Istio/OCNE connection pools and service configuration; designed saga recovery plus Redis idempotency keys so retries are safe by construction; built retry-and-dedupe Kafka pipelines; reworked SQL/JPA access paths over 300M+ monthly records. Also led a 4-person team shipping a production RAG assistant (LangChain, Cohere, Neo4j) for 500+ engineers.
Result
+37% throughput and −12% P95 latency on the gRPC path; 99.95% of transactions completing correctly under concurrent load; +35% Kafka delivery throughput with zero duplicate-charge incidents after rollout; −20% peak-cycle database latency. The assistant cut hallucinations 41% and ticket-resolution time 33%.
client systems; no public code. Recognition: Above and Beyond Award, Pat on the Back Award.
clients 8 Spring Boot services 20+ REST APIs · Java/Kotlin Redis idempotency key per charge saga recovery compensate partial writes SQL / JPA 300M+ rows/month −20% peak latency gRPC · HTTP/2 5G core · Wi-Fi control stream exhaustion found here pools tuned: +37% / −12% P95 Kafkaretry · dedupe 12+downstreamsystems +35% delivery, 0 duplicate charges observed with Prometheus + OpenTelemetry; 10M+ transactions/day sustained
The two highlighted boxes are why retries are safe. The dashed edge is where the throughput problem actually lived.

Open source

Upstream fixes in vLLM, verl, NeMo RL, SGLang, JAX, Tunix, Grain, and python-chess. Each one links to the pull request.

vLLM
  • During speculative decoding, Model Runner V2 ran an independent draft model past its own context length, indexed past its rotary cache and KV cache, and later faulted with a CUDA illegal memory access. Ported V1's capacity gate so the draft is skipped when the batch doesn't fit, with a data-parallel all-reduce so ranks agree. vllm#59517
  • Under speculative decoding, Model Runner V2 built the lm_head LoRA mapping with one id per request instead of one per logits row (the bonus row plus the draft rows), so an adapter could land on a neighboring request in a mixed batch. The row mapping now matches the logits layout. vllm#59341
  • /v1/chat/completions silently dropped turns whose role the chat template doesn't render, such as a typo like 'usr', and still returned 200. It now returns a 400 naming the role, while custom roles the template does render keep working. vllm#59108
verl
  • With FSDP merged LoRA and free_cache_engine, hybrid sleep at level 2 discarded vLLM weights that the PEFT-export sync never refilled, so rollouts turned to garbage with no error. FSDP merged LoRA now sleeps at level 1; Megatron is unchanged. verl#8046
NeMo RL
  • The vLLM workers dropped the result of reset_prefix_cache, so every worker returned None and invalidate_kv_cache() ran all() on an empty list and reported success even when the reset failed. The workers now pass the result through, and no reported result counts as a failure. nemo-rl#4647
SGLang
  • Falcon-H1 with tied embeddings called .float() on the shared embedding in place, making it fp32 while the model was bf16, so fused RMSNorm crashed during server warmup. Tied heads now use tie_weights and keep fp32 logits via a matmul upcast. sglang#41652
JAX
  • jax.scipy.special.rel_entr computed p*log(p) - p*log(q), which cancels when p is near q and returns nan for large equal inputs; kl_div inherited it and could go negative. It now uses the log1p formulation SciPy uses, with float32 and float64 checks in JAX's numerics suite. jax#41260
Google Tunix
  • The low_var_kl (Schulman K3) estimator exp(d)-d-1 could round negative in float32 near d=0. Switched to expm1(d)-d, which avoids the cancellation. tunix#2638
Google Grain
  • The stats registry kept a weak reference per iterator, keyed by id(), and never removed dead ones. Once CPython reused an address, a new iterator matched the stale entry and was never registered. Dead refs now remove themselves and no longer compare equal to a live iterator. grain#1429
python-chess
  • After an illegal move inside a variation, read_game() ended the variation but never popped its board, so the rest of the mainline was checked against the wrong position and legal moves were dropped; a stray ")" after a mainline error crashed the builder. read_game() now tracks why it is skipping and closes variations correctly. python-chess#1218

Experience

Jan 2026 —
present

Graduate Researcher, Systems & Runtime Engineering

SSAIL, Supercomputing Systems & AI Lab, UIUC · advisor Prof. Minjia Zhang
  • Built an NCSA DeltaAI-to-AWS execution path for Dockerized SWE-Bench workloads, validating reproducible orchestration across HPC and cloud.
  • Profiled TRL/TorchForge across latency, memory, utilization, and placement; found asymmetric GPU allocation behind an apparent speedup and redesigned matched experiments.
  • Debugged FSDP2/vLLM/NCCL synchronization failures on 4× RTX PRO 6000 GPUs using source tracing, controlled A/B tests, and process-level telemetry.
Jun 2026 —
present

Graduate Research Assistant, Human-Robot Systems

University of Illinois Urbana-Champaign
  • Operate a 6-DOF manipulator and built RGB-D/3D data pipelines with <50 ms event latency; calibrated camera-to-robot mappings with SE(3) transforms.
  • Built Grasshopper components that turn parametric 3D geometry into machine-readable control data for robot sequencing and fabrication workflows.
Jul 2023 —
Jan 2026

Software Engineer

Oracle · client engagements: Batelco Telecom, PPC Retail India
  • Built 8 Java/Kotlin Spring Boot microservices and 20+ REST APIs handling 10M+ daily transactions; saga recovery and Redis idempotency held 99.95% correctness under concurrent load.
  • Traced gRPC HTTP/2 stream exhaustion with Prometheus and tuned Istio/OCNE connection pools: +37% throughput, −12% P95 latency. Kafka pipelines with idempotent retry across 12+ integrations: +35% throughput, zero duplicate charges.
  • Led a 4-person team building a production RAG assistant for 500+ engineers and 2K+ daily queries, cutting hallucinations 41% and ticket-resolution time 33%.
recognition: Above and Beyond Award · Pat on the Back Award
Dec 2022 —
May 2023

AI Research Intern, Financial Markets

Nibha Tech Solutions, Nashik, India
  • C++/Python feature pipelines over 25 years of market data, producing 30,000+ research features; integrated 15 GB+ datasets into backend/ML pipelines and evaluated SARIMA/SHAP models at 96.2% normal-market classification accuracy.

Other work

Smaller, but each one taught me something the big projects didn't.

Distributed storage · C++

ScDFS, a scalable distributed file system

Chunk-based storage with consistent hashing, Cassandra-backed metadata, 3× replication, and node recovery; multithreaded sharding cut query latency 40% while keeping replicas consistent under contention.

Code on GitHub
Real-time collaboration · TypeScript

SynCode, a synchronized code editor

Operational-transform sync over WebSockets at <100 ms, WebRTC audio/video, Redis-backed horizontal scaling; load-tested at 300+ concurrent users with Jest integration tests on AWS.

Code on GitHub
Inference serving · Python

Modality-aware KV-cache tiering

A from-scratch serving layer for vision-language RAG that reuses KV cache across GPU, CPU, and SSD tiers to raise queries per second and fit larger batches on the same GPU.

Code on GitHub
Embedded · on-device RL

Self-powered safety wearable (OPEHS)

Piezo-triboelectric harvesting at 10–50 mW per stride feeding an ARM Cortex-M0 + BLE board; on-device DQN/PPO classifies routine vs. high-risk movement at 93% F1 on 40k+ footfall samples. Indian provisional patent filed, March 2025.

Code on GitHub
Operating systems · C, Assembly

xv6 kernel extensions

Extensions to the MIT xv6 teaching kernel, done to understand the scheduler, syscalls, and memory paths from the inside instead of from a textbook.

Code on GitHub
Computer vision · published

Forest-fire detection with a custom RCNN

PyTorch RCNN over 10k+ augmented images at 97.3% mAP, with a Kafka + Flink pipeline for real-time anomaly flags. Published at ICEPTP '22, Lisbon.

Paper on ResearchGate

Skills

Languages
C++, Python, Java, Go, Rust, TypeScript/JavaScript, SQL, Bash
Distributed & ML systems
PyTorch, FSDP2, vLLM, CUDA, NCCL, TRL/GRPO, GPU profiling, LLM inference and KV-cache work, Slurm/HPC, Linux, concurrency
Backend
Spring Boot, gRPC, REST, microservices, Kafka, Redis, PostgreSQL, Cassandra, JPA, Node.js, React
Infrastructure
AWS, Docker, Kubernetes, Istio, CI/CD (Jenkins), Prometheus, OpenTelemetry, MLflow, Git
Embedded & robotics
Embedded C, ARM Cortex-M0, BLE, Arduino, RGB-D/3D pipelines, SE(3) calibration, 6-DOF manipulator operation
certifications: Oracle Certified Generative AI Professional · Oracle Certified Data Science Professional · runner-up, CGIU Memorial AI Hackathon

The work I gravitate toward is the part that resists autocomplete: a hang that only appears on the first policy update, a benchmark that lies because of where the GPUs were placed, a duplicate charge that only happens under a partition. Those get found by someone who reads the stack all the way down.

I spent two and a half years at Oracle shipping backend systems that telecom and retail clients depended on, then came to UIUC to work on the layer underneath machine learning. Earlier I built hardware, which is why I'm also working on RGB-D pipelines for a 6-DOF manipulator here: I'd like to take distributed-systems and ML-infrastructure work into robotics and embodied systems over time.

University of Illinois Urbana-Champaign
Master of Computer Science, GPA 4.0/4.0
Champaign, ILExpected 2027
University of Pune
B.Tech, Electronics and Computer Science, CGPA 3.62/4.0
Pune, IndiaAug 2019 – May 2023

Systems playground: a partition just happened. Pick two.

Under a network split, a system gives up either consistency or availability, and the network always splits eventually. The interesting part is knowing which to give up for a given workload. Seven rounds, drawn from systems I've built or debugged.

Round 1 of 7

If you disagreed with a call, I'd like to hear why. That argument is most of what distributed-systems work actually is.

Argue with me

These are simplified partition trade-offs for the sake of a game, not claims that every real system reduces to textbook CAP. Round 6 in particular is a version-consistency call dressed as CAP; the feedback says so.

Let's talk about a role, a project, or a system that's misbehaving.

bhatkar3@illinois.edu linkedin.com/in/shr3yash github.com/Shr3yash

Looking for software engineering roles starting 2027 in distributed systems, ML infrastructure, or robotics software. Champaign, IL.

Download résumé (PDF)
one page · updated September 2026 · expected graduation 2027