Wiring Android's WorkManager to a Quantized On-Device LLM for Background Summarization

--- title: "Wiring WorkManager to On-Device LLMs for Background Summarization" published: true description: "Schedule quantized LLM inference in Android WorkManager, handle Doze-mode constraints, promote foreground services, and choose the right model tier for mid-range devices." tags: android, kotlin, architecture, mobile canonical_url: https://blog.mvpfactory.co/wiring-workmanager-on-device-llm-background-summarization --- What We Are Building Let me show you a pattern I use when on-device AI needs to run reliably in the background. We are wiring Android's WorkManager to a quantized LLM — specifically llama.cpp via JNI — to perform chunked document summarization without OOM kills, Doze-mode deferrals, or angry users staring at a frozen UI. By the end of this tutorial you will have a chained Worker architecture that selects the right model tier, respects memory ceilings, and promotes to a foreground service only when the model demands it. Prerequisites Android project targeting API 26+ WorkManager 2.9+ on the classpath A GGUF model file bundled or downloaded to internal storage (llama.cpp, MLC LLM, or MediaPipe LLM Inference API all fit this pattern) Basic familiarity with CoroutineWorker Step 1 — Pick Your Model Tier Before Writing Any Scheduling Code Here is the gotcha that will save you hours: the memory ceiling on a mid-range device determines your entire architecture. A Snapdragon 6 Gen 1 with 6 GB RAM leaves your app process roughly 1.8–2.2 GB before the OOM killer becomes aggressive. Model Size INT8 RAM INT4 RAM Safe on 6 GB device? 1B ~1.0 GB ~0.6 GB Both tiers 1.5B (Phi-2 class) ~1.5 GB ~0.9 GB Both with headroom 3B ~3.0 GB ~1.7 GB INT4 only 7B ~7.0 GB ~4.0 GB Neither — move server-side For background Workers without foreground promotion, target sub-1B INT4 or sub-1.5B INT4. If you find yourself rationalizing a 7B model on a 6 GB device, that is a signal to move inference server-side, not to keep tuning constraints. Step 2 — Set Constraints That Actually Matter The docs do not mention this, but setRequiresBatteryNotLow is non-negotiable for inference workloads — LLM inference drains battery fast enough to trigger system-level throttling mid-run. val inferenceConstraints = Constraints.Builder() .setRequiresBatteryNotLow(true) .setRequiredNetworkType(NetworkType.NOT_REQUIRED) .build() val summarizeRequest = OneTimeWorkRequestBuilder() .setConstraints(inferenceConstraints) .setExpedited(OutOfQuotaPolicy.RUN_AS_NON_EXPEDITED_WORK_REQUEST) .setInputData(workDataOf("chunk_index" to 0, "total_chunks" to 3)) .build() setExpedited is critical for user-triggered summarization. Without it, Doze-mode deferral can push your work by hours. Expedited tasks require a getForegroundInfo() override — WorkManager calls it on older API levels to attach a notification. Step 3 — Chain Workers for Chunked Documents Here is the minimal setup to get chunked summarization working. Most teams try to load the entire document in one Worker and either blow the memory budget or hit the 10-minute execution window. The correct pattern is parallel chunk Workers feeding a serial reduce Worker. val chunkWorkers = (0 until totalChunks).map { index -> OneTimeWorkRequestBuilder() .setInputData(workDataOf("chunk" to index)) .build() } val reduceRequest = OneTimeWorkRequestBuilder().build() WorkManager.getInstance(context) .beginWith(chunkWorkers) // parallel fan-out .then(reduceRequest) // serial reduce .enqueue() Each ChunkSummarizeWorker loads the model, runs inference on a ~500-token window, unloads, and writes its partial summary to the output Data map. Model load/unload per chunk costs ~200–400 ms for INT4 1B models on a Snapdragon 6 Gen 1 — expensive, but it keeps peak RSS below the OOM threshold. Step 4 — Promote to Foreground for 3B Models If your product requires a 3B model, you must promote the Worker to a foreground service. On CoroutineWorker: override suspend fun getForegroundInfo(): ForegroundInfo { val notification = buildSummarizationNotification() return ForegroundInfo( NOTIFICATION_ID, notification, ServiceInfo.FOREGROUND_SERVICE_TYPE_SHORT_SERVICE ) } FOREGROUND_SERVICE_TYPE_SHORT_SERVICE (API 34+) gives you up to 3 minutes of guaranteed execution without a declared use-case permission — the practical sweet spot for 3B INT4 inference on a chunked document. Gotchas Silent OOM kills. The OOM killer does not throw an exception — your Worker just disappears. Profile peak RSS on your minimum-spec device under load before committing to a model size. Forgetting getForegroundInfo() with expedited tasks. WorkManager will crash on older API levels if you mark a task expedited without implementing this override. Loading the whole document in one Worker. Any document over ~1,500 tokens should be chunked. One Worker, one window, bounded memory. Treating 7B as an on-device target. At ~4 GB INT4 footprint, 7B exceeds available RAM on most 6 GB devices even with foreground promotion. Move it server-side. Conclusion Three decisions determine whether this architecture ships or collapses in production. Profile peak RSS and select your model tier first — everything else follows from that number. Use chained Workers for documents over ~1,500 tokens to stay within memory and execution time bounds. Always set setExpedited for user-initiated work to avoid Doze-mode deferrals measured in hours. Get those three right and background LLM inference becomes a reliable product feature rather than a source of silent failures. Further reading: WorkManager guides · llama.android · MediaPipe LLM Inference API