The first time I saw my family struggle to interpret the NHS and council letters, I decided to create an application that explains these letters in plain English. Government letters are unstructured data full of dates, instructions, codes, and jargon, but mostly people only need to know three things: what it’s about, when it’s happening, and what to do next. That became the starting point for LetterLens.

Purpose

I aimed to reduce the anxiety people feel when dealing with government paperwork. With LetterLens, the user simply scans the letter, and the system translates it into easy-to-understand English with clear next steps. It highlights the key actions the letter expects, so users know exactly what to do.

Disclaimer: LetterLens is an educational prototype, not an alternative for legal advice.

Tech Stack (Decision-making)

Under the Hood(Deep Dive)

Camerax: capture and preview

Camerax makes capturing and analyzing images. Here's the setup to bind a preview and analyzer to the lifecycle.

val cameraProviderFuture = ProcessCameraProvider.getInstance(ctx)
cameraProviderFuture.addListener({
   val provider = cameraProviderFuture.get()
   val preview = Preview.Builder()
       .setTargetAspectRatio(AspectRatio.RATIO_4_3) // keep 4:3
       .build()
       .also { it.setSurfaceProvider(view.surfaceProvider) }
   provider.unbindAll()
   provider.bindToLifecycle(
       lifecycle,
       CameraSelector.DEFAULT_BACK_CAMERA,
       preview,
       imageCapture
   )
}, ContextCompat.getMainExecutor(ctx))

Tip: Rotate based on EXIF and crop margins; sharper, upright images improve OCR markedly.

ML Kit OCR: Extract raw text on-device

ML Kit processes the image and extracts raw text and confidence scores.

val img = InputImage.fromFilePath(context, Uri.fromFile(photo))
val rec = TextRecognition.getClient(TextRecognizerOptions.DEFAULT_OPTIONS)
rec.process(img)
   .addOnSuccessListener { onText(it.text) }
   .addOnFailureListener { e -> onError("OCR failed: ${e.message}") }

Note: I keep work entirely on-device; no letter images leave the phone.

Ktor 'explains' endpoint: classify + extract

A small Ktor service classifies text and pulls deadlines/actions.

routing {
   post("/explain") {
       val req = call.receive<ExplainReq>()
       val type = classify(req.text, req.hint)
       val deadline = extractDeadline(req.text, type)
       val (summary, actions, citations) = explainForType(type, req.text)
       call.respond(ExplainRes(type, deadline, summary, actions, citations))
   }
}

Keyword heuristics (examples)

Data Parsing & Classification

Beyond the /explain endpoint, the core of LetterLens is its classifier. Government letters are messy—mixed fonts, spacing, codes, dates, so I added helpers for normalization, fuzzy matching, and deadline detection.

Normalization helpers:

private fun norm(s: String) = s
   .lowercase()
   .replace('’', '\'')
   .replace('–', '-')
   .replace(Regex("\\s+"), " ")
   .trim()

Fuzzy matching (so “N H-S” still matches “NHS”):

private fun fuzzyRegex(token: String): Regex {
   val letters = token.lowercase().filter { it.isLetterOrDigit() }
   val pattern = letters.joinToString("\\W*")
   return Regex(pattern, RegexOption.IGNORE_CASE)
}

Classify by domain:

private fun classify(textRaw: String, hint: String?): String {
   val n = norm("${hint ?: ""} $textRaw")
   if (hasAny(n, "nhs", "appointment", "vaccination")) return "NHS Appointment"
   if (hasAny(n, "electoral register", "unique security code")) return "Electoral Register"
   if (hasAny(n, "council tax", "arrears")) return "Council Tax"
   if (hasAny(n, "hmrc", "self assessment")) return "HMRC"
   if (hasAny(n, "dvla", "vehicle tax")) return "DVLA"
   if (hasAny(n, "ukvi", "visa", "biometric residence")) return "UKVI"
   return "Unknown"

}

Deadline extraction (supports both 12 Sept 2025 and 12/09/2025 formats):

private fun extractDeadline(raw: String, type: String? = null): String? {
   val n = norm(raw)
   return DATE_DMY_SLASH.find(n)?.value ?: DATE_DMY_TEXT.find(n)?.value
}

Explain response with summary + actions + citations:

private fun explainForType(type: String, text: String): Triple<String, List<String>, List<String>> {
   return when (type) {
       "Electoral Register" -> Triple(
           "Looks like an Electoral Register annual canvass letter.",
           listOf("Go to website", "Enter unique security code", "Confirm/update household"),
           listOf("https://www.gov.uk/register-to-vote")
       )
       "NHS Appointment" -> Triple(
           "An NHS clinic invite (likely vaccination).",
           listOf("Add to calendar", "Bring Red Book", "Call if reschedule needed"),
           listOf("https://www.nhs.uk/nhs-services/appointments-and-bookings/")
       )
       else -> Triple("Generic gov letter", listOf("Read carefully", "Follow instructions"), listOf("https://www.gov.uk"))
   }
}

Show an example API call/output:

Sample request:

POST /explain
{
   "text": "Your NHS vaccination appointment is on 25 Sept at Glasgow Clinic. Please bring your Red Book."
}

Sample response:

{
   "type": "NHS Appointment",
   "deadline": "25 Sept 2025",
   "summary": "This looks like an NHS appointment invite (e.g., vaccination). When: 25 Sept. Location: Glasgow Clinic.",
   "actions": [
   "Add the appointment date/time to your calendar.",
   "Bring any requested documents (e.g., child Red Book).",
   "If you need to reschedule, call the number on the letter."
   ],
   "citations": ["https://www.nhs.uk/nhs-services/appointments-and-bookings/"]
}

Compose UI:

A simple card shows type, deadline, summary, and actions.

ElevatedCard(Modifier.fillMaxWidth()) {
   Column(
       Modifier.padding(16.dp),
       verticalArrangement = Arrangement.spacedBy(8.dp)
   ) {
       Row(horizontalArrangement = Arrangement.spacedBy(16.dp)) {
           Text("Type:", fontWeight = FontWeight.SemiBold)
           Text(r.type)
       }
       Row(horizontalArrangement = Arrangement.spacedBy(16.dp)) {
           Text("Deadline:", fontWeight = FontWeight.SemiBold)
           Text(r.deadline ?: "—")
       }
       Divider()
       Text("Summary", style = MaterialTheme.typography.titleMedium)
       Text(r.summary)
       Divider()
       Text("Actions", style = MaterialTheme.typography.titleMedium)
       r.actions.forEach { a -> Text("• $a") }
       Divider()
       Text("Citations", style = MaterialTheme.typography.titleMedium)
       r.citations.forEach { c -> Text(c) }
   }
}

Results

Lessons Learned

What's next

Screenshots

Conclusion

LettersLens demonstrates how small, focused AI tools can make everyday tasks, such as opening a letter, less stressful and more actionable.

Try it