Production AI Analytics Platform

Business-critical data sat in spreadsheets and disconnected systems. Every question a stakeholder had — trends, comparisons, anomalies — required someone to manually pull, clean, and format it. Turnaround was measured in days. By the time an answer arrived, the question had usually changed.

The requirement was a system where non-technical users could ask questions in plain language and get answers they could trust.

Three services, deliberately separated by workload characteristics rather than by convenience.

Ingestion and API layer — Go. Chosen for concurrency and predictable memory under sustained load. The pipeline pulls from multiple upstream sources on independent schedules, normalises heterogeneous schemas into a common internal representation, and writes to Postgres. Go's goroutine model made fan-out ingestion straightforward without an external job queue, which kept the deployment surface small — a real constraint on a solo project.

ML and embedding layer — Python. Runs separately from the request path. Handles feature extraction, clustering, and embedding generation. Kept out of the hot path deliberately: embedding generation is slow and bursty, and coupling it to user requests would have made p99 latency unpredictable. Results are precomputed and written back to the datastore.

Interface — Next.js / TypeScript. Server components for the data-heavy views, client components only where interaction demanded it. Query results stream progressively rather than blocking on a complete response, because some aggregations legitimately take seconds and a spinner for five seconds reads as broken.

Precomputed embeddings over on-demand generation. On-demand would have kept the index perfectly fresh. Precomputing meant accepting staleness measured in minutes in exchange for sub-second semantic search and a bounded, predictable inference cost. For this use case — analytical questions, not real-time monitoring — minutes of staleness were invisible to users. Sub-second response was not.

Two languages instead of one. Running the whole backend in Python would have been simpler to maintain. But the ingestion layer's concurrency profile and the ML layer's compute profile pull in opposite directions, and forcing them into one runtime meant one of them would be badly served. The cost was a second deployment target and a serialisation boundary. Worth it.

Designing for model uncertainty as a first-class state. The hardest interface problem wasn't displaying results — it was displaying low-confidence results. An analytics tool that confidently returns a wrong number is worse than one that returns nothing. The UI surfaces confidence explicitly and degrades to showing the underlying records when the semantic layer isn't sure, so users can always fall back to verifying the source rather than trusting a black box.

Postgres over a dedicated vector database. pgvector was sufficient at this data volume and avoided operating a second stateful system. If the corpus grows an order of magnitude this becomes the first thing to revisit — a known, accepted ceiling rather than an oversight.

Reporting turnaround moved from days to minutes. The system runs in production and is used for recurring operational decisions.

The ingestion layer grew organically as new sources were added and now carries per-source special-casing that should have been abstracted into a plugin interface around the third connector. It works and it's tested, but adding a fourth source costs more than it should. Recognising that boundary earlier is the clearest lesson from the build.