← all work N.BANZON_JR
case 01·ai / vector search·vivacity app

Retrieve, rerank, reason — an AI agent grounded in property data

Property information, room prices and each organization's FAQs are synced to Elasticsearch daily as dense vectors, embedded with Gemini text embedding. When a user message arrives the agent queries Elasticsearch, reranks the hits with the Vertex AI reranker, and passes the top results to Google Vertex AI for reasoning — with Azure OpenAI also in the mix on the model side. Prompt rules are tuned continuously as we see how the agent answers.

role
Backend Engineer
retrieval
Dense vector + rerank
embeddings
Gemini text embedding
reasoning
Google Vertex AI
walkthrough
Accommodation
mini program
CS Customer service tap
Any en-suite rooms near campus with bills included?
{{ thinkLabel }}
Three en-suite rooms match, all bills included. Rates and contract lengths pulled from the property record.
message…

What happens when a user taps customer service

{{ r.name }}
{{ r.detail }}

Retrieval and reranking happen before the model sees anything, so the answer is assembled from the property record rather than generated around it.

01 — context

People ask questions, not keywords

Students looking for accommodation ask things a keyword index answers badly: what a room costs, whether bills are included, how far a building is from campus, what the contract length is. Keyword search returns documents. The question needed an answer — but an answer that can only come from the platform's own property data, never invented.

02 — analysis

Retrieval is the hard part, not the model

The Elasticsearch migration had already put the index under the team's control, so dense vectors could live next to the documents they describe rather than in a separate store to keep in sync. Vector similarity alone, though, returns things that are merely close in meaning. The work since has been in the two stages after retrieval: reranking the candidate set so the genuinely relevant rooms and FAQ entries rise to the top, and constraining what the model is allowed to reason about once they do.

retrieve
Gemini text embedding turns property, pricing and per-organization FAQ documents into dense vectors in Elasticsearch.
rerank
The Vertex AI reranker reorders candidates so the top of the set is what the question actually asked for.
reason
Google Vertex AI and Azure OpenAI reason over the reranked set, under prompt rules we keep updating.
03 — architecture

Daily embeddings in, retrieve → rerank → reason out

daily sync → dense vectors → es · message → retrieve → rerank → reason
property info room prices org FAQs gemini text embedding · daily sync Elasticsearchdense vectors query path user message vector search Vertex AI rerank reasoning · vertex ai prompt rules, updated continuously candidates
01
Daily sync of embedded organization data
Property information, room prices and each organization's FAQs are embedded with Gemini text embedding and written to Elasticsearch every day, so the agent is never answering from a stale index.
02
Reranking before reasoning
Elasticsearch returns candidates by vector similarity; the Vertex AI reranker reorders them against the actual question, so Vertex AI reasons over a short, genuinely relevant set instead of everything that looked close.
03
Prompt rules as an ongoing surface
Prompt rules are updated continuously as real questions come in — tightening what the agent may infer, and how it should answer when the retrieved data does not cover the question.
04 — outcome

Questions answered from the platform's own data

The agent handles property information queries, organization FAQs and room prices by retrieving first, reranking second, and reasoning last. Because retrieval and keyword search share a cluster, index work benefits both the classic search box and the agent — and because ranking and prompt rules are tuned separately from the code, improving answer quality does not mean redeploying the service.

user
how much is an en-suite near campus, bills included?
retrieval
vector hits → 12 candidates · rooms · price fields · bills FAQ
rerank
vertex ai reranker → top 3 · en-suite rates, inclusions
vertex ai
reasons over the reranked set, under current prompt rules_
Vector Search Embedding Elasticsearch Google Vertex AI Vertex AI Reranker Azure OpenAI Gemini text embedding Prompt rules NestJS Behavioral Analytics
next · case 02
A custom KrakenD gateway →
case 03
Migrating search off Algolia →