Advanced Analysis Tools
Context
ReviewLens AI is a review intelligence portal for ORM consultancies. The agent currently has 10 tools — 5 data tools (search, sentiment, chart, stats, follow-ups), 3 knowledge tools, and 2 report tools. These are all flat data access — the agent pulls raw data and does all analytical work in-context.
I want to add tools that give the agent structured analytical capabilities so it behaves more like an experienced ORM analyst with workflows, not just a chatbot that searches text.
New Tools to Add in app/tools.py
1. compare_segments
Purpose: Compare two segments of reviews side by side. This is the core analytical workflow — 5-star vs 1-star, recent vs older, reviews mentioning topic A vs topic B.
Input schema:
{
"segment_a": {
"type": "object",
"properties": {
"label": { "type": "string", "description": "Human label for this segment, e.g. 'Positive reviews' or 'Recent (last 3 months)'" },
"query": { "type": "string", "description": "Optional semantic search query to filter this segment" },
"min_rating": { "type": "number" },
"max_rating": { "type": "number" },
"date_after": { "type": "string", "description": "ISO date string, only include reviews after this date" },
"date_before": { "type": "string", "description": "ISO date string, only include reviews before this date" }
},
"required": ["label"]
},
"segment_b": { "same structure as segment_a" }
}
Implementation approach:
- For each segment, filter the full review set (from vectordb.get_all_reviews) by the provided criteria (rating range, date range)
- If a query is provided, use vectordb.search_reviews with those filters instead
- For each segment, compute: review count, average rating, common words/phrases (simple word frequency excluding stopwords), and collect the review texts
- Return a structured comparison: counts, avg ratings, top terms unique to each segment, top terms shared, and a sample of 3-5 representative review texts per segment
The agent receives a pre-structured diff and narrates it, rather than burning multiple tool calls and synthesizing raw reviews manually.
2. extract_themes
Purpose: Discover and rank themes/topics across the review corpus. Goes beyond semantic search by analyzing a broad slice of the dataset rather than just returning the top N matches for a query.
Input schema:
{
"focus": { "type": "string", "description": "Optional focus area — e.g. 'complaints', 'praise', 'feature requests'. Leave empty for general theme extraction." },
"min_rating": { "type": "number", "description": "Optional: only analyse reviews with rating >= this" },
"max_rating": { "type": "number", "description": "Optional: only analyse reviews with rating <= this" },
"max_reviews": { "type": "integer", "description": "Max reviews to analyse. Default 50, max 100.", "default": 50 }
}
Implementation approach:
- Pull reviews from vectordb.get_all_reviews, apply rating filters if provided
- If focus is provided, use vectordb.search_reviews with a broad query based on focus to get relevant subset
- Run n-gram extraction (bigrams and trigrams) across the review texts using simple tokenization — no external NLP library needed, just split on whitespace/punctuation, lowercase, remove stopwords, count n-gram frequencies
- Group related n-grams into themes (e.g., "noise cancellation", "noise cancelling", "ANC" could cluster). Simple approach: if two n-grams share a content word, they're related
- For each theme: frequency count, percentage of reviews mentioning it, average rating of reviews containing it, 2-3 representative review IDs
- Return themes ranked by frequency, top 10-15
This is the difference between "here are some reviews, figure it out" and "here are the 8 themes across 50 reviews, ranked by frequency, with evidence."
Build a small stopwords set directly in the tool (common English words — the, a, is, was, etc. — plus review-specific noise like "product", "review", "bought", "ordered"). Don't import nltk or spacy. Keep it self-contained. Maybe 100-150 stopwords.
3. find_anomalies
Purpose: Scan the dataset for suspicious patterns and data quality signals. This is the "surprise us" tool — nobody building a review chatbot includes review manipulation detection, but it maps directly to the ORM business context.
Input schema:
{}
No inputs needed — it scans the full dataset.
Implementation — check for these patterns:
-
Rating-text mismatches: Reviews where the rating contradicts the text sentiment. Simple heuristic: 4-5 star reviews containing strong negative phrases ("terrible", "worst", "awful", "waste of money", "don't buy", "returning") or 1-2 star reviews containing strong positive phrases ("amazing", "perfect", "love it", "best ever", "highly recommend"). Return the mismatched review IDs and the conflicting signals.
-
Duplicate/near-duplicate text: Reviews with very similar text that might indicate astroturfing. Simple approach: normalize texts (lowercase, strip punctuation), check for exact duplicates. For near-duplicates, compare first 50 characters — if multiple reviews share the same opening, flag them.
-
Review clustering: Unusual concentrations of reviews in short timeframes. Group reviews by date (daily buckets) and flag any day with 3x+ the average daily volume — could indicate review bombing or incentivised campaigns.
-
Suspiciously short/long reviews: Flag reviews that are unusually short (< 20 chars) or unusually long (> 3x the average length) as potential quality outliers.
Return all findings as a structured object with categories, each containing the flagged review IDs, the pattern detected, and a severity indicator. The agent then interprets the findings and decides what's worth surfacing.
4. get_review_by_id
Purpose: Direct lookup of a specific review by ID. Useful when the user references a specific review from a prior answer, or when the agent needs to cross-reference a previously cited review.
Input schema:
{
"review_id": { "type": "string", "description": "The review ID to look up (e.g., 'review_42')" }
}
Implementation: Call vectordb to get the review by ID from the ChromaDB collection using col.get(ids=[review_id]). Return the full review text, all metadata, and any other reviews by the same author if present in the dataset.
This requires a small addition to app/vectordb.py — add a get_review_by_id(session_id, review_id) function.
5. Date filtering on search_reviews
Not a new tool — enhance the existing search_reviews tool with date range params.
Add to the existing input schema:
{
"date_after": { "type": "string", "description": "Only return reviews after this date (ISO format, e.g. '2024-06-01')" },
"date_before": { "type": "string", "description": "Only return reviews before this date (ISO format, e.g. '2024-09-30')" }
}
Implementation: ChromaDB supports metadata filtering. Add $and conditions for date comparisons to the existing where clause builder. ChromaDB stores dates as ISO strings, so string comparison works for ISO format dates (lexicographic order matches chronological order for ISO dates).
Update app/vectordb.py
Add the get_review_by_id function:
def get_review_by_id(session_id: str, review_id: str) -> dict[str, Any] | None:
"""Get a single review by ID."""
# Use col.get(ids=[review_id], include=["documents", "metadatas"])
# Return the review dict or None if not found
Update app/prompts.py
Add the new tools to the Tools section of the system prompt. Brief descriptions:
- compare_segments — Compare two groups of reviews side by side (e.g., positive vs negative, recent vs older, by topic). Use for any "how does X differ from Y" question.
- extract_themes — Discover and rank the main themes/topics across the review corpus. Use when the user asks broad questions like "what are people talking about?" or when you need to understand the landscape before drilling in.
- find_anomalies — Scan for data quality issues and suspicious patterns: rating-text mismatches, duplicate reviews, unusual clustering, outlier lengths. Use proactively when doing the initial briefing, or when the user asks about data quality/fake reviews.
- get_review_by_id — Look up a specific review by its ID. Use when the user references a specific review from a prior answer.
Also update the search_reviews description to mention the new date filtering capability.
Update the knowledge files
In knowledge/analysis-patterns.md, add a section on "Data Quality & Anomaly Detection" that describes what each anomaly pattern means in ORM context (astroturfing indicators, review bombing, incentivised reviews, etc.). This way the agent has domain context for interpreting find_anomalies results.
What NOT to change
- Don't change existing tool signatures (except adding date params to search_reviews)
- Don't change the agent loop, SSE streaming, frontend, or templates
- Don't add external NLP dependencies (no nltk, spacy, etc.) — keep text analysis self-contained with simple tokenization and frequency counting
- Don't change the knowledge module or store module (beyond what's specified)
Tool priority
If you need to implement incrementally: compare_segments and extract_themes first (these transform response quality), then find_anomalies (the wow factor), then get_review_by_id and date filtering (quality of life).