top of page
Logo of Nebula, Digital Marketing Agency

Nebula Personalization Tech Solutions Pvt. Ltd.

Est. 2010

Nebula AI Research

RASA-Analyst Guide
How to Install, Run, and Interpret the RASA Evaluation Engine

Implementation Guide · Nebula Personalization Tech Solutions Pvt. Ltd.
Model: ollama.com/nebulatech/rasa-analyst · 4.9GB · 128K context
Research basis: Verma & Agarwal (2026), DOI: 10.5281/zenodo.20325460

RASA-Analyst is the official evaluation engine for the Retrieval-Aware Semantic Architecture (RASA) framework, developed by Nebula Personalization Tech Solutions Pvt. Ltd. It runs locally on your hardware via Ollama, requires no API key or internet connection after installation, and scores content chunks across all five RASA dimensions — Retrieval Probability (RP), Semantic Chunk Coherence (SCC), Entity Clarity Score (ECS), Synthesis Compatibility Index (SCI), and Citation & Grounding Potential (CGP) — in a single evaluation pass.

​

This guide covers everything you need to install RASA-Analyst, run your first evaluation, interpret the score report, and apply the improvement recommendations it returns.

What RASA-Analyst Does

Given a content chunk as input, RASA-Analyst returns a structured evaluation report containing:

  • A score from 1–10 for each of the five RASA dimensions

  • A weighted composite RASA score (formula: RP×0.25 + SCC×0.20 + ECS×0.20 + SCI×0.20 + CGP×0.15)

  • A verdict: PUBLISH (≥ 8.0), REVISE (6.0–7.9), or REJECT (< 6.0, or SCI < 6.0 regardless of composite)

  • For each dimension scoring below 8.0: one or more quoted phrases from the input that produced weak signals, with a brief explanation of why each phrase lowers the score

  • One concrete, actionable improvement recommendation per underperforming dimension

RASA-Analyst does not rewrite your content. It diagnoses it. The improvement work is performed by the content team using the specific guidance the model returns.

System Requirements

Requirement
Minimum
Recommended
RAM
8 GB
16 GB+
Storage (model)
4.9 GB free
10 GB+ free
Operating System
macOS 12+, Ubuntu 20.04+, Windows 10+
macOS 14+ or Ubuntu 22.04+
Ollama
0.1.x or later
Latest stable release
GPU (optional)
CPU inference supported
NVIDIA GPU with 8GB+ VRAM for faster inference

RASA-Analyst uses a 128K context window, supporting content chunks up to approximately 90,000 words per evaluation pass.

​

Installation

Step 1 — Install Ollama

Ollama is the runtime that hosts and runs RASA-Analyst. Install it from ollama.com.

macOS / Linux:

curl -fsSL https://ollama.com/install.sh | sh

Windows: Download the installer from ollama.com/download/windows and run it.

Verify the installation:

ollama --version

​

Step 2 — Pull the RASA-Analyst model

ollama pull nebulatech/rasa-analyst

This downloads the 4.9 GB model to your local machine. The download is a one-time operation. After pulling, the model is available offline.

​

Step 3 — Verify the model is available

ollama list

You should see nebulatech/rasa-analyst in the list of available models.

Running a Single-Chunk Evaluation

Start the model

ollama run nebulatech/rasa-analyst

RASA-Analyst opens an interactive prompt. Paste your content chunk at the >>> prompt and press Enter (or Ctrl+D on macOS/Linux, Ctrl+Z on Windows, to submit a multi-paragraph chunk).

​

What to submit

  • Submit one chunk at a time — one H2/H3 section or one discrete topic unit (150–400 words is optimal)

  • Do not include page titles, navigation text, footer content, or HTML markup

  • Submit plain text — remove all formatting tags before pasting

  • For tables or lists, paste the content in plain text form with line breaks between items

 

Context tip: RASA-Analyst evaluates the chunk as a standalone unit, not in the context of the surrounding page. This is intentional — it mirrors how AI retrieval systems encounter content chunks. Do not add "this section is part of a larger article about X" context before the chunk. Score it as it would be retrieved.

Reading the Score Report

RASA-Analyst returns a structured evaluation report with the following format:

RASA EVALUATION REPORT

DIMENSION SCORES

  • Retrieval Probability (RP): [score]/10

  • Semantic Chunk Coherence (SCC): [score]/10

  • Entity Clarity Score (ECS): [score]/10

  • Synthesis Compatibility Index (SCI): [score]/10

  • Citation & Grounding Potential (CGP): [score]/10

COMPOSITE RASA SCORE: [score]/10 VERDICT: [PUBLISH / REVISE / REJECT]

The report then provides a Dimension Analysis section that details observations and recommendations for each dimension scoring below 8.0, including quoted examples from the input chunk and specific improvement actions. It concludes with a Summary paragraph highlighting the chunk’s primary retrieval strengths and the highest-priority areas for improvement.

Interpreting the verdict

RASA-Analyst Verdict Guidelines

​

The final verdict is determined by the composite RASA score and individual dimension thresholds:

  • PUBLISH (Composite ≥ 8.0): The chunk is retrieval-ready with all dimensions at acceptable levels. Publish as-is. You may optionally address any dimension below 9.0 for further optimisation.

  • REVISE (Composite 6.0–7.9): The chunk has retrievable elements, but one or more dimensions are limiting its performance. Apply the recommendation for the lowest-scoring dimension and re-score after revision.

  • REJECT (Composite < 6.0, or SCI < 6.0): The chunk is not retrieval-ready. This occurs when the composite score is too low, or when the Synthesis Compatibility Index (SCI) falls below 6.0.

SCI Override: If RASA-Analyst returns REJECT even when the composite score is above 6.0, check the SCI score first. An SCI score below 6.0 triggers an automatic REJECT regardless of the overall composite. This is the only dimension with a hard override, reflecting the critical impact of synthesis-incompatible content in AI retrieval environments.

Prompt Templates

These templates can be pasted directly into the RASA-Analyst prompt for different evaluation modes.

Standard single-chunk evaluation

Please score the following content chunk using the RASA framework. Return scores for all five dimensions (RP, SCC, ECS, SCI, CGP), the composite score, and the verdict (PUBLISH / REVISE / REJECT). For each dimension scoring below 8.0, quote the specific phrase causing the issue and provide one concrete improvement recommendation. [PASTE CHUNK HERE]

Targeted dimension analysis

Please score the following content chunk on the RASA Synthesis Compatibility Index (SCI) dimension only.

 

Focus your analysis on: factual precision, internal consistency, synthesis safety, and absence of contradiction signals.

 

Quote specific phrases that lower the SCI score and provide one concrete revision recommendation.

[PASTE CHUNK HERE]

Comparative scoring (before/after revision)

Please score both of the following content chunks using the full RASA framework. Label them ORIGINAL and REVISED. Return scores for all five dimensions and the composite for each. Identify which dimensions improved, which regressed, and whether the revised chunk has crossed a verdict threshold.

 

ORIGINAL:

[PASTE ORIGINAL CHUNK]

 

REVISED:

[PASTE REVISED CHUNK]

Batch audit (multiple chunks from one page)

I am conducting a RASA content audit on the following [N] chunks from a single page. Please score each chunk separately. For each, return: all five dimension scores, composite score, verdict, and the single highest-priority improvement recommendation.

 

Label each chunk as CHUNK 1, CHUNK 2, etc.

 

CHUNK 1:

[PASTE CHUNK 1]

 

CHUNK 2:

[PASTE CHUNK 2]

 

[continue for each chunk]

Applying Improvement Recommendations

RASA-Analyst's recommendations follow a consistent pattern per dimension. Here is how to interpret and apply each type:

Dimension
Typical recommendation type
How to apply
RP
"Replace '[generic phrase]' with a named entity or precise technical term"
Identify the generic phrase in your text. Replace it with the specific framework name, tool name, methodology, or organisation it refers to.
SCC
"This chunk references '[pronoun/shorthand]' without establishing the referent within the chunk"
Find the pronoun or shorthand. Replace it with the full entity name, or add a brief re-identification at the start of the chunk.
ECS
"'[entity]' is named inconsistently across the chunk — used as '[variant A]' and '[variant B]'"
Choose one canonical name and apply it consistently. Check for abbreviation/full-name inconsistency.
SCI
"'[sentence]' contains a claim that cannot be synthesised without qualification"
Add a specific scope boundary ("in RAG pipelines," "for content scoring below 6.0"), a data source, or a date range. Or split into two sentences: one stating the general claim, one providing the caveat.
CGP
"No named source, statistic, or institutional attribution present in this chunk"
Add at minimum: one named author/organisation + one grounded claim with origin. For research-heavy content, add a DOI.

After applying recommendations, re-run the chunk through RASA-Analyst to confirm improvement. A revised chunk should be re-scored before being marked as complete in your audit log.

RASA-Analyst Model Specifications

Specification
Value
Model name
nebulatech/rasa-analyst
Hosted at
https://ollama.com/nebulatech/rasa-analyst
Model size
4.9 GB
Context window
128,000 tokens (~90,000 words)
Runtime
Ollama (local, offline-capable)
API key required
No
Data privacy
All inference runs locally — content is not transmitted to any external server
Developer
Nebula Personalization Tech Solutions Pvt. Ltd.
Framework basis
Retrieval-Aware Semantic Architectures (RASA), DOI: 10.5281/zenodo.20325460

Related Resources

​

Framework Reference: Verma, A. & Agarwal, S. (2026). Retrieval-Aware Semantic Architectures (RASA) for AI-Native Search. Nebula Personalization Tech Solutions Pvt. Ltd. DOI: 10.5281/zenodo.20325460

bottom of page