Start with the research job, not a brand ranking
AI research products can return different things: a synthesized answer with links, a list of public posts, a comment sample, or an exportable dataset. A polished answer and a raw collection are not interchangeable outputs. Before comparing tools, state the decision you need to make and the kind of evidence that would change it.
For example, a team checking a vendor policy update might need the effective date, the exact changed clause, any exception, and the official policy page. A product-research task might instead need first-hand reports from public discussions, the original post URLs, and enough surrounding context to distinguish a new-version issue from an older one. Those are different test jobs, even if the same chatbot can be asked about both.
Define the scope before opening the products: question, date range, geography, sources that count, access limits, and the evidence units a useful result must contain. If the task is open-ended, say that there is no complete reference set. Do not call an incomplete sample a recall score.
- Choose one decision-relevant task and define what counts as evidence.
- Set the time window, geography, source types and acceptable access route.
- Create a short reference list when the question has a bounded, checkable answer.
- Mark unknowns and unavailable material explicitly rather than treating them as negative evidence.
NIST's Generative AI Profile recommends iterative, documented test, evaluation, validation and verification processes as part of managing generative-AI risk.
NIST AI 600-1, Generative AI Profile (July 2024)
What it does not establish: This is voluntary, cross-sector risk-management guidance. It does not prescribe this article's checklist or certify any research product.
Score separate questions separately
One overall score hides useful differences. A tool can find the right document but summarize it inaccurately; it can write a correct answer while omitting a key citation; it can cite a page that does not support the sentence beside it. Keep retrieval, answer correctness and attribution as separate observations.
Research on citation-supported generation makes this distinction concrete. The ALCE benchmark evaluates answer correctness and citation quality, including whether cited sources are relevant and whether necessary sources are present. That is a useful vocabulary for a human audit; the benchmark results themselves are not a score for today's commercial products.
| Dimension | What to record | Simple calculation or label | Main caveat |
|---|---|---|---|
| Task coverage | Did the answer include the required evidence units? | Required units correctly covered ÷ required units defined | Meaningful only when the task has a bounded reference set; not a measure of all web content. |
| Claim support | Does the linked original evidence support the factual statement? | Supported, partial, unsupported or not checked for each audited claim | A citation being present does not prove that it supports the claim. |
| Citation precision | Do the sampled citations support the nearby claims? | Relevant/supporting citations ÷ citations opened and audited | State the sample and the support rule; do not audit only the easiest links. |
| Citation completeness | Are the important factual claims linked to adequate evidence? | Key claims with adequate citations ÷ key claims requiring support | This is not the same denominator as citation precision. |
| Source profile and freshness | What source types, dates and independent publishers appear? | Describe primary/secondary sources, dates, duplication and conflicts | Do not treat domain labels or citation counts as proof of truth. |
| Traceability and context | Can another reviewer reopen the source and understand what was retrieved? | Record URL, date, input type, surrounding context and missing items | A snippet or summary is not equivalent to a read source or full comment thread. |
| Access and effort | What did the run require and how long did verification take? | Record plan, model/mode, API or account requirement, cost and review time | Compare like-for-like access; a free-tier result and a paid research mode are different conditions. |
ALCE separates answer correctness from citation quality and discusses citation precision and recall for citation-supported answers.
Gao et al., Enabling Large Language Models to Generate Text with Citations (EMNLP 2023)
What it does not establish: ALCE is a research benchmark on defined datasets and systems, not a live evaluation of current Perplexity, ChatGPT, Gemini, Brave or Kruqond products.
The TREC Retrieval-Augmented Generation track treats retrieval and retrieval-plus-generation as distinct but complementary evaluation tasks.
TREC Retrieval-Augmented Generation Track
What it does not establish: Track tasks and collections are benchmark-specific; their scores do not transfer automatically to consumer research assistants or social-comment workflows.
Make the comparison repeatable
Compare tools that attempt the same job. If one product is asked to write a researched report and another is asked to retrieve comments from a supplied URL, the results answer different questions. You can compare a workflow chain, but label each stage and score each output on the job it actually performs.
Use the same task wording where the interfaces allow it. If one product requires a URL and another accepts a keyword, keep the task constant, record the different input path, and do not pretend the prompts were identical. Capture the product name, date and time, plan tier, model or mode, region, filters, query, source list and any error or missing-data notice.
A small pilot might use five to ten realistic tasks and repeat unstable or high-impact tasks more than once. That is a practical starting design, not a statistically universal sample size. If you need a stronger result, increase the task set, sample sources consistently, involve a second reviewer, and publish disagreements rather than hiding them in a mean score.
- Prewrite the task set and scoring rules before seeing outputs.
- Use comparable account tiers, settings, dates and geographic context; record any mismatch.
- Save each exact prompt and the full set of returned source URLs, not just a screenshot of the answer.
- Audit a declared sample of citations and source records using the same support labels.
- Repeat selected tasks when output variation matters; keep individual run results as well as summaries.
- Have a second reviewer independently score a subset when the result will influence a consequential decision.
NIST's profile says robust TEVV processes can be applied iteratively and documented across the AI lifecycle, and calls out retaining TEVV history.
NIST AI 600-1, sections on pre-deployment testing and documentation
What it does not establish: The profile offers risk-management actions rather than a required number of tasks, runs or reviewers; the pilot sizes above are practical suggestions, not NIST requirements.
Open citations and audit the claim beside each one
For every factual statement you sample, open the linked source and check the exact support. Use four labels: supported, partly supported, not supported, and not checked. Add a note if the page is unavailable, the cited passage is not the original source, or the wording depends on a date, version or exception that the answer leaves out.
Keep two tallies. Citation precision asks how many of the citations you checked actually support the associated claims. Citation completeness asks how many of the key claims that need evidence have adequate citations. A system can do well on one and poorly on the other. Record your denominator and sampling rule so a reader can reproduce the calculation.
If a result concerns social discussion, preserve the post or video URL and whatever context is necessary to interpret the comment. A search result, a directly opened post and a retrieved comment sample are different evidence states. Missing or deleted material is a retrieval limit; it is not proof that nobody discussed the topic.
OpenAI's Search help article says web-search answers may include citations and warns that citations can be incomplete, outdated or incorrect; it recommends opening sources and checking the relevant date and authority.
OpenAI Help Center: Searching the web with ChatGPT
What it does not establish: This is current product guidance from OpenAI, not an independent measurement of citation accuracy or a guarantee about any particular answer.
Perplexity's help center describes numbered citations linking to original sources and its Search and Research workflows.
Perplexity Help Center: How does Perplexity work?
What it does not establish: This is a vendor description of product behavior; it does not establish citation support, completeness or quality for a test run.
Compare the documented workflow before scoring outputs
Product documentation helps you choose the right test, but documentation is not performance evidence. Perplexity describes cited web answers and a Research mode. OpenAI documents web search with links and source review. Kruqond's source guide distinguishes public-source discovery from tools that open one discussion or retrieve available comments from several results. Those descriptions suggest different test cases; they do not prove that one product retrieves more, cites better or is more accurate.
For each product, write down the relevant task and supported input: keyword search, public URL, browser capture, API query, or uploaded file. Check the access tier and what the result exposes. A tool that finds public posts may be useful for discovery but still may not provide a complete research corpus. A cited answer may be useful for a first pass but still need claim-by-claim verification.
- Answer engines: test answer support and citation completeness on a defined question set.
- Social-source discovery: test whether relevant original posts are found and links/context are preserved.
- Comment workflows: test which comments and replies were available for a declared set of URLs.
- Dataset tools: test inclusion rules, export fields, collection windows and repeatability.
- If combining tools, score each handoff separately instead of assigning one combined winner.
Kruqond's source coverage page distinguishes topic/post discovery from opening a public URL and retrieving available comments; it warns that comment results are a retrieved sample, not a survey of everyone who saw a post.
Kruqond Sources: source coverage and limits
What it does not establish: This is Kruqond's own product documentation. It supports the stated workflow description, not a claim that the product outperforms another service.
The ChatGPT and Perplexity help pages describe cited web answers and ways to inspect source links.
OpenAI and Perplexity product help centers
What it does not establish: Vendor documentation is useful for defining the tested feature but cannot substitute for a matched, observed comparison under recorded conditions.
Avoid scores that look more certain than the evidence
Do not rank a product by the number of citations, the length of its answer, or how authoritative the domains look at a glance. Duplicate links can inflate counts; a prestigious domain can be irrelevant to the claim; a fluent answer can omit the one fact that changes the decision. Separate accuracy, retrieval coverage, attribution, freshness, context and operational fit.
For open-web or social-listening questions, a complete gold-standard set often does not exist. In that case, report what you searched, what you inspected, how the sample was selected, and which sources or comments could not be retrieved. Use descriptive comparisons rather than false precision. Do not generalize from a few visible posts to all users or claim that a system found every relevant result.
If you publish an overall score for a local decision, disclose the weights and why they matter to that decision. For instance, an investigator may value source provenance more than response speed, while a customer-support team may care about response time and escalation visibility. The weighting is a choice—not a universal property of the tools.
ALCE's paper separates correctness and citation quality and notes limitations in its automated citation evaluation, reinforcing that a metric and its test set have boundaries.
Gao et al. (2023), ALCE paper and evaluation discussion
What it does not establish: The paper's findings apply to its benchmark setup and models; this guide uses its metric distinctions, not its reported scores as a current product ranking.
Keep the audit log with your conclusion
Download the CSV template above and keep one row per claim/source pair. Repeat run metadata on each row, use a blank-source row to record a missing expected item, and add your own columns if your field needs them. The log asks for the prompt, tool mode, access tier, source URL, claim, audit label and context gap; it does not calculate a universal score for you.
When sharing results, include the task set, date, product and mode, account conditions, reference sources, sample size, scoring rules, disagreements and limitations. If a feature or plan changes, rerun the relevant test rather than carrying old observations forward as current facts. A transparent small pilot is more useful than a broad winner claim that readers cannot reproduce.
- Keep the blank source-level log with your notes and the original links.
- Publish the method and sample limits beside any comparison table.
- Label documentation-only summaries separately from observed product tests.
- Do not imply independent review when the publisher is affiliated with a product being discussed.
NIST's profile recommends documenting TEVV work and maintaining its history as part of responsible AI risk management.
What it does not establish: This article's downloadable log is an editorial resource, not a NIST form, compliance artifact or certification.
Scope and disclosure
Kruqond publishes this guide and is one of the products readers may choose to evaluate. This page presents a source-linked evaluation method; no new matched benchmark of current AI search products was run for it, and no tool is named a quality winner. It was prepared with AI assistance from the linked public sources, checked on November 17, 2026. The template is an unvalidated blank worksheet, not research data. Product features, access rules and plans can change; confirm the current documentation before relying on a comparison.