THE PRACTICAL VIEW
The question to ask is not simply whether an answer sounds right. It is whether the system found the right evidence, interpreted it correctly and was allowed to show it to this user.
Understand the retrieval step
A question is used to find relevant material. Selected passages are provided to the model alongside instructions, and the model produces a response. This differs from asking a general chatbot to answer from its training alone. It does not mean the model has permanently learned your business.
Start with a controlled knowledge base
Choose documents with clear owners, current versions and agreed scope. Remove duplicate or superseded guidance. Define which source takes precedence when documents disagree. If an answer depends on a policy’s effective date, that date needs to be available to the system.
WORKED EXAMPLE · ILLUSTRATIVE
One question, two versions of the policy
Suppose a consultant asks which documents a contractor needs before mobilisation. The knowledge base contains last year’s checklist and a newer client-specific requirement. A useful response must identify the applicable client, use the current source and flag any missing context. It should make the supporting material available to the authorised reviewer.
Where it can go wrong
Returning a citation to an old checklist does not make the answer reliable. Test retrieval accuracy, version handling and permission enforcement separately. Where the evidence conflicts, asking for clarification is a better outcome than guessing.
Carry permissions into the answer
Access control must apply to retrieval, not simply to the screen where someone asks a question. A recruiter should not receive restricted payroll information through an AI answer. Test permissions with representative users and check how logs, caches and connected services handle information.
Make uncertainty visible
An answer should identify the material that supports it and say when the evidence is insufficient. Retrieval can miss the right document, and a model can misinterpret a passage. Source references help a person check an answer, but references themselves must also be validated.
Test with questions people actually ask
Include straightforward requests, ambiguous wording, outdated policies, conflicting documents and questions the system should refuse to answer. Compare responses against an agreed answer set. Measure whether the correct evidence was retrieved separately from whether the final response was useful.
Connect knowledge to action carefully
Answering a question and updating a business record are different responsibilities. Any action needs explicit permission, validation and an approval path where appropriate. Ella’s connected context can support useful answers and workflows, with the sources and permitted actions agreed for each business.
PUT IT TO WORK
Take this into your next team conversation
Tick off the points you have covered. This is a working checklist, not a compliance score. Selections reset when you leave the page.
0 of 4 points covered
Questions worth asking
Does RAG eliminate hallucinations?
No. It can provide relevant material, but retrieval can fail and a model can misread or overstate that material. Test the evidence selection and the answer, and retain human review where the consequence requires it.
Is RAG the same as training a model on our documents?
No. In a typical RAG workflow, selected source material is supplied at the time of the request. That is different from changing a model’s parameters through training. Retention and processing still depend on the chosen services and configuration.
Further reading
For data-use decisions, refer to the OAIC’s guidance on commercially available AI products. For a structured risk-management reference, see the NIST AI Risk Management Framework. These sources support further assessment; they do not validate an individual product or configuration.
Continue with LLMs in Recruitment: Connect Data Safely.
This guide builds on Jordan’s Hyper Automate resource library, updated for Wiity. Worked examples illustrate the approach; they are not customer case studies.