For success in the dashboard, you need to read the definition first
Microsoft Learn's Copilot Studio official documentation distinguishes session results such as user confirmed resolution and system presumed resolution, and provides a path to view specific sessions. This example reminds us that statistical labels have their own definitions. We cannot just see a solved ratio and assume that every answer has been manually verified. Product functions and permissions are subject to the current official documents.
The following is a set of post-launch review methods proposed by ChinaOPC. It is suitable for service portals maintained by a small number of business personnel. It does not require the purchase of specific products, nor is it an acceptance conclusion for a certain customer project that has been completed. It supplements pre-launch testing and focuses on how to find and track problems in real use.
Start with a possible consequence reply
When labor time is limited, you can first arrange the review sequence according to business consequences. Involving price and delivery commitments, customer disputes, assertions that cannot be found in the data, repeated questioning, and responses after calling business actions, are suitable for priority inspection. Regular questions should also be extracted in rotation to avoid looking at only a few records complained by users. There is no fixed sampling ratio that applies to all teams.
When checking a reply, first retain the necessary context: what the user is asking, what information the system relied on, whether an action was performed, and who ended the processing. You may not be able to tell the problem just by looking at a single sentence answer. Data should be used within the approved purpose and access scope, and all customer conversations should not be copied to unrelated tools for review.
How to write a review sheet
- Tasks and basis: question number, business topic, knowledge items used and versions.
- Judgment and impact: Whether the answer is supported, whether conditions are missing, and whether it has affected subsequent processing.
- Takeover and Correction: Who handles customer issues, who modifies knowledge or processes, and which conclusions remain to be confirmed.
- Verification and closure: Which questions were retested after the repair, when they were reviewed, and who confirmed the closure.
For example, in an after-sales scenario clearly marked as simulation, the data stated a time range that required manual confirmation, but the robot gave a firm promise. The review should not just give the answer a low score, but should determine whether the data version is wrong, the conditions were not retrieved, or the response rules are not constrained, and make it clear who will take over the customer's problem. This example is not a complaint that occurred.
Repair must go back to next use
After revising the knowledge items, retest the original questions, the questions asked in a different way, and the boundary questions that were originally supposed to be handed over to a person. If you only correct the text on the page without checking whether the system has read the new version, the old reply may still be given next time. Each time, only the range that you have actually tested is recorded, and individual sample passes are not written as the overall accuracy.
When summarizing problems every week or according to business rhythm, you can compare whether similar errors reoccur, whether manual takeover has been completed, and whether data repair can be traced back. It is more helpful to record the scope of the inspection and the parts not covered than to give a favorable number without a denominator.
Extended reading: FAQ retrieval test before going online; Conditions for AI customer support handover to a person; Multiple customer knowledge base isolation.