Editorial Standards · Keyword: desk review vs practical test · Scheduled: Day 03, 21:00 PM
Desk Reviews vs. Hands-On Testing: Our Strict Transparency Policy
Readers should be able to tell whether a reviewer opened a course, tried software, tested a device, or evaluated only the information available on public pages. These methods answer different questions. A desk review can compare a seller’s stated curriculum, price, and policies. Hands-on testing can examine the actual interface or materials. Neither method, by itself, proves that every buyer will have the same experience.
Our transparency policy is to describe the method in direct language. We do not use “tested,” “used,” or “experienced” to describe a product unless the reviewer actually did the relevant action. If a review is based on public materials, it should say so, explain the sources, and avoid conclusions that require direct product access.
What a desk review can establish
A desk review uses sources such as an official offer page, published terms, sample materials, seller documentation, and reliable independent references. It can record what a seller currently states: for example, the listed course modules, formats, price, access duration, or refund instructions. It can identify contradictions between public pages or note that a detail is missing.
A desk review can also place a product category in context. An independent source may explain what a common technical term means or what a relevant safety authority advises. That context does not make the independent source a test of the product. A review must not imply that an agency, researcher, or publication approved a seller’s offer when no such endorsement exists.
There are important limits. Reading a course outline does not show whether lessons are clear. Reviewing screenshots does not prove that a login works. A public refund policy does not demonstrate how a particular request will be handled. A desk review cannot measure performance, verify every seller claim, or predict an individual result.
What hands-on testing should describe
Hands-on testing starts with actual access to the product or service. A reviewer should say what was accessed, when, on which device or version, and for how long. For a digital course, relevant steps might include opening lessons, using downloads, testing captions, completing exercises, or contacting support. For software, the method might include installation, supported features, and the operating environment. The method should fit the claim being evaluated.
Testing should also be proportionate. A reviewer does not need to test every possible feature to provide useful observations, but the article should not let a narrow test sound comprehensive. A two-hour trial might reveal navigation and a sample lesson; it cannot establish whether a learner retains a skill over months. A few successful support messages cannot establish that response times are always fast. When outcomes depend on the learner’s time, prior knowledge, or practice, those factors should remain visible in the conclusion.
A credible test also says what was not tested. Trying one lesson does not verify the entire course. Testing one browser version does not establish compatibility with every device. Contacting support once cannot prove typical response time. A short trial cannot demonstrate long-term learning outcomes. Scope statements help prevent one observation from becoming a broad guarantee.
Reviewers should record relevant conditions, such as a test date, account tier, software version, region, network, or whether a sample account was supplied. If the seller provided access or another benefit, disclose that relationship. This helps readers judge whether the test resembles their own use.
Choose words that match the evidence
| Evidence available | Accurate wording | Wording to avoid |
|---|---|---|
| Seller’s public module list | “The seller lists these modules.” | “The course teaches these skills effectively.” |
| One sample lesson viewed | “We watched one sample lesson; it used these steps.” | “We tested the full course.” |
| Account and materials used for a stated period | “We tested these features on this device during this period.” | “It works for everyone.” |
| Customer testimonial on a sales page | “The seller presents this customer’s account.” | “Customers typically achieve this result.” |
| No direct product access | “This is a desk review based on listed sources.” | “In our experience using the product…” |
Why we do not imply testing through design or language
Words such as “review,” “tested,” and “hands-on” can lead a reader to assume direct use. So can photographs, screenshots, or first-person statements if they are not explained. A polished page does not make a desk review hands-on. We therefore identify the review method near the article’s beginning or in a clear methodology section, and we distinguish a seller’s description from our own observation.
We also avoid fabricated testimonials, invented ratings, and simulated test results. If a graphic is an editorial illustration rather than an actual product screenshot, it should be labeled accordingly. If the review uses a seller-supplied image, the caption can identify it as a seller-provided image; it should not be presented as proof that the reviewer personally used the product.
How sources support different kinds of statements
Primary sources are useful for current offer details. Independent sources can support general background. Direct testing can support observations about features actually tried. These evidence types should not be substituted for one another. A government page about subscription disclosures may explain general consumer guidance, but it does not confirm that a specific course’s renewal flow is easy to cancel.
For each important statement, a reviewer should ask: Who says this? When was it checked? What evidence would change the conclusion? If the answer is “the seller says so,” attribute the statement. If it is an observation, describe the test. If evidence is unavailable, label the uncertainty. This process keeps the review useful without overstating what research can establish.
When evidence changes
Digital products are updated, prices change, and sellers revise terms. A review should include a date checked for details that may change and should be revisited when a material update appears. A previous test of an older app version may not describe today’s interface. If a seller changes the product or policy, the article should distinguish the current statement from any historical observation.
Corrections should be made when a published statement is wrong or becomes misleading. For a material change—such as a new subscription requirement, removed feature, or revised refund procedure—the article should make the updated status clear. A reviewer should not silently preserve an outdated statement just because it once matched a source.
Actionable summary
- Look for a clear label explaining whether the review is desk-based or hands-on.
- Check the date, sources, access level, device, and testing period when testing is claimed.
- Read the limits: a sample or one device does not stand for every lesson or user.
- Notice whether seller claims are attributed and observations are described separately.
- Use the article as one evidence source and verify current price and terms before acting.
Good review writing tells readers what was examined and what was not. A desk review can still be valuable when it is specific, sourced, and candid about its limits. Hands-on testing adds direct observations only when the method is transparent and appropriate to the claim. Learn more in our How We Review and Editorial Standards sections.
Source: Federal Trade Commission, Consumer Reviews and Testimonials Rule Q&A, for U.S. guidance on specified deceptive review and testimonial practices.