Method

How we test and report

Two kinds of review, two methods

How a review is made

One real task, every product,
and the records to show for it.

Each review gives one task to several products a small team can buy, with frontier models alongside them, and reports what the recorded outputs did.

  1. TASK ABCDFrontier
    1Pick one real taskProducts a small team can buy, with frontier models alongside them.
  2. same brief, every run recorded
    2Run every product on itSame task and inputs; the sample is stated next to the results.
  3. SCOREDOUR GAP, NOT SCORED with outputno output = 0 unreachablenot tested
    3Sort each attemptEvery attempt gets one of four run states. Our gaps never count against a product.
  4. RULES 0100
    4Score against the recordsPer-task results first; a total only with its weights.
  5. analyst.ai 123 No sponsors UPDATED
    5Publish and keep it currentNo sponsors. Every correction and re-test is dated.

When a major new model ships, we run every product again from step 2 and date the change.

Every review states its sample, how outputs were judged and against which rules, and what it did not measure.

Run states

Every attempt is sorted into one state: tested with output, tested with no output, unreachable, or not tested. If a product was set up and given the task but produced no deliverable within the test window, that counts as a failure on that task and is scored 0, with its state shown (for example “output incomplete” or “still waiting”). Gaps on our side are never scored as product failures: a task we did not run, a product we could not reach, or records we do not have.

Strength of evidence

Each review states its sample next to its results: the brands, companies or questions tested and how many runs each product had.

Scores, composites and rankings

Per-task results are always shown next to any total. A composite appears only with its weights stated (the AI-visibility review weights questions 30%, diagnosis 30% and actions 40%), and every place is numbered; neighbours that a small scoring change would reorder are marked “could swap”. The marketing-copy review reports four questions separately, with no total.

Who scored

Each review says how its outputs were judged, against which records and rules, and when a rule was added after testing.

Updates and re-tests

We keep every review up to date. When a major new model is released, we re-run the tests and update the results; new evidence and corrections are added as they arrive. Each review lists every change with its date in its Updates section.

What we do not measure yet

Business outcomes (visibility, replies, conversions, sales), cost and staff time were not measured in these reviews. Each article says so next to its results.

Commercial relationships

analyst.ai has no commercial relationship with any product it reviews: no sponsorship, paid placement, affiliate links, or vendor-provided accounts or credits. Each review states this for its vendors.

Method and disclosures in each review

Get new reviews by email

New AI tool reviews and updates, sent when we publish.

How we use your email