Evidence and Data Requirements

Foreword. This Standard defines what evidence a VRS rating may rest on and how an assessor decides whether a given piece of evidence establishes a criterion. From the VRS 2026 edition evidence is judged by admissibility — does it meet the criterion’s evidence bar at the rating tier in force — and not by weight: the evidence-grade letters of earlier drafts survive as descriptive labels only, and their confidence points are withdrawn together with the confidence model they fed (VRS-GEN-005). New in this edition are the rules for recognised third-party evidence, for company-wide and null-result evidence, and — in Clause 11 — the kinds of public source an Unverified rating may use. Requirements use “shall”; recommendations use “should”.

Introduction

A rating is only as defensible as the evidence beneath it, and it is only adoptable if the evidence a manufacturer must supply is evidence it already produces for CE, UL or ISO conformity. This Standard fixes both ends. It states, for each rating tier, what counts: public material for an Unverified rating; the information pack for a Verified rating; the witnessed Demonstration Session for a Certified rating. And it maps each kind of evidence a criterion may ask for to the conformity artefacts manufacturers already hold, so that adoption is a mapping exercise rather than new testing. It aligns with the data-quality vocabulary (ISO 8000-2:2022; ISO 8000-8:2015), the provenance model (W3C PROV-O:2013) and the privacy framework (ISO/IEC 29100:2024), so that VRS evidence records interoperate with the tooling manufacturers, insurers and lenders already run.

1 Scope

This Standard defines: admissibility of evidence by rating tier (Clause 4); the evidence-record schema (Clause 5); recognised third-party evidence and the reuse of conformity artefacts (Clause 6); company-wide and null-result evidence (Clause 7); provenance, consent and retention (Clause 8); labelling (Clause 9); a worked example (Clause 10); and the public source classes admissible for Unverified ratings (Clause 11). It governs the evidence behind every criterion of VRS-GEN-101…107 and of the sector Assessment Protocols. It excludes scoring mechanics (VRS-GEN-005, VRS-GEN-006), the contents of information packs (VRS-GEN-013), demonstration protocols (VRS-GEN-203, VRS-<CODE>-110) and the registry record (VRS-GEN-009).

2 Normative references

• VRS-GEN-001, Vocabulary and Terminology (undated).
• VRS-GEN-002, Designation, Drafting and Style Rules (undated) — criterion layout §6.5; certificate, null-result and parent-evidence rules §7.5.
• VRS-GEN-005, Intrinsic Rating — Methodology (undated) — rating tiers (Clause 7), tier ceilings (Clause 6).
• VRS-GEN-009, Robot Risk Passport and Registry Schema (undated) — where records resolve; no-PII rule.
• VRS-GEN-204, Unverified Rating Procedure (undated) — collection period, notice and comment window.
• Dataset vocabulary vocabularies.json (dated to this edition) — the authoritative key list for source-type and evidence-record fields; keys are defined there once.
• ISO 8000-2:2022, Data quality — Part 2: Vocabulary.

3 Terms and definitions

For the purposes of this document the terms in VRS-GEN-001, VRS-GEN-002 §3.8–3.11 and ISO 8000-2:2022 apply, together with the following. Robot terms follow ISO 8373:2021.

• evidence record — one atomic, dated, source-attributed assertion about a robot model, its manufacturer or its fleet that is offered in support of a criterion, structured per Clause 5.
• admissible — of an evidence record: meeting Clause 4 for the rating tier in force and the criterion’s evidence bar; only admissible records may place a criterion on a rung.
• evidence grade — a descriptive label (A–D, 4.6) recording how adversarially a record was tested; a data-quality vocabulary, never a weight and never a rating.
• public source class — one of the kinds of publicly accessible source listed in Clause 11.
• information pack — the fixed list of documents a manufacturer supplies for a Verified or Certified rating (VRS-GEN-013), with an officer’s attestation.
• Demonstration Session record — the video, data-feed and step log produced by a witnessed demonstration under a demonstration protocol.
• certificate (publicly verifiable) — a certificate or listing whose existence, scope and validity can be confirmed in the issuing body’s own register or by certificate number on the issuing body’s site.
• null result — a recorded search of one or more public registers that found no entry for the model, stated with the registers, query, date and the registers’ coverage.
• company-wide evidence — evidence about the manufacturer’s product line or organisation, offered for a field-record criterion in place of model-specific evidence.
• lane — the physical processing separation between sellable rating evidence and never-sold intelligence material (8.4).

4 Admissibility by tier

4.1 Evidence establishes a criterion only if it is admissible at the rating tier in force:

4.2 Evidence bar. Each criterion states the minimum evidence for any points at all (VRS-GEN-002 §6.5.2). An assessor shall first decide whether admissible evidence meets the bar (yes/no); if not, the criterion scores 0 and no further judgement is made. If so, the assessor places the criterion on the highest rung the admissible evidence establishes, not exceeding the tier’s ceiling for that criterion.

4.3 No weighting. Admissibility is binary. No factor, multiplier or confidence point is derived from the kind, grade or source of evidence and applied to the points. Two admissible records establishing the same rung award the same points whatever their grade.

4.4 Strongest admissible record governs. Where admissible records conflict, the record that is more recent, more specific to the rated revision, and more independent of the manufacturer governs, in that order; the assessor shall record the conflict and the choice.

4.5 Applicability to the model and revision. A record establishes a criterion only for the model, hardware revision and software version range it demonstrably concerns. A renamed model with unchanged hardware and software is the same model; a change of controller or platform breaks applicability for the criteria that depend on it. For company-level criteria the record shall concern the manufacturer or, per 6.5, a parent whose system governs the rated product line.

4.6 Evidence grades as labels. Every record may carry a descriptive grade — A adversarial or regulator-filed · B independently verified · C plausible second-hand · D manufacturer-asserted — for audit and readability. Wherever a grade appears near a rating it shall be labelled “evidence grade” (VRS-GEN-002 §7.4.3). Grades never alter points.

5 Evidence record schema

5.1 Every evidence record shall carry at least the fields below. Field keys are defined once in the vocabularies.json twin; the names here map to that twin 1:1.

5.2 A record missing any shall field shall be inadmissible and shall not enter the dataset. Inadmissible records shall be logged with the missing field named, so the gap is auditable rather than silent.

5.3 Claimed-versus-measured comparisons (VRS-GEN-104) shall store the claim and the measurement as separate records linked by criterion reference, each with its own source and date.

6 Recognised third-party evidence and conformity-artefact reuse

6.1 Certificates and test reports. A certificate, listing or test report issued by an accredited laboratory, a notified body or a classification society that covers a criterion establishes that criterion at its certificate rung (VRS-GEN-002 §7.5.3):

• at Unverified, where the certificate is publicly verifiable, up to and including the criterion’s Unverified ceiling — the assessor shall not reduce the points merely because the evidence is a certificate rather than a report;
• at Verified, where the manufacturer supplies the report, at the Verified ceiling for the criteria the report’s scope covers;
• at Certified, only where the demonstration protocol names the certificate as an accepted equivalent to a specific step; otherwise the step is performed.

6.2 Scope governs. A certificate establishes only what its scope names. A certificate on a safety component (a scanner, a controller) is component evidence, not machine evidence; an expired or withdrawn certificate establishes nothing.

6.3 Statement is not certificate. A manufacturer’s statement that a product “is certified”, without a certificate number resolvable in the issuing body’s register, is documentation and is scored at the documentation rung of the criterion, not the certificate rung.

6.4 Three kinds of third party. Criteria that score third-party assessment distinguish an accredited testing laboratory (tests performed), a notified or approved body acting under a regulation (conformity reviewed), and a regulator’s own clearance or authorisation (dossier reviewed). Each is a different rung and a different fact; an assessor shall not treat them as interchangeable.

6.5 Parent-company evidence. For company-level criteria — quality system, vulnerability handling, going concern, warranty and support conduct — evidence held by the manufacturer’s parent or group is admissible only where the parent’s system demonstrably governs the rated product line: a quality certificate whose scope names the site or product; a security advisory process that issued this product’s advisories; insolvency or ownership events at the entity that owns the product line. Otherwise it is context and establishes nothing (VRS-GEN-002 §7.5.7).

6.6 Demonstration evidence — parity by protocol. Evidence produced in a Demonstration Session is graded by the protocol it was produced under, not by the venue or the route: a session witnessed remotely, a session at an accredited laboratory running the same protocol, and a session witnessed on site produce equally admissible evidence (VRS-GEN-005 §7).

6.7 Conformity-artefact reuse. For each kind of evidence a criterion may ask for, the assessor shall accept the existing conformity artefact named below where it is current and in scope, so that adoption costs a mapping and not new testing:

7 Company-wide and null-result evidence

7.1 Company-wide fallback. A field-record criterion (incidents, failures, availability, repairs, warranty conduct, recalls) accepts, in place of model-specific evidence, evidence about the manufacturer’s product line as a whole — at the reduced ceiling the criterion states (VRS-GEN-005 §6.5). The registry record shall show that the fallback was used. Company-wide evidence is never accepted for a criterion that does not state a company-wide ceiling.

7.2 Cold-start parity. A model with no fleet history is assessed as any other model: its field-record criteria are scored on company-wide evidence at the stated reduced ceilings, and no criterion requires a minimum fleet age or exposure as a precondition of scoring (VRS-GEN-005 §6.6).

7.3 Null results. For a criterion whose Unverified evidence is a search of public registers, a documented null result is evidence: the assessor shall record the registers searched, the query terms, the date and each register’s stated coverage (jurisdiction, product scope, reporting obligation). A null result scores the criterion’s null-result rung and never its top rung; where the registers searched do not cover the model’s sector or jurisdiction, the null result is noted but scores 0.

7.4 Exposure, where a criterion uses it. Where a rung is defined against an exposure quantity (a rate per operating hour, a fleet size, a window in months), the exposure shall be stated with the record and the rate expressed in the units the criterion names; an exposure that cannot be established leaves the criterion at the rung its other evidence supports.

8 Provenance, consent and retention

8.1 Every evidence record shall carry: source type (per the dataset vocabulary), capture date, and a consent/rights basis using ISO/IEC 29100:2024 terminology. A record without a lawful basis shall not enter the dataset.

8.2 Provenance shall be recorded as a chain of entity, activity and agent per the W3C PROV-O:2013 model, sufficient for a third party to retrace how the value was obtained.

8.3 No personal information shall be stored in rating evidence; operator identities are referenced by verification status, not identity (aligns the VRS-GEN-009 no-PII rule).

8.4 Lane discipline: evidence for sellable rating products and never-sold intelligence material shall be stored and processed in separate lanes (the two-lane rule; the ingest guard enforces the physical separation).

8.5 Records used for a published rating shall be retained for the life of the rating plus the period VRS-GEN-004 fixes for challenge and review; retention beyond the consent/rights basis shall not occur.

9 Labelling

9.1 Every published rating shall disclose its rating tier and edition (VRS-GEN-002 §7.4) and, in the registry record, the evidence records that placed each criterion on its rung (VRS-GEN-009).

9.2 Where the company-wide fallback (7.1) or a null result (7.3) determined a criterion’s points, the registry record shall say so against that criterion.

9.3 The edition change log shall publish the derivation of every prior changed in that edition (VRS-GEN-005 §8.4) — the standing defence against “arbitrary methodology” criticism.

10 Worked example

The following record set is illustrative and is not a Veyrum finding about any real robot.

A criterion of VRS-GEN-102 scores the model’s field failure rate (0–9, company-wide ceiling 5). Three assessments of fictional model X at three tiers:

Unverified. The assessor finds no operator disclosure of failure data and no editorial endurance test. A search of the workplace-safety and consumer-recall registers covering the model’s market finds no entry; the search is recorded with registers, query, date and coverage. The record is admissible (Clause 11, class 5); it places the criterion on the null-result rung (7.3) — 3 points — not on “record clean”.

Verified. The manufacturer’s pack contains an ISO 14224-style failure record for the product line (24,000 fleet hours, three failures) but none for model X, which has shipped for four months. The record is admissible (4.1) and is company-wide (7.1): the criterion is placed on the rung its rate supports, capped at the company-wide ceiling of 5. The registry record notes the fallback (9.2). The manufacturer’s separate claim of “MTBF 30,000 h” is retained as a claimed value under VRS-GEN-104 and establishes nothing here (6.3 by analogy: a claim is not a record).

Certified. The demonstration protocol names no step for this criterion (a field record cannot be witnessed), so the Certified points are those established at Verified (VRS-GEN-002 §6.5.2). Had the manufacturer instead supplied a third-party reliability test report to a named standard, 6.1 would have placed the adjacent reliability-test criterion at its certificate rung — a different criterion, scored on its own evidence.

11 Public evidence for Unverified ratings

11.1 An Unverified rating shall rely only on evidence from the following kinds of public source:

1. the manufacturer’s own published technical documentation — datasheets and specifications, user and service manuals, safety-function tables, release notes, support-horizon pages, security and privacy documentation, warranty, service and licence terms;
2. declarations of conformity published by the manufacturer;
3. certificate registers of accredited laboratories, notified bodies and classification societies, and published class marks;
4. regulators’ clearance, approval and authorisation records;
5. regulators’ incident, recall and adverse-event databases;
6. vulnerability databases and the manufacturer’s published security advisories and disclosure policy;
7. independent editorial or laboratory tests that publish their method;
8. peer-reviewed measurements of the named model;
9. quality-system certificate registers;
10. court, insolvency, enforcement and securities filings, and regulated financial reporting of them;
11. secondary-market listings for the model;
12. published disclosures by named operators or integrators.

Veyrum maintains the register of sources within these kinds, their coverage and their collection cadence.

11.2 Public evidence shall be admissible only where it is: accessible without contract, login or payment at the time of collection; attributable to an identifiable publisher; dated or datable; archived by Veyrum at collection with URL and retrieval time; and applicable to the rated model and revision or — for company-level criteria — to the manufacturer or a parent whose scope demonstrably covers the model (4.5, 6.5). Press reports, forums, marketplaces’ user reviews and unsourced aggregators are not evidence; they may lead the assessor to evidence. Mirrored copies of manufacturer documents are admissible only where the document carries the manufacturer’s identity, model and version and the manufacturer’s own copy could not be located.

11.3 A certificate is evidence under 11.1(3) or (9) only where it can be verified in the issuing body’s register or by certificate number on the issuing body’s site (6.3).

11.4 A search of a regulator’s database that finds nothing shall be recorded per 7.3 and scores the criterion’s null-result rung; it shall not score the top rung.

11.5 Public evidence never lifts a criterion above its Unverified ceiling. Within the ceiling the rung is determined by what the evidence shows; the kind of source neither adds to nor subtracts from the points.

11.6 Where a sector is subject to a mandatory public reporting regime for incidents, recalls or occurrences, the sector protocol may set the Unverified ceiling of the corresponding field-record criteria at their full value and shall name the regime by kind (VRS-GEN-002 §7.5.5).

11.7 Manufacturer publications about what users should do, white papers and marketing establish nothing about the rated product; only statements about the product, its revision and its record do (VRS-GEN-005 §6.10).

NOTE — The Unverified Rating Procedure (VRS-GEN-204) states the collection period, the manufacturer notice and the comment window. This clause governs only what may be counted.

Bibliography

• ISO 8000-2:2022, Data quality — Part 2: Vocabulary (edition 5).
• ISO 8000-8:2015, Data quality — Part 8: Information and data quality: Concepts and measuring.
• ISO/IEC 25012:2008, Software engineering — SQuaRE — Data quality model.
• ISO/IEC 17000:2020, Conformity assessment — Vocabulary and general principles — the certification / attestation vocabulary of Clause 6.
• ISO/IEC 17025:2017, General requirements for the competence of testing and calibration laboratories — what “accredited laboratory” means in 6.1 and 6.4.
• ISO/IEC 29147:2018 and ISO/IEC 30111:2019, Vulnerability disclosure and Vulnerability handling processes — the disclosure-process artefacts of 6.7.
• ISO/IEC 29100:2024, Information technology — Security techniques — Privacy framework (edition 2).
• ISO 14224:2016, Collection and exchange of reliability and maintenance data for equipment.
• ISO 8373:2021, Robotics — Vocabulary.
• W3C PROV-O, The PROV Ontology, W3C Recommendation, 30 April 2013.

Change history