Skip to content

ADR-017 — Vendor Assessment Requires Evidence-Based Workshop

Status

Accepted

Context

UC-BoK v0.5–v1.1 included a catalogs/vendor-mapping.yaml file with placeholder vendor entries (Comerzzia, Generic ERP) mapped to UC-BoK capabilities, services, and applications. These mappings were created without verified evidence — they were based on publicly available marketing materials and UC-BoK team assumptions.

This creates a structural problem: UC-BoK's core value proposition is vendor-neutral, evidence-driven knowledge. Publishing unverified vendor capability claims directly contradicts this principle and risks misleading practitioners who use UC-BoK for platform selection or gap analysis.

Additionally, real vendor capability assessment is complex:

  • The same platform can behave very differently depending on configuration, version, and integration depth.
  • Capability coverage claims made by vendors in slide decks or datasheets are frequently inconsistent with what actually works in a live deployment.
  • Only a System Integrator or certified assessor who has deployed and tested a solution against UC-BoK canonical scenarios can produce a reliable vendor profile.

Decision

No vendor capability profile shall be included in the UC-BoK canonical repository unless it has been produced through a structured assessment workshop following the UC-BoK Vendor Assessment Protocol.

The Vendor Assessment Protocol requires:

  1. A qualified System Integrator (SI) or certified assessor conducts the assessment.
  2. Each capability claim is backed by a verifiable test scenario drawn from the UC-BoK Process Catalog.
  3. Scores are assigned using the UCMI maturity rubric (0–4), not binary coverage flags.
  4. The assessment report is peer-reviewed by at least one independent UC-BoK assessor.
  5. The resulting vendor profile is published as a separate, versioned document — not as a core catalog entry.

Consequences

  • catalogs/vendor-mapping.yaml is reset to an empty framework skeleton. The Comerzzia and Generic ERP placeholder entries are removed.
  • The generated page docs/generated/catalogs/vendor-mapping.md is updated to reflect zero vendor profiles.
  • The docs/11-vendor-mapping/1100-vendor-coverage.md page is replaced with the Vendor Assessment Protocol.
  • The nav entry "Vendor Catalog" pointing to the generated placeholder is removed.
  • Future vendor profiles, once produced through the Protocol, will live in catalogs/vendor-profiles/<vendor-id>/ as self-contained assessment documents.

Positive Consequences

  • UC-BoK maintains its vendor-neutral integrity.
  • Practitioners can trust that any vendor profile in the repository is evidence-based.
  • The Protocol creates a clear, auditable path for SIs and vendors to contribute verified assessments.

Negative Consequences

  • No vendor profiles are available in the repository until the first workshop is completed.
  • This is acceptable: an empty, correct model is better than a populated, incorrect one.