liveBook · Manning

Evaluation

As explained in Decision Engineering

The decision activity of considering alternatives and converging on a choice.

Evaluation is the part of a technical decision where the option space widens before narrowing to a choice. It follows framing: first clarify the problem, requirements, and context, then compare plausible ways forward. A choice is not the finish line; it still has to be put into practice, and failed prototypes or new constraints may require another pass. section 1.3

Figure 1.2 The decision lifecycle: framing and making the decision stick each move through mandatory steps, while evaluation widens the option space before narrowing it to a single choice

The lifecycle diagram shows evaluation widening the option space before converging, between framing and making the decision stick. section 1.3

For an external technology choice, scale the effort to the stakes:

  • Define the job the technology must do. Source candidates from curated trend reports and research, but adjust general maturity signals to the organization’s risk appetite. section 4.5
  • Screen candidates against decisive architectural requirements, service tiers, compliance constraints, and system, team, and company context. Drop clear mismatches before deeper investigation. section 4.5
  • Investigate the survivors through public community experience, conversations with production users, and focused proofs of concept. Decide what each prototype must demonstrate before building it. section 4.5 section 4.2.3
  • For open source, examine maintainability, community health, governance, adoption, licence fit, and free-version limits. For commercial products, investigate pricing, renewal terms, data handling, product continuity, exit costs, and contracts—not just technical fit. section 4.5 section 4.4

Testing an editor rather than its feature list

A visual editor may satisfy every written requirement yet feel difficult to use. A short trial can expose that difference and show how long a teammate takes to make a working change. If several products remain plausible, build comparable prototypes around the same use case; a generated “Hello world” proves little unless it exercises a make-or-break concern. section 4.2.3

Evaluation should reduce uncertainty without becoming a rigid ritual. Cheap prototypes can broaden hands-on comparison, but they do not replace a question worth testing. Commercial candidates also require early attention to financial and legal questions: discovering a renewal increase or data constraint after choosing a vendor weakens the decision. section 4.2.3 section 4.4

How Decision Engineering covers it

How deep it goes
Provides a staged external-technology evaluation procedure, from screening to focused trials and commercial checks, rather than a universal scoring formula.
Examples
Uses a visual-editor trial, comparable technology prototypes, and vendor pricing or data questions to expose gaps in paper comparisons.
What it assumes
Assumes an engineering decision can be framed in terms of requirements and organizational context; teaches how to screen and investigate candidates.
What you can do after
Practice/perform a stakes-calibrated technology comparison that tests decisive concerns before committing to a candidate.
How it connects
Connects framing and decision execution with requirements screening, proofs of concept, open-source governance, and vendor contracts.

Also explained in

Compare how 29 books explain evaluation →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Decision Engineering. The section links point to the parts of the book it draws on.