liveBook · Manning

Chapter 4: Evaluating external technologies

What you will learn in chapter 4 of Decision Engineering

The introduction explains why adopting external technology deserves a deliberate decision grounded in the organization’s needs and proportionate to the commitment.

After this chapter you will be able to

What it builds on

Section by section

Introduction

The introduction explains why adopting external technology deserves a deliberate decision grounded in the organization’s needs and proportionate to the commitment.

Key terms: digital asset manager, problem, context, accidental adoptions, external technology, dependency stack, commitment, calibrated to the stakes, reversible adoptions, multi-year platform commitments

4.1 Qualifying candidates

The section frames external-technology selection as a hiring-like funnel: source possible candidates, then screen out clear mismatches before deeper evaluation.

Key terms: qualifying candidates, source, screen, decisive requirement

4.1.1 Candidates sourcing

The section shows how to source and triage technology candidates using evidence of real-world adoption, curated trend reports, and analyst reports without treating rankings as decisions.

Key terms: fit in practice, track record, independent practitioner voices, InfoQ Trend Report, ThoughtWorks Technology Radar, adoption stages, curated trend reports, Adopt, Trial, Assess

4.1.2 Screening against key requirements

This section shows how to narrow a technology shortlist by testing candidates against decision-specific requirements and context before investing in closer evaluation.

Key terms: screening phase, rejection filter, architecturally significant requirements, ASRs, genuine gap, ASR, nice-to-have, framing phase, service tiers, availability

4.2 Running closer evaluation

This section explains why a technology that passes initial screening needs closer evaluation through public developer experience, production users, and a focused proof of concept.

Key terms: first screening, closer evaluation, Voice of Developer analysis, production, proof of concept

4.2.1 Voice of Developer analysis

This section explains how Voice of Developer analysis uses public developer experience to evaluate external technologies beyond controlled tests.

Key terms: Voice of Developer (VoD) analysis, developer feedback, lived experience, GitHub issues, community discussions, long-form experience reports, patterns, independent posts, open issues, discussion volume

4.2.2 Learning directly from people been there

This section explains how direct conversations with people who have used a technology in production can improve an evaluation, and how to seek those conversations responsibly.

Key terms: one-to-one conversation, production use, VoD research, soft signals, existing network, specific question, small asks, one-way-door choices

4.2.3 Getting own first hand experience

The section explains how to use focused, hands-on proofs of concept to evaluate external technologies without mistaking a quick demo for sufficient evidence or production-ready code.

Key terms: proof of concept (PoC), hands-on time, learning curve, agentic AI tools, working prototype, comparable base, make-or-break questions, key use case, features of questionable maturity, VoD research

4.3 Open-source technologies evaluation

The section explains why open-source adoption carries responsibilities despite having no direct purchase price, and why evaluation effort should reflect the dependency’s impact.

Key terms: open-source adoption, long-term dependency on maintainers, SLA, level of diligence, dependency, commitment, ecosystem norms, calibrate

4.3.1 Assessing usage and popularity

This section shows how to compare open-source candidates using signals of interest and actual adoption without treating popularity as a verdict.

Key terms: GitHub stars, downloads, usage indicators, adoption, proven projects, rising star, early or niche projects, invisible infrastructure

4.3.2 Community health and maintainability signals

The section explains how to assess an open-source dependency’s maintainability, community health, and governance rather than relying on popularity alone.

Key terms: maintainability, popularity, active maintainers, releases, issues, community health, responsive community, governance model, bus factor, single-maintainer projects

4.3.3 Understanding limitations of the free version

This section explains why evaluating an open-core technology means accounting for paid features that production may require, even when the free version works for a prototype.

Key terms: open-core, free tier, paid tiers, intended scale, risk profile

4.3.4 Understanding licences

Explains when open-source licence review can be a quick check and when distribution, network use, customer restrictions, or transitive dependencies require closer evaluation.

Key terms: permissive licences, licence risk, licence review, distribution, copyleft, GPL family, process boundaries, AGPL, corresponding source code, SaaS

4.3.5 Calibrating for ecosystem differences

This section explains how to interpret open-source evaluation signals in light of ecosystem norms and why engaging with upstream projects helps keep dependencies viable.

Key terms: ecosystem's norms, social metrics, community health, maintainer availability, migration, engage upstream, private fork, local workarounds, sponsorship, dependency health

4.4 Commercial solutions evaluation

This introduction explains why commercial software needs a different evaluation mindset from open-source software: less information is visible upfront, and the purchase begins an ongoing vendor relationship.

Key terms: open source, commercial vendors, commercial incentives, signals

4.4.1 Dealing with limited information

This section explains how limited access to information changes commercial technology evaluation and how to close the gaps through vendor questions, customer references, and representative trials.

Key terms: information asymmetry, open source, self-serve products, product-led products, sales-led products, information-control posture, architectural constraints, data ownership, operational limits, reference customers

4.4.2 Pricing and negotiation dynamics

This section explains why commercial software pricing must be evaluated at both the initial purchase and renewal, and how to negotiate without being caught unprepared.

Key terms: initial negotiation, list prices, sales-led products, renewal, price-increase cap, dependency, multiple vendors, renewal scenarios, year-one cost, vendor negotiations

4.4.3 Including runner-up products

The section explains why vendor behavior during evaluation and the value offered by runner-up products should influence commercial technology choices.

Key terms: evaluation process, customer relationship, strategically valuable, marginal account, runner-up vendors, market leaders, serious challenger, total value, taking less for less

4.4.4 Assessing product continuity risks

This section explains how to assess the business-driven continuity risks of commercial technologies and plan for the dependency they create.

Key terms: product continuity risks, vendor, incentives, vendor sustainability, acquisitions, product retirement, pricing, licensing, alignment with core business, portfolio fit after an acquisition

4.4.5 What switching to a different technology could cost us

This section explains how to assess the cost of leaving a commercial technology before comparing candidates.

Key terms: exit cost, contractual restrictions, widely adopted standards, proprietary, vendor's SDK, open-source core, self-host, data ownership and portability, export, migration path

4.4.6 Evaluating contract conditions

Contract terms are part of evaluating a commercial technology because they can change its cost, risks, and suitability even when the product itself looks strong.

Key terms: draft contract, contract conditions, technical selection, contract duration and exit clauses, auto-renewal, price increase mechanisms, additional usage, liability clauses, risk allocation, intellectual property claims

4.4.7 Running the tracks in parallel

The section explains why technical, commercial, and legal evaluation should run in parallel and how to calibrate scrutiny of a commercial vendor.

Key terms: technical assessment, commercial track, legal and procurement, parallel tracks, commercial adoption, commercial incentives, sales cycle, exit cost

4.5 Summary

Concepts explained in this chapter

← Chapter 3: Reading the context

These guides are generated from the text of Manning books by Manning's knowledge graph and link to the sections they draw on. The full books are available on liveBook and at manning.com.