liveBook · Manning

External provider

As explained in Terraform in Depth: Infrastructure as Code with Terraform and OpenTofu

The Terraform external provider is a provider that obtains data by executing an external program and exposes the program's output as a data source.

Need a value that Terraform cannot calculate or retrieve through an existing data source? The external provider lets a local program supply that value. Unlike a provisioner, which runs commands around resource creation or destruction, it brings data back into Terraform. Use it as a last resort: the script and everything needed to run it become part of your delivery environment. section 10.4

The provider has one data source, `external`, and no infrastructure resources:

  • Set `program` to an array containing the command followed by its arguments. This is the only required argument. section 10.4.1
  • Optionally set `query` to a map of strings. Terraform sends its JSON representation to the program on standard input. Set `working_dir` if the program should run somewhere other than the current working directory. section 10.4.1
  • Write a JSON object to standard output whose values can become strings. Terraform exposes that map as `data.external.NAME.result`, so an individual value can be selected by key. section 10.4.1 section 10.4.3

Calculate square roots

For one fixed input, a Bash script can take `128` as a program argument and print a JSON object such as `{"value":"11"}`; Terraform reads `result["value"]`. For multiple inputs, a Python program can instead parse the `query` JSON from standard input, calculate each square root, and return a JSON object with matching keys and string values. section 10.4.3

Listing 10.23 Square root query data source
data "external" "query" {
  program = ["python", "${path.module}/square_root.py"]
 
  query = {                                                  #1
    "0"      = "128"                                         #2
    "1"      = "256"                                         #2
    "2"      = "512"                                         #2
    "random" = "1024"                                        #2
  }
}
 
output "random" {
  value = data.external.query.result["random"]
}

The configuration passes several named inputs through `query` and selects one returned value with `result["random"]`. section 10.4.3

Listing 10.24 Square root query with Python
import json                                                  #1
import math                                                  #1
import sys                                                   #1
 
data = json.load(sys.stdin)                                  #2
 
return_data = {}
 
for key, value in data.items():                              #3
    return_data[key] = str(math.sqrt(float(value)))          #4
 
json.dump(return_data, sys.stdout)                           #5

The matching Python program reads JSON from standard input and writes the string-valued result object to standard output. section 10.4.3

Prefer an existing data source or provider function when one fits. For a web-service query, consider the HTTP provider; for a larger Go-based integration, consider a custom provider. If an external program remains necessary, choose its language with deployment in mind: Bash can keep a small script simple, while higher-level languages may add dependencies that make a module harder to reuse. section 10.4.4 section 10.4.2

How Terraform in Depth: Infrastructure as Code with Terraform and OpenTofu covers it

How deep it goes
Works through the data-source arguments and JSON input/output contract, then implements single- and multiple-input scripts without exploring provider internals.
Examples
Uses Bash and Python square-root scripts, including a Terraform query with several named inputs.
What it assumes
Assumes basic Terraform configuration and script execution while explaining the data source’s arguments and JSON exchange.
What you can do after
Build it from scratch: configure an `external` data source, write a JSON-speaking program, and read its result by key.
How it connects
Contrasts external data sources with provisioners, the HTTP provider, provider functions, and custom providers.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Terraform in Depth: Infrastructure as Code with Terraform and OpenTofu. The section links point to the parts of the book it draws on.