liveBook · Manning

Tofupy

As explained in Terraform in Depth: Infrastructure as Code with Terraform and OpenTofu

tofupy is a Python library created in this book for programmatic interaction with Terraform or OpenTofu projects.

Suppose you need to check a Terraform or OpenTofu plan before a security-group change reaches deployment. tofupy is a Python library for working with those projects programmatically. Its Tofu client runs commands in a chosen working directory and makes their results available to Python code. section 11.1.10 section 11.1.2

Listing 11.2 Client usage
from tofupy import Tofu                                      #1
 
workspace = Tofu(                                            #2
    cwd="/path/to/terraform/workspace",                      #3
    log_level="DEBUG",                                       #4
    binary="terraform"                                       #5
)

Import Tofu and point it at a workspace. The client defaults to the tofu binary; this example selects terraform and requests DEBUG logging. section 11.1.2

The client accepts a working directory, binary choice, log level, and environment variables. It depends on the selected tofu or terraform executable being available and raises an error if it is not. Its command runner uses the selected directory and captures output; a CommandResults object holds the command, standard output, standard error, and return code. Helpers parse JSON or line-delimited JSON and report command failures. section 11.1.2

Listing 11.3 Command results
import json                                                   #1
from dataclasses import dataclass                             #1
from typing import Any, Generator                             #1
 
@dataclass                                                    #2
class CommandResults:
    command: str                                              #3
    stdout: str                                               #4
    stderr: str                                               #5
    returncode: int                                           #6
 
    def json(self) -> Dict[str, Any]:                         #7
        return json.loads(self.stdout)
 
    def jsonl(self) -> Generator[Dict[str, Any], None, None]: #8
        for line in self.stdout.splitlines():                 #9
            yield json.loads(line)                            #10
 
    def raise_error(self) -> None:                            #11
        try:
            for log in self.jsonl():                          #12
                if log.get("@level") == "error" and (
                    diagnostics := log.get("diagnostic")
                ):
                    raise RuntimeError(
                        f"Terraform command '{self.command}' failed: 
{diagnostics.get('summary')}\n{diagnostics.get('detail')}",
                        diagnostics,
                    )
        except Exception:                                     #13
            raise RuntimeError(
                f"Terraform command '{self.command}' failed: {self.stdout}."
            )
#1 We need to add a few new imports to use below.
#2 By turning this into a dataclass we allow Python to create methods like __init__ for us.
#3 We want to store the command to make it easier to generate useful error messages.
#4 The stdout is the most import piece, as it contains the output of the command.
#5 Most of the time we won’t need to use this, as Terraform logs to stdout in automation mode.
#6 For most commands, this records the success or failure of the command, but for Plan it has more meaning.
#7 This method will parse the stdout as JSON and return it.
#8 This method will parse the stdout as JSONL and return it.
#9 We split the stdout into lines, since JSONL is line delimited.
#10 We parse each line as JSON and yield it for use.
#11 This helper command allows us to raise an error with the command output.
#12 Some errors come in the form of JSONL, so we parse it to improve our messaging.
#13 If we can’t parse the logs as JSONL, we raise a generic error.

CommandResults shows the boundary between a command-line run and Python processing: keep the raw result, then parse structured output when needed. section 11.1.2

Flag an open ingress rule

A scanner creates Tofu for a target module, calls init(), then obtains a plan with plan(). It checks whether a plan exists or has errored before inspecting changes. Its rule looks for a security-group ingress change that would allow traffic from anywhere. The test module deliberately sets cidr_ipv4 to 0.0.0.0/0 on a TCP port 443 ingress rule. After installing typer and tofupy, run python scanner.py ./module; the scanner reports the open rule. section 11.1.10

Listing 11.31 Example failing module
data "aws_vpc" "default" {                                 #1
  default = true
}
 
resource "aws_security_group" "example" {                  #2
  name   = "example_group"
  vpc_id = data.aws_vpc.default.id
}
 
resource "aws_vpc_security_group_ingress_rule" "example" { #3
  security_group_id = aws_security_group.example.id
  cidr_ipv4         = "0.0.0.0/0"                          #4
  from_port         = 443
  ip_protocol       = "tcp"
  to_port           = 443
}

The deliberately failing module gives the scanner a concrete plan change to catch. section 11.1.10

Use tofupy when a Python tool needs more than a command’s exit status: the scanner uses plan changes to enforce a rule, and the same kind of introspection can support security or cost-control tools. Account for the local executable dependency when setting up the tool. section 11.1.10 section 11.1.2

How Terraform in Depth: Infrastructure as Code with Terraform and OpenTofu covers it

How deep it goes
Works through client initialization, command results, command execution, and a plan-based scanner.
Examples
Uses a configurable client, a deliberately open TCP ingress rule, and scanner output warning about internet-wide access.
What it assumes
Assumes an available tofu or terraform binary while introducing the client and command-result model.
What you can do after
Build it from scratch: create a Python plan scanner that warns about an open security-group ingress rule.
How it connects
Connects Python subprocess execution and JSON parsing to Terraform plans and security-group ingress rules.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Terraform in Depth: Infrastructure as Code with Terraform and OpenTofu. The section links point to the parts of the book it draws on.