A span records one operation within a distributed trace. A trace follows a request across services; its spans separate work such as saving an order, calling Payment, and querying a database. A shared trace ID lets you find the request’s journey in Jaeger. section 9.1.1
A span’s metadata identifies the operation. For a gRPC server span, rpc.service and rpc.method name the endpoint, rpc.system identifies gRPC, and span.kind distinguishes server-side from client-side instrumentation. Inspecting these fields helps locate which service handled a request. section 9.1.1
To capture spans across the Order and Payment services:
- Configure an OpenTelemetry tracing provider with a Jaeger collector endpoint and set trace-context propagation so the Payment call carries the existing trace metadata. section 9.3.5
- Add the otelgrpc unary server interceptor to the gRPC server, then inspect traces in the Jaeger UI. section 9.3.5
import (
"go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
)
...
grpcServer := grpc.NewServer(
grpc.UnaryInterceptor(otelgrpc.UnaryServerInterceptor()),
)
...
The Go server configuration shows where the OpenTelemetry gRPC interceptor is installed. section 9.3.5
Finding time spent in a create-order request
A create-order trace takes 4.2 ms. Its Payment Create operation takes 3.36 ms, including a 2.88 ms database operation. Opening the spans exposes operation metadata; the database span can also show the SQL statement, giving a starting point for investigating query performance. section 9.3.6
