liveBook · Manning

Test data generation

As explained in Software Testing with Generative AI

Test data generation is the creation of input data sets for exercising software during testing activities.

Test data generation creates input data sets for testing. LLMs are most useful when the task is specific and the prompt supplies explicit rules, formats, relationships, and examples. Generated data can support exploratory, automated, and performance testing, but testers must verify correctness, risks, and privacy before use. section 3.3.1 section 6.4

A practical generation workflow includes:

  • Choose a narrow task, such as producing JSON objects, SQL inserts, or seed-file records. section 3.3.1 section 8.2.2
  • Provide a machine-readable structure when possible. OpenAPI 3.0 and XSD communicate fields, types, enumerations, and constraints without recreating those rules manually. section 6.2.1
  • State output instructions, delimit the source structure, require validation, and add special rules for unusual types such as SQL arrays. section 6.2.1 section 8.2.2
  • Preserve relationships by supplying the relevant schema or SQL rather than isolated insert statements; otherwise foreign keys may reference nonexistent records. section 6.2.2
  • Review generated values against the application, database constraints, test purpose, organizational policy, and user-data privacy requirements. section 6.2.2 section 6.4

Schema-driven room records

A focused prompt can ask an LLM to generate three randomized JSON objects from a delimited OpenAPI schema and confirm that every object matches its rules before output. The schema constrains fields such as accessibility, room type, features, image, room number, and price; the resulting array supplies concrete records for testing. If an image field needs URLs rather than arbitrary strings, revise the schema and prompt to express that constraint, then validate the response independently. section 6.2.1

###
CREATE TABLE rooms (roomid int NOT NULL AUTO_INCREMENT, room_name
➥varchar(255), type varchar(255), beds int, accessible boolean,
➥image varchar(2000), description varchar(2000),
➥features varchar(100) ARRAY, roomPrice int, primary key (roomid));
 
INSERT INTO rooms (room_name, type, beds, accessible, image,
➥description, features, roomPrice) VALUES ('101', 'single', 1, 
➥true, 'https://www.mwtestconsultancy.co.uk/img/testim/room2.jpg',
➥'Aenean porttitor mauris sit amet lacinia molestie. In posuere
➥accumsan aliquet. Maecenas sit amet nisl massa. Interdum et
➥malesuada fames ac ante.', ARRAY['TV', 'WiFi', 'Safe'], 100);
 
CREATE TABLE bookings (bookingid int NOT NULL AUTO_INCREMENT, roomid
➥int, firstname varchar(255), lastname varchar(255),
➥depositpaid boolean, checkin date, checkout date,
➥primary key (bookingid), FOREIGN KEY (roomid) REFERENCES
➥ROOMS(roomid));
 
INSERT INTO bookings (roomid, firstname, lastname, depositpaid,
➥checkin, checkout) VALUES (1, 'James', 'Dean', true, '2022-02-01',
➥'2022-02-05');
###

Inspect the related rooms and bookings schema, including the foreign-key relationship that a generator must preserve when creating connected test data. section 6.2.2

How Software Testing with Generative AI covers it

How deep it goes
Descends from prompt structure through schema and relationship handling, but stops short of trusting generated data without verification.
Examples
Demonstrates JSON rooms from OpenAPI, related rooms and bookings from SQL, and large booking seed sets for exploratory sessions.
What it assumes
Assumes access to data structures or specifications and teaches prompt rules for formats, relationships, validation, and privacy constraints.
What you can do after
Apply it via LLM prompts to produce structured or relational test data, then verify validity, usefulness, and privacy.
How it connects
Connects LLMs, prompt engineering, OpenAPI, XSD, SQL, exploratory testing, performance testing, and data privacy.

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Software Testing with Generative AI. The section links point to the parts of the book it draws on.