Validation checks data against expected formats and constraints before it is processed. In an AI application, start with a threat model: identify public endpoints, user-input points, and sensitive data. Input checks can reduce the risk of malicious data reaching a model and help limit excessive data usage, but they are only one layer of security. section 9.1.1 section 9.7
Put checks at the boundary they protect:
- Validate submitted input on the server, even when the client also checks it. Client-side limits improve the experience but can be bypassed. section 9.1.1
- For a prompt endpoint, require a nonempty string and impose a length limit before passing the prompt to processing logic. section 9.1.1
- For document uploads, check allowed file types and sizes on both client and server before resource-intensive parsing and chunking. section 11.2.2
Listing 9.1 Validating incoming requests using Zod
import { z } from 'zod'; #1
const promptSchema = z.object({ #2
prompt: z.string().min(1).max(1000),
});
export default async function POST(req, res) {
try {
const validatedData =
promptSchema.parse(req.body); #3
const response =
await process(validatedData.prompt); #4
res.status(200).json({ response });
} catch (error) {
return res.status(400).json({ error: error.errors });
}
}
The Zod request-handler example defines a prompt as a string of 1–1,000 characters, parses the request body, and passes the validated prompt to processing logic. section 9.1.1
Validation also applies to generated output. A test can check word-count bounds and required keywords; semantic similarity is a separate check for meaning. Mocked model responses make these tests faster and more controlled, but cannot reveal every issue in real model behavior. Production response checks for relevance, coherence, and safety are identified as further work rather than implemented in the initial backend. section 5.1.3 section 2.5.5
Listing 5.4 Testing constraints
function validateConstraints(text, minWords, maxWords, requiredWords) { #1
const words = text.split(/\s+/);
return (
words.length >= minWords &&
words.length <= maxWords &&
requiredWords.every((word) =>
text.toLowerCase().includes(word.toLowerCase())
)
);
}
test("generated text meets constraints", async () => {
generateText.mockResolvedValue({
text: "Regular exercise significantly improves
health and fitness by boosting energy, enhancing mood,
and reducing the risk of many chronic diseases."
});
const { text } = await generateText({
model: "fakeModel",
prompt:
"Summarize the benefits of exercise in 20-30 words.
Include 'health' and 'fitness'.",#2
});
console.log("Word count:", text.split(/\s+/).length);
expect(validateConstraints(text, 20, 30, ["health", "fitness"])).toBe(true);#3
});
The constraint test counts words and checks for required terms in a mocked generated response, illustrating an assertion that goes beyond an exact-text match. section 5.1.3
