JSONL stores independent JSON records one per line. For fine-tuning chat models, each record can hold a messages array that defines a system instruction, user prompt, and assistant completion. section 7.4.1
A chat-training record
A record can instruct a data analyst to address the general public, provide household-chore percentages as the user input, and supply a plain-language commentary as the assistant output. section 7.4.1
Listing 7.1 The structure of a JSONL file
{
"messages": [
{
"role": "system",
"content": "You are a data analyst showing data to the general public."
},
{
"role": "user",
"content": "Distribution of household chores
(Cooking 35% Cleaning 30% Laundry 20%, Yard work 15%)"
},
{
"role": "assistant",
"content": "Cooking takes up the largest portion at 35%.
Cleaning follows at 30% while laundry
and yard work accounts for 20% and 15% respectively."
}
]
}
Inspect the message roles and fields used in a JSONL training record. section 7.4.1
To create the file from CSV data, load prompt and completion columns into a pandas DataFrame, build one messages object per row, then write each object with json.dump followed by a newline. section 7.4.1
Listing 7.2 How to convert the CSV file into JSONL
import pandas as pd
import json
df = pd.read_csv('general-public.csv')
json_list = []
for index, row in df.iterrows():
json_object = {
"messages": [
{
"role": "system",
"content": "You are a data analyst showing data to the general public."
},
{
"role": "user",
"content": row['prompt']
},
{
"role": "assistant",
"content": row['completion']
}
]
}
json_list.append(json_object)
with open('general-public.jsonl', 'w') as outfile:
for json_object in json_list:
json.dump(json_object, outfile)
outfile.write('\n')
Follow the Python loop that converts each CSV row into a newline-delimited JSON object. section 7.4.1
