Gradio turns a machine-learning function into an interactive web interface with little or no frontend code. It is useful for quickly testing chatbot behavior, collecting user feedback, and sharing data-science demos from a familiar Python or Jupyter environment. section 8.1.3
A Gradio chatbot can provide:
- A ChatInterface connected to a function that makes an API request. section 8.1.3
- Streaming responses when the generation function is implemented as a generator. section 8.1.3
- Built-in interaction features such as Stop, Retry, and Undo. section 8.1.3
Listing 8.3 Local LLM Gradio chat app with Stop, Retry, and Undo
import gradio as gr
import requests
import json
url = "http://localhost:8000/generate" #1
def generate(message, history):
history_transformer_format = history + [[message, ""]]
messages = "".join(
[
"".join(["\n<human>:" + h, "\n<bot>:" + b])
for h, b in history_transformer_format
]
)
data = json.dumps({"prompt": messages})
full_response = ""
with requests.post(url, data=data, stream=True) as r: #2
for line in r.iter_lines(decode_unicode=True):
full_response += line.decode("utf-8")
yield full_response + "▌" #3
yield full_response
gr.ChatInterface(generate, theme="soft").queue().launch()
The listing shows a compact local-LLM chat app that transforms conversation history, streams an HTTP response, and launches a themed interface. section 8.1.3
