Structllm – structured output support to any LLM provider

1 day ago 1

structllm is a universal and lightweight Python library that provides Structured Outputs functionality for any LLM provider (OpenAI, Anthropic, Mistral, local models, etc.), not just OpenAI. It guarantees that LLM responses conform to your provided JSON schema using Pydantic models.

If your LLM model has 7B parameters or more, it can be used with structllm.

Or using uv (recommended):

from pydantic import BaseModel from structllm import StructLLM from typing import List class CalendarEvent(BaseModel): name: str date: str participants: List[str] client = StructLLM( api_base="https://openrouter.ai/api/v1", api_key="sk-or-v1-...", ) messages = [ {"role": "system", "content": "Extract the event information."}, {"role": "user", "content": "Alice and Bob are going to a science fair on Friday."}, ] response = client.parse( model="openrouter/moonshotai/kimi-k2", messages=messages, response_format=CalendarEvent, ) if response.output_parsed: print(response.output_parsed) # {"name": "science fair", "date": "Friday", "participants": ["Alice", "Bob"]} else: print("Failed to parse structured output")

StructLLM works with 100+ LLM providers through LiteLLM. Check the LiteLLM documentation for the full list of supported providers.

from pydantic import BaseModel, Field from typing import List, Optional from enum import Enum class Priority(str, Enum): LOW = "low" MEDIUM = "medium" HIGH = "high" class Task(BaseModel): title: str = Field(description="The task title") description: Optional[str] = Field(default=None, description="Task description") priority: Priority = Field(description="Task priority level") assignees: List[str] = Field(description="List of assigned people") due_date: Optional[str] = Field(default=None, description="Due date in YYYY-MM-DD format") client = StructLLM( api_base="https://openrouter.ai/api/v1", api_key="sk-or-v1-...", ) response = client.parse( model="gpt-4o-2024-08-06", messages=[ { "role": "user", "content": "Create a high-priority task for John and Sarah to review the quarterly report by next Friday." } ], response_format=Task, ) task = response.output_parsed print(f"Task: {task.title}") print(f"Priority: {task.priority}") print(f"Assignees: {task.assignees}")

response = client.parse( model="gpt-4o-2024-08-06", messages=messages, response_format=CalendarEvent, ) if response.output_parsed: # Successfully parsed event = response.output_parsed print(f"Parsed event: {event}") else: # Parsing failed, but raw response is available print("Failed to parse structured output") print(f"Raw response: {response.raw_response.choices[0].message.content}")

client = StructLLM( api_base="https://api.custom-provider.com/v1", api_key="your-api-key" ) response = client.parse( model="custom/model-name", messages=messages, response_format=YourModel, temperature=0.1, top_p=0.1, max_tokens=1000, # Any additional parameters supported by the LiteLLM interface custom_parameter="value" )

StructLLM uses prompt engineering to ensure structured outputs:

Schema Injection: Automatically injects your Pydantic model's JSON schema into the system prompt
Format Instructions: Adds specific instructions for JSON-only responses
Intelligent Parsing: Extracts JSON from responses even when wrapped in additional text
Validation: Uses Pydantic for robust type checking and validation
Fallback Handling: Gracefully handles parsing failures while preserving raw responses

By default it uses low temperature and top_p settings to ensure consistent outputs, but you can customize these parameters as needed.

Run the test suite:

# Install dependencies uv sync # Run tests uv run pytest uv run pytest -m "not integration" # Run integration tests (requires external services) uv run pytest -m "integration" # Run linting uv run ruff check .

Contributions are welcome! Please feel free to submit a Pull Request.