Create a configuration file at config/initializers/raif.rb
Copy Raif's database migrations to your application
Mount Raif's engine at /raif in your application's config/routes.rb file
Run the migrations. Raif is compatible with both PostgreSQL and MySQL databases.
If you plan to use the conversations feature or Raif's web admin, configure authentication and authorization for Raif's controllers in config/initializers/raif.rb:
Raif.configuredo |config|
# Configure who can access non-admin controllers# For example, to allow all logged in users:config.authorize_controller_action=->{current_user.present?}# Configure who can access admin controllers# For example, to allow users with admin privileges:config.authorize_admin_controller_action=->{current_user&.admin?}end
Configure your LLM providers. You'll need at least one of:
See Adding LLM Models for more information on adding new OpenRouter models.
When using Raif, it's often useful to use one of the higher level abstractions in your application. But when needed, you can utilize Raif::Llm to chat with the model directly. All calls to the LLM will create and return a Raif::ModelCompletion record, providing you a log of all interactions with the LLM which can be viewed in the web admin.
Call Raif::Llm#chat with either a message string or messages array.:
llm=Raif.llm(:open_ai_gpt_4o)# will return a Raif::Llm instancemodel_completion=llm.chat(message: "Hello")putsmodel_completion.raw_response# => "Hello! How can I assist you today?"
The Raif::ModelCompletion class will handle parsing the response for you, should you ask for a different response format (which can be one of :html, :text, or :json). You can also provide a system_prompt to the chat method:
llm=Raif.llm(:open_ai_gpt_4o)messages=[{role: "user",content: "Hello"},{role: "assistant",content: "Hello! How can I assist you today?"},{role: "user",content: "Can you you tell me a joke?"},]system_prompt="You are a helpful assistant who specializes in telling jokes. Your response should be a properly formatted JSON object containing a single `joke` key. Do not include any other text in your response outside the JSON object."model_completion=llm.chat(messages: messages,response_format: :json,system_prompt: system_prompt)putsmodel_completion.raw_response# => ```json# => {# => "joke": "Why don't skeletons fight each other? They don't have the guts."# => }# => ```putsmodel_completion.parsed_response# will strip backticks, parse the JSON, and give you a Ruby hash# => {"joke" => "Why don't skeletons fight each other? They don't have the guts."}
If you have a single-shot task that you want an LLM to do in your application, you should create a Raif::Task subclass, where you'll define the prompt and response format for the task and call via Raif::Task.run. For example, say you have a Document model in your app and want to have a summarization task for the LLM:
rails generate raif:task DocumentSummarization --response-format html
This will create a new task in app/models/raif/tasks/document_summarization.rb:
classRaif::Tasks::DocumentSummarization < Raif::ApplicationTaskllm_response_format:html# options are :html, :text, :jsonllm_temperature0.8# optional, defaults to 0.7llm_response_allowed_tags%w[pbidivstrong]# optional, defaults to Rails::HTML5::SafeListSanitizer.allowed_tagsllm_response_allowed_attributes%w[style]# optional, defaults to Rails::HTML5::SafeListSanitizer.allowed_attributes# Any attr_accessor you define can be included as an argument when calling `run`. # E.g. Raif::Tasks::DocumentSummarization.run(document: document, creator: user)attr_accessor:documentdefbuild_system_promptsp="You are an assistant with expertise in summarizing detailed articles into clear and concise language."sp += system_prompt_language_preferenceifrequested_language_key.present?spenddefbuild_prompt<<~PROMPT Consider the following information: Title: #{document.title} Text: ```#{document.content} ``` Your task is to read the provided article and associated information, and summarize the article concisely and clearly in approximately 1 paragraph. Your summary should include all of the key points, views, and arguments of the text, and should only include facts referenced in the text directly. Do not add any inferences, speculations, or analysis of your own, and do not exaggerate or overstate facts. If you quote directly from the article, include quotation marks. Format your response using basic HTML tags. If the text does not appear to represent the title, please return the text "#{summarization_failure_text}" and nothing else. PROMPTendend
And then run the task (typically via a background job):
document = Document.first # assumes your app defines a Document model
user = User.first # assumes your app defines a User model
task = Raif::Tasks::DocumentSummarization.run(document: document, creator: user)
summary = task.parsed_response
JSON Response Format Tasks
If you want to use a JSON response format for your task, you can do so by setting the llm_response_format to :json in your task subclass. If you're using OpenAI, this will set the response to use JSON mode. You can also define a JSON schema, which will then trigger utilization of OpenAI's structured outputs feature. If you're using Claude, it will create a tool for Claude to use to generate a JSON response.
This will create a new task in app/models/raif/tasks/web_search_query_generation.rb:
moduleRaifmoduleTasksclassWebSearchQueryGeneration < Raif::ApplicationTaskllm_response_format:jsonattr_accessor:topicjson_response_schemadoarray:queriesdoitemstype: "string"endenddefbuild_prompt<<~PROMPT Generate a list of 3 search queries that I can use to find information about the following topic:#{topic} Format your response as JSON. PROMPTendendendend
You can also pass in a requested_language_key to the run method. When this is provided, Raif will add a line to the system prompt requesting that the LLM respond in the specified language:
Would produce a system prompt that looks like this:
You are an assistant with expertise in summarizing detailed articles into clear and concise language.
You're collaborating with teammate who speaks Spanish. Please respond in Spanish.
The current list of valid language keys can be found here.
Raif provides Raif::Conversation and Raif::ConversationEntry models that you can use to provide an LLM-powered chat interface. It also provides controllers and views for the conversation interface.
This feature utilizes Turbo Streams, Stimulus controllers, and ActiveJob, so your application must have those set up first.
To use it in your application, first set up the css and javascript in your application. In the <head> section of your layout file:
<%= stylesheet_link_tag "raif" %>
In an app using import maps, add the following to your application.js file:
And then in the view where you'd like to display the conversation interface:
<%= raif_conversation(@conversation) %>
If your app already includes Bootstrap styles, this will render a conversation interface that looks something like:
If your app does not include Bootstrap, you can override the views to update styles.
If your application has a specific type of conversation that you use frequently, you can create a custom conversation type by running the generator. For example, say you are implementing a customer support chatbot in your application and want to have a custom conversation type for doing this with the LLM:
rails generate raif:conversation CustomerSupport
This will create a new conversation type in app/models/raif/conversations/customer_support.rb.
You can then customize the system prompt, initial message, and available model tools for that conversation type:
classRaif::Conversations::CustomerSupport < Raif::Conversationbefore_create->{self.available_model_tools=["Raif::ModelTools::SearchKnowledgeBase","Raif::ModelTools::FileSupportTicket"]}defsystem_prompt_intro<<~PROMPT You are a helpful assistant who specializes in customer support. You're working with a customer who is experiencing an issue with your product. PROMPTenddefinitial_chat_messageI18n.t("#{self.class.name.underscore.gsub("/",".")}.initial_chat_message")endend
Raif also provides Raif::Agents::ReActAgent, which implements a ReAct-style agent loop using tool calls:
# Create a new agentagent=Raif::Agents::ReActAgent.new(task: "Research the history of the Eiffel Tower",available_model_tools: [Raif::ModelTools::WikipediaSearch,Raif::ModelTools::FetchUrl],creator: current_user)# Run the agent and get the final answerfinal_answer=agent.run!# Or run the agent and monitor its progressagent.run!do |conversation_history_entry|
Turbo::StreamsChannel.broadcast_append_to(:my_agent_channel,target: "agent-progress",partial: "my_partial_displaying_agent_progress",locals: {agent: agent,conversation_history_entry: conversation_history_entry})end
On each step of the agent loop, an entry will be added to the Raif::Agent#conversation_history and, if you pass a block to the run! method, the block will be called with the conversation_history_entry as an argument. You can use this to monitor and display the agent's progress in real-time.
The conversation_history_entry will be a hash with "role" and "content" keys:
{"role"=>"assistant","content"=>"a message here"}
You can create custom agents using the generator:
rails generate raif:agent WikipediaResearchAgent
This will create a new agent in app/models/raif/agents/wikipedia_research_agent.rb:
moduleRaifmoduleAgentsclassWikipediaResearchAgent < Raif::Agent# If you want to always include a certain set of model tools with this agent type,# uncomment this callback to populate the available_model_tools attribute with your desired model tools.# before_create -> {# self.available_model_tools ||= [# Raif::ModelTools::WikipediaSearchTool,# Raif::ModelTools::FetchUrlTool# ]# }# Enter your agent's system prompt here. Alternatively, you can change your agent's superclass# to an existing agent types (like Raif::Agents::ReActAgent) to utilize an existing system prompt.defbuild_system_prompt# TODO: Implement your system prompt hereend# Each iteration of the agent loop will generate a new Raif::ModelCompletion record and# then call this method with it as an argument.defprocess_iteration_model_completion(model_completion)# TODO: Implement your iteration processing hereendendendend
You can create your own model tools to provide to the LLM using the generator:
rails generate raif:model_tool GoogleSearch
This will create a new model tool in app/models/raif/model_tools/google_search.rb:
classRaif::ModelTools::GoogleSearch < Raif::ModelTool# For example tool implementations, see: # Wikipedia Search Tool: https://github.com/CultivateLabs/raif/blob/main/app/models/raif/model_tools/wikipedia_search.rb# Fetch URL Tool: https://github.com/CultivateLabs/raif/blob/main/app/models/raif/model_tools/fetch_url.rb# Define the schema for the arguments that the LLM should use when invoking your tool.# It should be a valid JSON schema. When the model invokes your tool,# the arguments it provides will be validated against this schema using JSON::Validator from the json-schema gem.## All attributes will be required and additionalProperties will be set to false.## This schema would expect the model to invoke your tool with an arguments JSON object like:# { "query" : "some query here" }tool_arguments_schemadostring:query,description: "The query to search for"end# An example of how the LLM should invoke your tool. This should return a hash with name and arguments keys.# `to_json` will be called on it and provided to the LLM as an example of how to invoke your tool.example_model_invocationdo{"name": tool_name,"arguments": {"query": "example query here"}}endtool_descriptiondo"Description of your tool that will be provided to the LLM so it knows when to invoke it"end# When your tool is invoked by the LLM in a Raif::Agent loop, # the results of the tool invocation are provided back to the LLM as an observation.# This method should return whatever you want provided to the LLM.# For example, if you were implementing a GoogleSearch tool, this might return a JSON# object containing search results for the query.defself.observation_for_invocation(tool_invocation)return"No results found"unlesstool_invocation.result.present?JSON.pretty_generate(tool_invocation.result)end# When the LLM invokes your tool, this method will be called with a `Raif::ModelToolInvocation` record as an argument.# It should handle the actual execution of the tool. # For example, if you are implementing a GoogleSearch tool, this method should run the actual search# and store the results in the tool_invocation's result JSON column.defself.process_invocation(tool_invocation)# Extract arguments from tool_invocation.tool_arguments# query = tool_invocation.tool_arguments["query"]## Process the invocation and perform the desired action# ...## Store the results in the tool_invocation# tool_invocation.update!(# result: {# # Your result data structure# }# )## Return the result# tool_invocation.resultendend
Raif supports images, files, and PDF's in the messages sent to the LLM.
To include an image, file/PDF in a message, you can use the Raif::ModelImageInput and Raif::ModelFileInput.
To include an image:
# From a local fileimage=Raif::ModelImageInput.new(input: "path/to/image.png")# From a URLimage=Raif::ModelImageInput.new(url: "https://example.com/image.png")# From an ActiveStorage attachment (assumes you have a User model with an avatar attachment)image=Raif::ModelImageInput.new(input: user.avatar)# Then chat with the LLMllm=Raif.llm(:open_ai_gpt_4o)model_completion=llm.chat(messages: [{role: "user",content: ["What's in this image?",image]}])
To include a file/PDF:
# From a local filefile=Raif::ModelFileInput.new(input: "path/to/file.pdf")# From a URLfile=Raif::ModelFileInput.new(url: "https://example.com/file.pdf")# From an ActiveStorage attachment (assumes you have a Document model with a pdf attachment)file=Raif::ModelFileInput.new(input: document.pdf)# Then chat with the LLMllm=Raif.llm(:open_ai_gpt_4o)model_completion=llm.chat(messages: [{role: "user",content: ["What's in this file?",file]}])
Images/Files/PDF's in Tasks
You can include images and files/PDF's when running a Raif::Task:
To include a file/PDF:
file=Raif::ModelFileInput.new(input: "path/to/file.pdf")# Assumes you've created a PdfContentExtraction tasktask=Raif::Tasks::PdfContentExtraction.run(creator: current_user,files: [file])
To include an image:
image=Raif::ModelImageInput.new(input: "path/to/image.png")# Assumes you've created a ImageDescriptionGeneration tasktask=Raif::Tasks::ImageDescriptionGeneration.run(creator: current_user,images: [image])
Raif supports generation of vector embeddings. You can enable and configure embedding models in your Raif configuration:
Raif currently supports the following embedding models:
open_ai_text_embedding_3_small
open_ai_text_embed ding_3_large
open_ai_text_embedding_ada_002
bedrock_titan_embed_text_v2
By default, Raif will used Raif.config.default_embedding_model_key to create embeddings. To create an embedding for a piece of text:
# Generate an embedding for a piece of textembedding=Raif.generate_embedding!("Your text here")# Generate an embedding for a piece of text with a specific number of dimensionsembedding=Raif.generate_embedding!("Your text here",dimensions: 1024)# If you're using an OpenAI embedding model, you can pass an array of strings to embed multiple texts at onceembeddings=Raif.generate_embedding!(["Your text here","Your other text here"])
Or to generate embeddings for a piece of text with a specific model:
model=Raif.embedding_model(:open_ai_text_embedding_3_small)embedding=model.generate_embedding!("Your text here")
Raif includes a web admin interface for viewing all interactions with the LLM. Assuming you have the engine mounted at /raif, you can access the admin interface at /raif/admin.
The admin interface contains sections for:
Model Completions
Tasks
Conversations
Agents
Model Tool Invocations
Stats
You can override Raif's controllers by creating your own that inherit from Raif's base controllers:
classConversationsController < Raif::ConversationsController# Your customizations hereendclassConversationEntriesController < Raif::ConversationEntriesController# Your customizations hereend
You can customize Raif's views by copying them to your application and modifying them. To copy the conversation-related views, run:
rails generate raif:views
This will copy all conversation and conversation entry views to your application in:
app/views/raif/conversations/
app/views/raif/conversation_entries/
These views will automatically override Raif's default views. You can customize them to match your application's look and feel while maintaining the same functionality.
If you don't want to override the system prompt entirely in your task/conversation subclasses, you can customize the intro portion of the system prompts for conversations and tasks:
Raif.configuredo |config|
config.conversation_system_prompt_intro="You are a helpful assistant who specializes in customer support."config.task_system_prompt_intro="You are a helpful assistant who specializes in data analysis."# or with a lambdaconfig.task_system_prompt_intro=->(task){"You are a helpful assistant who specializes in #{task.name}."}config.conversation_system_prompt_intro=->(conversation){"You are a helpful assistant talking to #{conversation.creator.email}. Today's date is #{Date.today.strftime('%B %d, %Y')}."}end
You can easily add new LLM models to Raif:
# Register the model in Raif's LLM registryRaif.register_llm(Raif::Llms::OpenRouter,{key: :open_router_gemini_flash_1_5_8b,# a unique key for the modelapi_name: "google/gemini-flash-1.5-8b",# name of the model to be used in API calls - needs to match the provider's API nameinput_token_cost: 0.038 / 1_000_000,# the cost per input tokenoutput_token_cost: 0.15 / 1_000_000,# the cost per output token})# Then use the modelllm=Raif.llm(:open_router_gemini_flash_1_5_8b)llm.chat(message: "Hello, world!")# Or set it as the default LLM model in your initializerRaif.configuredo |config|
config.default_llm_model_key="open_router_gemini_flash_1_5_8b"end
Raif includes RSpec helpers and FactoryBot factories to help with testing in your application.
To use the helpers, add the following to your rails_helper.rb:
it"stubs a document summarization task"do# the messages argument is the array of messages sent to the LLM. It will look something like:# [{"role" => "user", "content" => "The prompt from the Raif::Tasks::DocumentSummarization task" }]# The model_completion argument is the Raif::ModelCompletion record that was created for this task.stub_raif_task(Raif::Tasks::DocumentSummarization)do |messages,model_completion|
"Stub out the response from the LLM"enduser=FactoryBot.create(:user)# assumes you have a User model & factorydocument=FactoryBot.create(:document)# assumes you have a Document model & factorytask=Raif::Tasks::DocumentSummarization.run(document: document,creator: user)expect(task.raw_response).toeq("Stub out the response from the LLM")endit"stubs a conversation"douser=FactoryBot.create(:user)# assumes you have a User model & factoryconversation=FactoryBot.create(:raif_test_conversation,creator: user)conversation_entry=FactoryBot.create(:raif_conversation_entry,raif_conversation: conversation,creator: user)stub_raif_conversation(conversation)do |messages,model_completion|
"Hello"endconversation_entry.process_entry!expect(conversation_entry.reload).tobe_completedexpect(conversation_entry.model_response_message).toeq("Hello")endit"stubs an agent"doi=0stub_raif_agent(agent)do |messages,model_completion|
i += 1ifi == 1"<thought>I need to search.</thought>\n<action>{\"tool\": \"wikipedia_search\", \"arguments\": {\"query\": \"capital of France\"}}</action>"else"<thought>Now I know.</thought>\n<answer>Paris</answer>"endendend
Raif also provides FactoryBot factories for its models. You can use them to create Raif models for testing. If you're using factory_bot_rails, they will be added automatically to config.factory_bot.definition_file_paths. The available factories can be found here.
Raif includes a demo app that you can use to see the engine in action. Assuming you have Ruby 3.4.2 and Postgres installed, you can run the demo app with:
git clone [email protected]:CultivateLabs/raif_demo.git
cd raif_demo
bundle install
bin/rails db:create db:prepare
OPENAI_API_KEY=your-openai-api-key-here bin/rails s
.png)
8 hours ago
1
