OpenAI

The OpenAI node connects to OpenAI’s hosted large language models. It accepts input prompts, processes them using a selected model, and returns generated answers. This node is typically used for tasks such as reasoning, summarization, content generation, and conversational response.

Inputs

  • Prompt – Text prompt for the model
  • Questions – This port accepts plain text prompts or user-generated questions. These are forwarded to the OpenAI model for processing
  • Documents – Document objects for context
  • System – System instructions for the model

Outputs

  • Text – Generated text output
  • Answers – This port outputs the model-generated response based on the input received. The output is a string of generated text that can be passed to other nodes

Configuration

Model Settings

  • Model – OpenAI model to use
    • Default – “gpt-4o”
    • Notes – Available models include gpt-4o, gpt-4, gpt-3.5-turbo
      • Use GPT-4o for highest quality and complex reasoning tasks
      • Use GPT-3.5-Turbo for faster responses and lower cost
      • Consider fine-tuned models for specialized applications
  • API Key – OpenAI API key
    • Notes – Required for authentication
  • Project (Organization)
    • Optionally enter your OpenAI project or organization name if applicable. This is used for project-level identification in the API
  • Temperature – Creativity/randomness level
    • Default – 0.7
    • Note – Range: 0.0-1.0
  • Max Tokens – Maximum response length
    • Default – 1024
    • Notes – Limits output size

Advanced Settings

  • Top P – Nucleus sampling parameter
    • Default – 1.0
    • Notes – Controls diversity
  • Frequency Penalty – Penalty for token frequency
    • Default – 0.0
    • Notes – Range: -2.0 to 2.0
  • Presence Penalty – Penalty for token presence
    • Default – 0.0
    • Notes – Range: -2.0 to 2.0
  • System Prompt – Default system instructions
    • Note – Sets model behavior
  • Stop Sequences – Sequences to stop generation
    • Default – []
    • Notes – Custom stop tokens
  • Timeout – API request timeout
    • Default – 60
    • Note – In seconds

Example Usage

Basic Text Generation

This example shows how to configure the OpenAI LLM for basic text generation:
{
"model": "gpt-3.5-turbo",
"apiKey": "your-api-key",
"temperature": 0.7,
"maxTokens": 1024,
"topP": 1.0
}

RAG Implementation with GPT-4

For a Retrieval-Augmented Generation (RAG) implementation using GPT-4:
{
"model": "gpt-4o",
"apiKey": "your-api-key",
"temperature": 0.3,
"maxTokens": 2048,
"systemPrompt": "You are a helpful assistant that answers questions based on the provided documents. Always cite your sources and maintain a professional tone.",
"topP": 0.9,
"frequencyPenalty": 0.2,
"presencePenalty": 0.2,
"timeout": 120
}

Best Practices

Prompt Engineering

  • Provide clear, specific instructions in your prompts
  • Use system prompts to establish consistent behavior
  • Include relevant context for knowledge-intensive tasks
  • Structure prompts with clear sections for complex tasks

Performance Optimization

  • Adjust temperature based on task requirements (lower for factual responses, higher for creative content)
  • Set appropriate max tokens to avoid unnecessary processing
  • Use streaming for responsive user interfaces

Troubleshooting

API Problems

  • Authentication errors – Verify API key validity
  • Rate limit exceeded – Implement request throttling or upgrade API tier
  • Timeout errors – Increase timeout setting or reduce prompt/context size

Response Quality Issues

  • Irrelevant responses – Refine prompts or adjust system instructions
  • Inconsistent outputs – Lower temperature for more deterministic responses
  • Truncated responses – Increase max tokens setting
Technical Reference

For detailed technical information, refer to: