Skip to content

AI Workflows Guide

Introduction

This guide explains how to create and manage AI workflows using Local-AI-Cyber-Lab's integrated tools. We'll cover workflow creation, automation, and best practices for different use cases.

Tools Overview

1. Flowise

  • Visual workflow builder
  • Node-based programming
  • Multiple LLM support
  • API generation

2. n8n

  • Automation platform
  • 200+ integrations
  • Custom JavaScript code
  • Webhook support

3. LangFuse

  • Workflow monitoring
  • Performance tracking
  • Error detection
  • Usage analytics

Creating Basic Workflows

1. Chat Interface Workflow

Using Flowise: 

graph LR
    A[User Input] --> B[Input Parser]
    B --> C[LLM Node]
    C --> D[Response Formatter]
    D --> E[Output]

Configuration: 

nodes:
  input_parser:
    type: text
    format: markdown

  llm:
    type: ollama
    model: mistral
    temperature: 0.7

  formatter:
    type: text
    format: html

2. Document Processing

Using n8n: 

// Document processing workflow
const workflow = {
  nodes: [
    {
      type: 'webhook',
      endpoint: '/process-doc'
    },
    {
      type: 'function',
      code: 'return {text: $input.body.text}'
    },
    {
      type: 'ai',
      model: 'mistral',
      prompt: 'Summarize:'
    }
  ]
}

Advanced Workflows

1. Multi-Model Chain

# Example chain configuration
chain = {
    "name": "Advanced Analysis",
    "models": [
        {
            "name": "mistral",
            "task": "initial_analysis"
        },
        {
            "name": "codellama",
            "task": "code_generation"
        },
        {
            "name": "llama2",
            "task": "final_review"
        }
    ]
}

2. Data Processing Pipeline

pipeline:
  steps:
    - name: data_ingestion
      type: file_reader
      format: csv,json,txt

    - name: preprocessing
      type: text_cleaner
      operations:
        - remove_html
        - normalize_text

    - name: embedding
      type: vector_encoder
      model: all-MiniLM-L6-v2

    - name: storage
      type: qdrant
      collection: processed_data

Integration Examples

1. API Integration

from fastapi import FastAPI
from flowise_sdk import FlowiseAPI

app = FastAPI()
flowise = FlowiseAPI()

@app.post("/process")
async def process_request(data: dict):
    result = await flowise.run_workflow(
        workflow_id="your-workflow-id",
        input_data=data
    )
    return result

2. Database Integration

from qdrant_client import QdrantClient
from langfuse.client import Client

# Initialize clients
qdrant = QdrantClient()
langfuse = Client()

async def search_and_process(query: str):
    # Search similar vectors
    results = qdrant.search(
        collection_name="data",
        query_vector=generate_embedding(query)
    )

    # Process with LLM
    response = await process_with_context(
        query=query,
        context=results
    )

    # Log interaction
    langfuse.log_interaction(
        type="query",
        input=query,
        output=response
    )

Monitoring and Optimization

1. Performance Monitoring

Using LangFuse: 

from langfuse import Langfuse

langfuse = Langfuse()

def monitor_workflow(workflow_id: str):
    metrics = langfuse.get_metrics(
        workflow_id=workflow_id,
        timeframe="1d"
    )

    return {
        "latency": metrics.avg_latency,
        "success_rate": metrics.success_rate,
        "error_rate": metrics.error_rate,
        "cost": metrics.total_cost
    }

2. Error Handling

try:
    result = await workflow.execute(input_data)
except WorkflowError as e:
    # Log error
    langfuse.log_error(
        workflow_id=workflow.id,
        error=str(e),
        severity="high"
    )

    # Fallback strategy
    result = await fallback_workflow.execute(input_data)

Best Practices

1. Workflow Design

  • Keep workflows modular
  • Implement error handling
  • Use version control
  • Document dependencies

2. Performance

  • Cache frequent requests
  • Optimize model selection
  • Monitor resource usage
  • Implement rate limiting

3. Security

  • Validate inputs
  • Sanitize outputs
  • Implement access control
  • Monitor usage patterns

Workflow Templates

1. Content Generation

name: content_generator
version: 1.0
steps:
  - name: topic_analysis
    type: llm
    model: mistral

  - name: research
    type: web_search
    sources: ["approved_sites"]

  - name: content_creation
    type: llm
    model: llama2

  - name: review
    type: human_review
    approval_required: true

2. Code Analysis

name: code_analyzer
version: 1.0
steps:
  - name: code_parsing
    type: parser
    language: auto

  - name: security_scan
    type: security_check
    rules: ["owasp_top_10"]

  - name: improvement_suggestions
    type: llm
    model: codellama

Support

For workflow-related assistance: