File Support

Learn how to handle file uploads in your DAIN service tools

Overview

The Butterfly interface can process uploaded files up to 1GB in size. When users upload files, they are provided to your tool as URLs that can be downloaded and processed.

How It Works

When a user uploads a file through the Butterfly interface:

  1. The file is securely stored and a URL is generated
  2. The AI agent receives this URL and can pass it to your tool
  3. Your tool can download and process the file contents

Example: Image Analysis Tool

import { DainResponse, ImageCardUIBuilder } from "@dainprotocol/utils";

const analyzeImageConfig: ToolConfig = {
  id: "analyze-image",
  name: "Analyze Image",
  description: "Analyzes an image from a URL and answers questions about its content",
  input: z.object({
    imageUrl: z.string().describe("URL of the image to analyze. Accepts URLS for png, jpeg, and webp"),
    question: z.string().describe("Question to answer about the image content")
  }),
  output: z.object({
    answer: z.string().describe("Answer to the question based on image content")
  }),
  handler: async ({ imageUrl, question }, agentInfo) => {
    // Download the image
    const response = await axios.get(imageUrl, {
      responseType: 'arraybuffer'
    });
    const buffer = Buffer.from(response.data);

    // Process with Claude
    const { text: answer } = await generateText({
      model: anthropic('claude-3-5-sonnet-20241022'),
      system: "You are an assistant that helps answer questions about images...",
      messages: [
        {
          role: 'user',
          content: [
            { type: 'text', text: question },
            { type: 'image', image: buffer }
          ]
        }
      ]
    });

    // Create UI with ImageCardBuilder
    const imageUI = new ImageCardUIBuilder(imageUrl)
      .title("Image Analysis Result")
      .description(answer)
      .imageAlt("Analyzed Image")
      .onViewImage({
        tool: "imageViewer",
        params: {
          url: imageUrl
        }
      })
      .build();

    return new DainResponse({
      text: "Generated image analysis",
      data: { answer },
      ui: imageUI
    });
  }
};

Supported File Types

Your tool can process any file type, as long as you handle the download and processing appropriately. Common use cases include:

  • Images (PNG, JPEG, WebP)
  • Documents (PDF, DOCX, TXT)
  • Data files (CSV, JSON, XML)
  • Audio files (MP3, WAV)
  • Video files (MP4, WebM)