Anthropic Claude Image Analysis

The Anthropic Claude Image Analysis solution leverages advanced AI capabilities for analyzing images and generating descriptive responses. It is a versatile tool applicable in various domains such as go-to-market strategies, service delivery, IoT, and data transformation.

This flow is useful for automatically analyzing image inputs and storing descriptive results for further use or processing.

This template can be found in the Services Catalog under the following categories:

• AI, Contextual Basics

What's Included

• 1 Flow

• 1 Object Type

• 1 Connection

What You'll Need

• An API key for the Anthropic Claude API

Ideas for Using Image Analysis

Visual Content Categorization for Go-to-Market Strategies

Use AI to analyze product images and automatically categorize them based on visual features. This can help tailor marketing campaigns by aligning products with specific customer segments, improving ad targeting and content personalization.

Automated Quality Control in Manufacturing

Implement image analysis to monitor production lines, detecting defects or inconsistencies in real-time. This can significantly reduce waste, improve product quality, and streamline the manufacturing process.

Enhanced Field Service Documentation

Use image analysis to automatically tag and categorize photos taken by field service technicians. This enables better documentation of on-site issues, faster issue resolution, and improved service records management.

Visual Data Transformation in IoT

Apply image analysis to interpret visual data from IoT devices, such as security cameras or environmental sensors. This can enhance automated decision-making, from identifying safety hazards to monitoring environmental conditions.

Flow Overview

1. Flow Start: The flow begins by receiving a prompt (such as a question or text) and an image URL.

2. Convert Image URL to Binary Buffer: The image URL is fetched and converted into a binary buffer, which is necessary for processing by the Anthropic Claude API.

3. Send Prompt and Image to Receive Response: The image buffer and text prompt are formatted and sent to the Anthropic Claude API, which processes the input and generates a descriptive response.

4. Format Response & Create Record: The API response is formatted, and a new record is created within the system. This record includes the original prompt, the image, and the AI's descriptive response.

5. Error Handling: Any errors encountered during the flow are captured and logged for review.

Flow Details

The flow includes several key steps, each handled by specific nodes within the system. Below is a detailed breakdown of the JavaScript and other configurations involved:

1. Inject Node Node: Test Prompt Purpose: Provides initial data for testing the flow. This node injects a sample prompt ("What is this?") and an image URL into the flow. The data is manually set for testing.

2. Prepare Prompt Function Node: Prepare Prompt Purpose: Formats the prompt and image data for submission to the Anthropic Claude API. This function prepares the data by encoding the image to base64 and structuring the payload for the API.

// Prepare the payload for Claude API request
msg.payload = {
    model: "claude-3-5-sonnet-20240620",
    max_tokens: 1000,
    temperature: 0.3,
    messages: [
        {
            role: "user",
            content: [
                {
                    type: "image",
                    source: {
                        type: "base64",
                        media_type: "image/jpeg",
                        data: msg.payload.toString('base64')
                    }
                },
                {
                    type: "text",
                    text: msg.event.prompt
                }
            ]
        }
    ]
};

// Return the modified msg object
return msg;

3. API Interaction Node: Prompt Anthropic Claude Purpose: Sends the formatted data to the Anthropic Claude API and retrieves the response, which contains the AI's descriptive analysis of the image.

4. Prepare Record Data Function Node: Prepare Record Data Purpose: Formats the API response and prepares it for storage as a new record in the system. The record includes the prompt, AI response, and image URL.

// Prepare data for Create Object node and assign to msg.payload
let body = {
    prompt: msg.event.prompt,
    aiResponse: msg.payload.response.content[0].text,
    image: msg.responseUrl
};

// The Create Object node will use the content of msg.payload to Create a new Record
msg.payload = body;

return msg;

5. Record Creation Node: Create AI Response Record Purpose: Creates a new record in the system using the formatted data from the previous node. This record stores the original prompt, the image, and the AI's response.

6. Error Handling Nodes: Catch, Error Catch Log Purpose: Captures and logs any errors that occur during the flow, ensuring smooth operation and easy troubleshooting.

Summary of Flow

• Flow Start: A sample prompt and image URL are injected.

• Data Preparation: The image is converted to a binary buffer, and the prompt is formatted for the API.

• API Interaction: The data is sent to the Anthropic Claude API, and the response is received.

• Record Creation: The API response is formatted and stored as a record.

• Error Handling: Any errors are captured and logged