HTTP Load Balancing

Overview

Load balancing in Contextual involves distributing incoming network traffic across multiple Agent instances. This ensures that no single instance is overwhelmed, improving the application's availability, fault tolerance, and scalability.

Key Concepts

1. Instances: The smallest deployable units in Contextual, each running an Agent. Instances handle individual parts of the workload.

2. Deployments: Contextual objects that manage the lifecycle of instances, including scaling up and down, updating, and rolling back instances.

3. Services: Contextual abstractions that define a logical set of instances and a policy by which to access them. Services provide load balancing across the instances.

4. Ingress Controllers: Components that manage external access to the services in a Contextual environment, typically providing HTTP and HTTPS routing.

How It Works

1. Deployment Management:

   • Replicas: A Contextual Deployment ensures that a specified number of Agent instances are running at any given time. If an instance fails, Contextual will automatically replace it.

   • Rolling Updates: The Deployment can perform rolling updates, ensuring that Agent instances are updated with zero downtime.

2. Service Discovery and Load Balancing:

   • Service Definition: A Contextual Service is created to expose the Agent instances. This service has a stable IP address and DNS name, allowing other services or external users to connect to Agents without needing to know the IP addresses of individual instances.

   • Load Balancing: The Service automatically balances HTTP requests across the available Agent instances. When a request comes in, it is distributed to one of the instances based on the service's load balancing policy, typically round-robin or random.

3. Ingress Management:

   • Ingress Controller: An Ingress Controller is deployed in the Contextual environment to manage external access. The Ingress Controller routes external HTTP and HTTPS traffic to the appropriate services based on defined rules.

   • Ingress Resource: An Ingress Resource is configured to define how incoming requests should be routed to the Agent Service. This allows for more advanced routing rules, SSL termination, and load balancing for HTTP traffic.

Example Flow

1. Client Request:

   • A client sends an HTTP request to the Agent application.

2. Ingress Controller:

   • The request hits the Ingress Controller, which examines the Ingress Resource to determine the correct Service to forward the request to.

3. Service Load Balancing:

   • The Service receives the request and forwards it to one of the Agent instances, distributing traffic evenly based on the configured load balancing policy.

4. Agent Instance:

   • The selected Agent instance processes the request and sends the response back to the client.

Benefits

• Scalability: Contextual can scale the number of Agent instances up or down based on traffic load, ensuring optimal resource usage.

• High Availability: By distributing traffic across multiple instances, Contextual ensures that the failure of a single Agent instance does not bring down the entire service.

• Resilience: Contextual automatically handles instance failures and restarts, maintaining the desired number of Agent instances running at all times.

Summary

Load balancing across multiple Agents in Contextual involves using Deployments to manage the lifecycle of Agent instances, Services to provide load-balanced access to these instances, and Ingress Controllers to manage external HTTP traffic. This architecture ensures that traffic is evenly distributed, improves fault tolerance, and allows the system to scale dynamically based on load.