Least Outstanding Requests steering

Least Outstanding Requests steering allows you to route traffic to endpoints that currently have the lowest number of outstanding requests.

This steering policy selects an endpoint by taking into consideration endpoint weights, as well as each endpoint’s number of in-flight requests. Endpoints with more pending requests are weighted proportionately less in relation to others.

Least Outstanding Requests steering is best to use if your endpoints are easily overwhelmed by a spike in concurrent requests. It supports adaptive routing and session affinity.

​​ Configure via the API

Pools
{
  "origin_steering": {
    "policy": "least_outstanding_requests"
  }
}

Refer to the API documentation Open API docs link for more information on the pool configuration.

​​ Limitations

Least Outstanding Requests steering can be configured for pools that are part of DNS-only load balancers, but is only supported in a no-operation form. When endpoint steering logic is applied for a pool on a DNS-only load balancer, all endpoint outstanding request counts are considered to be zero, meaning traffic is served solely based on endpoint weights.

Although it is configurable, it is not recommended to associate pools that use Least Outstanding Requests steering with DNS-only load balancers due to its partial support.