Overview

On this page

...

The Response Cache policy adds the capability for API Proxies to cache reoccurring and/or static HTTP responses to manage latency, decrease response times, and reduce load on a proxied server. By default, cached HTTP responses are stored based on the proxy's URL and request method. Additionally, you can configure cache responses for specified request header keys and query string keys. Following policy configuration, cached responses are returned to HTTP clients instead of requesting resources from the proxied server.

Use this policy to save time when network is congested and proxy API response is delayedfor highly reoccurring and static responses to manage latency and response times, which can reduce the load on the upstream API as well as the proxy server. An incoming HTTP request to one of your Proxy endpoints returns the cached response after that’s stored for a specified time period. The policy owner can configure the cache time-to-live (TTL) per policy and the cache key used to cache each request (with every key-value pair).

In the policy settings, you specify the cache key parameters that the HTTP requests map to and set the cache response time and refresh interval.

...

Policy Execution Order

This policy executes after authentication and traffic shaping policies.

Architecture

This policy uses Google Guava cache, which offers a map to send responses. The cache must be enabled through feature flags on the Snaplex. runs after every request and response policy.

Known Issues

Both the HTTP Cache Response and HTTP Retry Policies fail to evaluate expressions. As a result, the When this policy should be applied field should not be set. Doing so can cause issues with the way the policy is applied.

Architecture

Each Snaplex node has its own cache, so the cache for an incoming request might be missed on each node until properly distributed and hydrated on each node.

By default, the following cache keys are used:

...

• Protocol

• Host 

• PAth Path 

• Method

Headers/Query params supported

Hash Key SChe1 algorithm

Each key-value pair in the Response Cache Map will be:

{
  policy_instance_id: Google Guava Cache,
  ...
}

...

{
  [ Hash Hex of params specified by policy ]: Cached HTTP Response
  ...
}

• Original HTTP Response Object for preserving the flow of handling HTTP responses in the Gateway Resource.

  byte[] of the response content to preserve the stream that cannot be rewound

  HTTP Method

The policy supports incorporating headers and query parameters as part of the cache key (not to access the cache key). The headers and query parameter values are used as part of the generated cache key to ensure unique cache entries as needed.

Usage Guidelines

• We limit each policy to store 100 entries in the cache per policy. Accordingly, the policy evicts the least used entry after 100.

• Expired caches cannot be accessed, and new entries overwrite any existing ones.

...

• policy version to determine if the policy has been updated since the last save. An update to the policy will direct the flow to acquire a new HTTP response and overwrite the existing one.

Limitations

...

• The maximum quantity of bytes that an Response Caches in the default configuration is 85.83 MB of the payload.

Limitations

• Each response cache can only contain 85 MB. The policy always returns the response, but any data over the limit renders the payload incomplete without notice.

• A response cache is not effective to use for POST and PUT HTTP methods because these operations are meant to alter the state of data, and therefore should not be cached.

Settings

Best Practices

• Do not use any headers and query string parameters, like a timestamp, as part of your cache key, or it will never result in a fulfilled request. You can optionally remove those headers using the Request Transformer policy. Or, avoid headers query string parameters as part of the cache key and only as needed when a unique value is present per request.

• To invalidate cached data, use the Invalidate Response Cache API. The Invalidate Response Cache API enables you to keep cached data up to date with the latest changes. You can define caching rules for the specific API endpoint, specifying parameters such as cache duration, cache key generation, and cache storage. The cache invalidation mechanism monitors changes in the data source and updates the cache as configured to ensure data consistency.

Example

To demonstrate the advantage of caching reoccurring and/or static HTTP responses, the following proxy endpoint GET request is provided as an example.

This proxy endpoint reaches out to a calculator service that multiplies two integers, and the endpoint can be configured to cache responses whose query string keys are intA and intB.

GET
/gateway/proxy/multiply?intA=[valueA]&intB=[valueB]

With this configuration, HTTP responses are cached for HTTP requests containing query string keys of intA and intB .

The following requests have these query strings.

GET Request 1
/gateway/proxy/multiply?intA=5&intB=5

GET Request 2
/gateway/proxy/multiply?intA=2&intB=2

GET Request 3
/gateway/proxy/multiply?intA=3&intB=1

Where each cached HTTP response is:

Response 1 
{
    "operation": "5 x 5", 
    "result": 25
}

Response 2 
{
    "operation": "2 x 2", 
    "result": 4
}

Response 3
{
    "operation": "3 x 1", 
    "result": 3
}

When future requests are made to the proxy endpoint for Request 1, Request 2, and Request 3, HTTP responses can be served from the Response Cache policy with payloads from Response 1, Response 2, and Response 3 instead of querying the upstream server, and therefore, freeing up computation resources for other operations.

Furthermore, each policy can be configured to evict cached responses for a specified timeframe. In a case where cached responses are expected to change after a 40-hour cycle, a Cache Interval of 40 and Time Unit of HOURS can be set so that the policy can evict previous HTTP responses and serve updated cached responses.

For example, the following GET request for billable work hours per week demonstrates the use of specifying a Cache Interval of 40 and a Time Unit of HOURS:

GET
/gateway/proxy/last-weeks-billable-hours?user='admin'

The cached response is:

Response 
{
    "week_of":  "01-01-2024",
    "department":  "Engineering", 
    "billable_hours":  3000, 
    "user:  "admin"
}

The policy continues serving cached HTTP responses from the previous example until it is evicted per the policy configurations.

Any GET requests to this proxy endpoint on the week of 01-08-2024 elapse the specified cache interval, and future cached responses return payloads like the following for the specified interval:

Response 
{
    "week_of":  "01-08-2024",
    "department":  "Engineering", 
    "billable_hours":  4500, 
    "user:  "admin"
}