Skip to content

Optionally enable caching of any request method #24

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Feb 20, 2017
Merged

Optionally enable caching of any request method #24

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
52b1777
Enable caching of POST requests when requested
tuupola Feb 10, 2017
e0c1c0e
Make StyleCI happy
tuupola Feb 10, 2017
0a3b7c8
PSR-2 and short array syntax
tuupola Feb 11, 2017
8151701
Allow any valid request method as defined by RFC7230
tuupola Feb 13, 2017
baf4fbd
Include request body in cache key calculation
tuupola Feb 13, 2017
8d9d48c
Make StyleCI happy
tuupola Feb 13, 2017
5b21d4b
PHP 5.4 does not have ::class
tuupola Feb 13, 2017
cfa6135
Mention RFC7230 which defines valid request methods
tuupola Feb 14, 2017
8d8f8e9
Add new methods option to docblock
tuupola Feb 15, 2017
fbc1dc7
Request methods are case sensitive
tuupola Feb 15, 2017
e94b74a
Do not allow lowercase request methods
tuupola Feb 17, 2017
65d819d
Add delimiter between uri and body
tuupola Feb 17, 2017
10da16c
Remove unneeded use statement
tuupola Feb 17, 2017
9703885
Maintain cache key bc with v1.2.0
tuupola Feb 19, 2017
490da8b
Make StyleCI happy
tuupola Feb 19, 2017
727cbf0
Make StyleCI happy
tuupola Feb 19, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
# Change Log

## 1.3.0 - unreleased
### Added

- New `methods` setting which allows to configure the request methods which can be cached.

## 1.2.0 - 2016-08-16

### Changed

- The default value for `default_ttl` is changed from `null` to `0`.
- The default value for `default_ttl` is changed from `null` to `0`.

### Fixed

Expand Down
78 changes: 77 additions & 1 deletion spec/CachePluginSpec.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
use Psr\Http\Message\RequestInterface;
use Psr\Http\Message\ResponseInterface;
use Psr\Http\Message\StreamInterface;
use Symfony\Component\OptionsResolver\Exception\InvalidOptionsException;

class CachePluginSpec extends ObjectBehavior
{
Expand Down Expand Up @@ -41,6 +42,8 @@ function it_caches_responses(CacheItemPoolInterface $pool, CacheItemInterface $i

$request->getMethod()->willReturn('GET');
$request->getUri()->willReturn('/');
$request->getBody()->shouldBeCalled();

$response->getStatusCode()->willReturn(200);
$response->getBody()->willReturn($stream);
$response->getHeader('Cache-Control')->willReturn(array())->shouldBeCalled();
Expand Down Expand Up @@ -71,6 +74,8 @@ function it_doesnt_store_failed_responses(CacheItemPoolInterface $pool, CacheIte
{
$request->getMethod()->willReturn('GET');
$request->getUri()->willReturn('/');
$request->getBody()->shouldBeCalled();

$response->getStatusCode()->willReturn(400);
$response->getHeader('Cache-Control')->willReturn(array());
$response->getHeader('Expires')->willReturn(array());
Expand All @@ -85,7 +90,7 @@ function it_doesnt_store_failed_responses(CacheItemPoolInterface $pool, CacheIte
$this->handleRequest($request, $next, function () {});
}

function it_doesnt_store_post_requests(CacheItemPoolInterface $pool, CacheItemInterface $item, RequestInterface $request, ResponseInterface $response)
function it_doesnt_store_post_requests_by_default(CacheItemPoolInterface $pool, CacheItemInterface $item, RequestInterface $request, ResponseInterface $response)
{
$request->getMethod()->willReturn('POST');
$request->getUri()->willReturn('/');
Expand All @@ -97,6 +102,71 @@ function it_doesnt_store_post_requests(CacheItemPoolInterface $pool, CacheItemIn
$this->handleRequest($request, $next, function () {});
}

function it_stores_post_requests_when_allowed(
CacheItemPoolInterface $pool,
CacheItemInterface $item,
RequestInterface $request,
ResponseInterface $response,
StreamFactory $streamFactory,
StreamInterface $stream
) {
$this->beConstructedWith($pool, $streamFactory, [
'default_ttl' => 60,
'cache_lifetime' => 1000,
'methods' => ['GET', 'HEAD', 'POST']
]);

$httpBody = 'hello=world';
$stream->__toString()->willReturn($httpBody);
$stream->isSeekable()->willReturn(true);
$stream->rewind()->shouldBeCalled();

$request->getMethod()->willReturn('POST');
$request->getUri()->willReturn('/post');
$request->getBody()->willReturn($stream);

$response->getStatusCode()->willReturn(200);
$response->getBody()->willReturn($stream);
$response->getHeader('Cache-Control')->willReturn([])->shouldBeCalled();
$response->getHeader('Expires')->willReturn([])->shouldBeCalled();
$response->getHeader('ETag')->willReturn([])->shouldBeCalled();

$pool->getItem('e37195334979e7ca0dda534c48a02c7de8368d64')->shouldBeCalled()->willReturn($item);
$item->isHit()->willReturn(false);
$item->expiresAfter(1060)->willReturn($item)->shouldBeCalled();

$item->set($this->getCacheItemMatcher([
'response' => $response->getWrappedObject(),
'body' => $httpBody,
'expiresAt' => 0,
'createdAt' => 0,
'etag' => []
]))->willReturn($item)->shouldBeCalled();

$pool->save(Argument::any())->shouldBeCalled();

$next = function (RequestInterface $request) use ($response) {
return new FulfilledPromise($response->getWrappedObject());
};

$this->handleRequest($request, $next, function () {});
}

function it_does_not_allow_invalid_request_methods(
CacheItemPoolInterface $pool,
CacheItemInterface $item,
RequestInterface $request,
ResponseInterface $response,
StreamFactory $streamFactory,
StreamInterface $stream
) {
$this
->shouldThrow("Symfony\Component\OptionsResolver\Exception\InvalidOptionsException")
->during('__construct', [$pool, $streamFactory, ['methods' => ['GET', 'HEAD', 'POST ']]]);
$this
->shouldThrow("Symfony\Component\OptionsResolver\Exception\InvalidOptionsException")
->during('__construct', [$pool, $streamFactory, ['methods' => ['GET', 'HEAD"', 'POST']]]);
}

function it_calculate_age_from_response(CacheItemPoolInterface $pool, CacheItemInterface $item, RequestInterface $request, ResponseInterface $response, StreamInterface $stream)
{
Expand All @@ -107,6 +177,8 @@ function it_calculate_age_from_response(CacheItemPoolInterface $pool, CacheItemI

$request->getMethod()->willReturn('GET');
$request->getUri()->willReturn('/');
$request->getBody()->shouldBeCalled();

$response->getStatusCode()->willReturn(200);
$response->getBody()->willReturn($stream);
$response->getHeader('Cache-Control')->willReturn(array('max-age=40'));
Expand Down Expand Up @@ -141,6 +213,7 @@ function it_saves_etag(CacheItemPoolInterface $pool, CacheItemInterface $item, R
$stream->__toString()->willReturn($httpBody);
$stream->isSeekable()->willReturn(true);
$stream->rewind()->shouldBeCalled();
$request->getBody()->shouldBeCalled();

$request->getMethod()->willReturn('GET');
$request->getUri()->willReturn('/');
Expand Down Expand Up @@ -176,6 +249,7 @@ function it_adds_etag_and_modfied_since_to_request(CacheItemPoolInterface $pool,

$request->getMethod()->willReturn('GET');
$request->getUri()->willReturn('/');
$request->getBody()->shouldBeCalled();

$request->withHeader('If-Modified-Since', 'Thursday, 01-Jan-70 01:18:31 GMT')->shouldBeCalled()->willReturn($request);
$request->withHeader('If-None-Match', 'foo_etag')->shouldBeCalled()->willReturn($request);
Expand Down Expand Up @@ -205,6 +279,7 @@ function it_servces_a_cached_response(CacheItemPoolInterface $pool, CacheItemInt

$request->getMethod()->willReturn('GET');
$request->getUri()->willReturn('/');
$request->getBody()->shouldBeCalled();

$pool->getItem('d20f64acc6e70b6079845f2fe357732929550ae1')->shouldBeCalled()->willReturn($item);
$item->isHit()->willReturn(true);
Expand Down Expand Up @@ -233,6 +308,7 @@ function it_serves_and_resaved_expired_response(CacheItemPoolInterface $pool, Ca

$request->getMethod()->willReturn('GET');
$request->getUri()->willReturn('/');
$request->getBody()->shouldBeCalled();

$request->withHeader(Argument::any(), Argument::any())->willReturn($request);
$request->withHeader(Argument::any(), Argument::any())->willReturn($request);
Expand Down
14 changes: 11 additions & 3 deletions src/CachePlugin.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ final class CachePlugin implements Plugin
* @var int $cache_lifetime (seconds) To support serving a previous stale response when the server answers 304
* we have to store the cache for a longer time than the server originally says it is valid for.
* We store a cache item for $cache_lifetime + max age of the response.
* @var array $methods case insensitive list of request methods which can be cached.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its case sensitive, not insensitive

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes of course. Brain malfunction...

* }
*/
public function __construct(CacheItemPoolInterface $pool, StreamFactory $streamFactory, array $config = [])
Expand All @@ -64,7 +65,7 @@ public function handleRequest(RequestInterface $request, callable $next, callabl
{
$method = strtoupper($request->getMethod());
// if the request not is cachable, move to $next
if ($method !== 'GET' && $method !== 'HEAD') {
if (!in_array($method, $this->config['methods'])) {
return $next($request);
}

Expand Down Expand Up @@ -205,7 +206,6 @@ private function getCacheControlDirective(ResponseInterface $response, $name)
$headers = $response->getHeader('Cache-Control');
foreach ($headers as $header) {
if (preg_match(sprintf('|%s=?([0-9]+)?|i', $name), $header, $matches)) {

// return the value for $name if it exists
if (isset($matches[1])) {
return $matches[1];
Expand All @@ -225,7 +225,7 @@ private function getCacheControlDirective(ResponseInterface $response, $name)
*/
private function createCacheKey(RequestInterface $request)
{
return hash($this->config['hash_algo'], $request->getMethod().' '.$request->getUri());
return hash($this->config['hash_algo'], $request->getMethod().' '.$request->getUri().$request->getBody());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if the body is super large? Do we care?

Also, I suggest a space (or any other non-common delimiter) between uri and body.

Copy link
Contributor Author

@tuupola tuupola Feb 16, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Without space this is backwards compatible with previous versions if body is empty. Adding a space there will invalidate old caches, but it is of course ok with me if needed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great. I did not think about that. How about doing something like:

$body = (string) $request->getBody()
if (!empty($body)) {
  $body = ' ' . $body;
}

return hash($this->config['hash_algo'], $request->getMethod().' '.$request->getUri().$body);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we do not need to worry about keeping caches valid over deployments. but nyholms idea is good. as GET usually has empty body, it happens to keep old caches.

regarding big body: this method is only executed on methods that should be cached, right? so body will be usually empty, except for things like soap which hopefully won't be big file uploads. you would not want to cache a file upload, anyways.

Copy link
Member

@Nyholm Nyholm Feb 16, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GET and HEAD has (EDIT: generally) no bodies. If you change the default behavior, then it is on you if you get performance issues.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can add it. Why is the space needed BTW?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GET can have body, for example when talking to elasticsearch: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-body.html

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To make a difference between these two requests

POST /foo
Host: www.example.com

bar
POST /foob
Host: www.example.com

ar

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel we could unconditionally add the space. It is only cache so it is not that big deal if old cache gets invalidated when plugin is updated. What do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, sorry for the late response.

I would prefer not invalidating cache when we can.. Im using Google Translate paid services and I know I will have to pay 100USD if my cache gets invalidated.

Technically, this is okey. But we could be more nice to our users. =)

}

/**
Expand Down Expand Up @@ -273,12 +273,20 @@ private function configureOptions(OptionsResolver $resolver)
'default_ttl' => 0,
'respect_cache_headers' => true,
'hash_algo' => 'sha1',
'methods' => ['GET', 'HEAD'],
]);

$resolver->setAllowedTypes('cache_lifetime', ['int', 'null']);
$resolver->setAllowedTypes('default_ttl', ['int', 'null']);
$resolver->setAllowedTypes('respect_cache_headers', 'bool');
$resolver->setAllowedTypes('methods', 'array');
$resolver->setAllowedValues('hash_algo', hash_algos());
$resolver->setAllowedValues('methods', function ($value) {
/* Any VCHAR, except delimiters. RFC7230 sections 3.1.1 and 3.2.6 */
$matches = preg_grep('/[^[:alnum:]!#$%&\'*\/+\-.^_`|~]/', $value);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you please add a comment here with the RFC number that defines this regular expression? just for future reference and to make it clear we did not come up with a random expression here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i am horrible with regular expressions. is this forcing upper case? we should either force that or uppercase ourselves when processing the options. PSR-7 forces method to upper case. when you specify something lower case, it would never match.

the doc should explain that if you specify methods, you also need to specify GET if you still want those cached. and that you should do specific clients with different setup if you do soap and other stuff, to be sure only the right requests get cached...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does not force uppercase. [:alnum:] is alias for [0-9A-Za-z]. RFC2616 says method is case-insensitive so technically both get and GET should be valid but just different methods.

I have no strong feelings whether we should force uppercase or not. I think most of the time people probably use uppercase methods.

The regexp basically says is request method has any other characters than these the method is invalid.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks really complicated. Why not use [A-Z]+. That will also make sure you use upper values. Or am I too specific?

Copy link
Contributor Author

@tuupola tuupola Feb 16, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless I have misunderstood the specs request method can also contain lowercase letters. Valid request methods are described in RFC7230-3.1.1 which states it is a token. Token is defined in RFC7230-3.2.6 which says any visible USASCII character except delimiters "(),/:;<=>?@[\]{}.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, yes you are correct. (Reading the code and RFC more carefully...). I was wrong assuming that !#$%& etc should not be a part of the HTTP method.

HTTP methods are generally uppercase but PSR-7 specifies that case should not change. I think we are correct by running strtoupper(). This implementation will, however, always fail when using a lower case method in the configuration. I suggest changing the regex to fail directly if a lower-case method are used.

$matches = preg_grep('/[^[A-Z0-9]!#$%&\'*\/+\-.^_`|~]/', $value);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

were do we strtoupper? i think we should leave the method unchanged, given how psr-7 is specified. as mentioned above, guzzle psr-7 request does not follow the SHOULD of the spec.


return empty($matches);
});
}

/**
Expand Down