Configuring Server-Side Cache for PageSpeed
In order to rewrite resources, PageSpeed must cache them server-side.A file-system based cache is always employed on each server. Memcached may be used as a scalable network-accessible cache in addiiton to the file cache. The file cache is always required since files larger than 1Mb cannot be stored on memcached, and it is important to optimize large images. Finally, a per-process in-memory LRU cache and/or an interprocess shared-memory cache can be configured for rapid access to small objects.
Configuring the File Cache
PageSpeed must be configured with a path where it can write cache files, tuned to limit the amount of disk space consumed. On Linux, the file cache can be placed on a tmpfs partition or on a physical disk. The file cache has a built-in LRU mechanism to remove old files, targeting a certain total disk space usage, and a certain interval for the cleanup process. An example configuration is:
# Note that the configured path must be read/writeable for the IIS application-pool identity! pagespeed FileCachePath c:\PagespeedCache pagespeed FileCacheSizeKb 102400 pagespeed FileCacheCleanIntervalMs 3600000 pagespeed FileCacheInodeLimit 500000
It is important to note that
FileCacheInodeLimit do not define absolute limits on the cache size and inode count. The cache cleaning process will run at the time interval defined by
FileCacheCleanIntervalMs, and will only initiate cleaning if the cache size exceeds
FileCacheSizeKb or the cache inode count exceeds
FileCacheInodeLimit. When cache cleaning is initiated, the oldest files in the cache will be removed until the cache size is under
0.75 * FileCacheSizeKb and the inode count is under
0.75 * FileCacheInodeLimit.
PageSpeed previously reserved another file-path for future use as a shared database in a multi-server environment. This is no longer in the plan, and
GeneratedFilePrefix now generates a deprecation warning.
Configuring the in-memory LRU Cache
To optimize performance, a small in-memory write-through LRU cache can be instantiated in each server process. Note that in Apache's pre-fork mode this means dozens of processes, so the total memory consumed (
LRUCacheKbPerProcess * num_processes) must fit into the capabilities of the host machine. Nginx typically runs with many fewer processes, so a larger
LRUCacheKbPerProcess is appropriate there. The
LRUCacheByteLimit is the limit on how large a cache entry the LRU cache will accept. A sample configuration is:
- IIS WebSpeed
pagespeed LRUCacheKbPerProcess 1024 pagespeed LRUCacheByteLimit 16384
pagespeed LRUCacheKbPerProcess 8192 LRUCacheByteLimit 16384
Configuring the Shared Memory Metadata Cache
Note: Deprecated as of IISWebSpeed 2.0
As part of its operation, PageSpeed stores summaries of how to apply optimizations to web pages as part of a metadata cache. Metadata entries are small and frequently accessed. They should ideally be stored in local memory and shared across server processes, as opposed to on disk or a memcached server (see below). That is exactly what the shared memory metadata cache does, and it is the best place to cache your metadata entries.
If this cache is enabled, metadata will no longer be written to the filesystem cache, significantly improving metadata cache performance, but metadata information will be lost upon server restart. This will require resources to be reoptimized for each restart. If a memcached cache is available, cache entries will be written through to the memcached cache so that multiple PageSpeed servers can share metadata and the metadata cache will survive a server restart.
This feature is enabled using the
CreateSharedMemoryMetadataCache directive. This directive takes two arguments. The first is the exact string given as the argument to
FileCachePath in any virtual host where you want this cache active. The second is the size of the cache in kilobytes. Unlike the LRU cache, this cache is shared among all server processes, so far larger values are possible. For example:
pagespeed FileCachePath /var/cache/pagespeed/ pagespeed CreateSharedMemoryMetadataCache /var/cache/pagespeed/ 51200
will create a 50-megabyte metadata cache used for both www.example.com and alt.example.com, and shared among all server processes.
You can see how effective this layer of cache is at the PageSpeed statistics page, where at the bottom of the page every shared memory cache will be listed, including in particular information on its hit rate and how full it is (blocks used).
To enable memcached, specify the list of memcached servers with a comma-separated list of hosts and ports. If the port is omitted, the default memcached port of 11211 is assumed. PageSpeed's memcached integration by uses a background thread for communicating with the memcached servers. This allows PageSpeed to batch multiple Get requests into a single MultiGet request to memcached, which improves performance and reduces network round trips.
pagespeed MemCachedServers "host1:port1,host2:port2,host3:port3"
When you use memcached with PageSpeed, many vital memcached statistics can be found at the statistics pages.
These statistics are taken both from the PageSpeed perspective, aggregating all memcacheds as viewed from a single server...
...and also from the perspective of each memcached server, aggregating activity from each memcached client including all PageSpeed instances.
memcached server host1:6765 version 1.4.2 pid 1132 up 343306 seconds
memcached server host2:6765 version 1.4.2 pid 6568 up 343278 seconds
By default, PageSpeed uses a 50 millisecond (50,000 microsecond) timeout for cache operations. If the timeout is exceeded more than 4 times in a 30 second span, PageSpeed assumes that memcached is not healthy and will stop optimizing resources for 30 seconds before trying again.
This 50 millisecond timeout default is tuned for memcached running on the same local network as PageSpeed. For wide area networks, or for alternative implementations and proxies of the memcache protocol such as couchbase or moxi-server, it may be necessary to increase the I/O timeout. Please monitor the statistic 'memcache_timeouts' to help tune the timeouts. Select a new timeout with the
pagespeed MemcachedTimeoutUs timeout_in_microseconds
In-depth blog post
A cool in-depth blog post by Joshua Marantz about how Google's PageSpeed SDK uses caching internally for optimization: http://modpagespeed.jmarantz.com/2012/12/caching-in-modpagespeed.html.