Skip to main content
Home  ›  Docs › Feature

CacheAllStreams Data Source for crazy performance

Warning ⚠️: New Docs Available

These are old docs and we haven't found the time to completely move them. You will find comprehensive new docs on docs.2sxc.org.

Introduced in Version 07.02.00

The CacheDataSource will take any amount of in-streams and provide them with the same name as out - and will ensure that it caches everything. 

The source is very smart - so if your in-stream has a query with parameters (like a Id-Filter based on the URL) then each combination will be cached separately. 

This is what it looks like in the Visual Query Designer

How the Caching works

  1. The query engine is very advanced. It will assemble the query and be ready for data-retrieval - but not execute the query yet. 
  2. Each data source will give a cache-relevant key. So a simple data source like App (without any configuration) will just give a "App-F0D6BFB2-36F4-4104-A826-1DFBC77DB42A" (showing the internal guid to keep unique cache-items). A complex data source like a Sql may give a complex one like "SqlDataSource-guid&param1=Person&param2=27" so that the caches can be kept separate. 
  3. Based on the chain of data-sources a unique cache key is constructed, containing all the parameters used in all up-stream data sources. This ensures that as long as the chain is the same and uses the same query parameters, the full-cache-key will stay the same and will retrieve the same cached list.
  4. If the cache exists, it will be used, otherwise the query is actually executed and then cached for further use 

Streams in the CacheAllStreams Data Source

  • In
    • Any streams you want - typical would be a "Default" but you can have many streams. Note that all streams will have the same cache-configuration - so if you watn different caching-rules for various streams then add multiple CacheAllStreams DataSources to the query and configure each as you need it. 
  • Out - it provides the same streams as just came in - just cached

Configuration of the CacheAllStreams

  • Refresh on Source Refresh: This setting will cause the cached data to be refreshed IF any up-stream source says it has newer data. This is mainly used for queries which are based on 2sxc-data, because that data is already cached reliably, and will reliably tell this cache that it has been updated. 
    • Recommendations
      • Set this to True for queries which are based on 2sxc content-items, because these can check for refresh without actually re-loading the data at the source. 
      • Set this to False for queries based on external data (like SQL, RSS, XML, CSV) because these sources will usually re-load all the data from the source to then deliver a new date/time even though the data never changed. 
  • Cache Duration In Seconds: This is a timeout when the cache will become stale - so cached data will be removed after so many seconds. Common values are 86400 (1 day) or 3600 (1 hour). Note that the web-server optimizes the cache, so it may actually remove the data before this if it's never used. 
    • Recommendations
      • Set this to a very high number - like 86400 - when using 2sxc data, because you can control the cache-refresh based on the source-refresh. So if you primarily want to cache the query as it is - but want the query to re-run when data changes (2sxc content-items), then set this to a high number. 
      • Set this to a moderate number - like 3600 for an hour or even just 300 for 5 minutes - if you're accessing external data (SQL, RSS, etc.) which can change frequently. 
      • Set this to a high number - like 36000 for 10 hours - when accessing external data which rarely changes and your primary concern is performanc.