Ticket #18 (new task)

Opened 10 months ago

Last modified 4 months ago

Filters that convert outgoing data should optionally cache it

Reported by: mahlon Assigned to: mgranger
Priority: major Milestone: ThingFish 0.3 - The "N-Robot" Release
Component: ThingFish Keywords: thingfish filters volatile metadata resource
Cc: Depends on:

Description

Filters can create new arbitrary data against incoming uploads (previews for large images) and downloads (conversion and cache file format) but the filter framework doesn't have access to @metastore, @filestore, or @config.

This is by design, as direct manipulation/storage of resources would overcomplicate error edge cases, or force them to be re-implemented for each filter. (For example, filestore quota checks.)

Filters will need to:

  • Only create new content when appropriate
    • conversion filters need to check to see if the conversion already took place in the past, and offer the cached copy instead of re-generating
  • Optionally mark content as volatile
    • An image preview would probably be static, as applications might rely on its existence. A resource that is a differing format (a jpg version of a png) that could be regenerated would be marked as 'volatile', for a potential future cleanup sweep from an admin handler or something.
    • Or just to answer the question, "how much space is the filestore using in data that is cache?"
  • Signal that new content is waiting somewhere for addition
    • Perhaps storing stuff in the response.data scratch space, for daemon/defaulthandler to pick up on and do the actual storage after the original upload is successful -- or...
    • A new ThingFish::Cache class, that acts as a bridge from Filters to storage. By default, all our volatile data will be stored in the regular FileStore?. Abstracting it out would allow volatile stuff to be dumped directly into something like memcache.

Change History

12/04/07 08:00:46 changed by mahlon

Furthermore, the request and response case are separate problems.

Data generated on Request is generally data that wouldn't really be 'volatile' -- it's just more data that happens to be linked to a parent object.

ThingFish::Request could just make @entity_bodies an accessor for this case.

12/20/07 07:04:34 changed by mahlon

  • milestone changed from ThingFish 0.2 - The "Butterhand" Release to ThingFish 0.3 - The "N-Robot" Release.

06/19/08 07:56:21 changed by mahlon

  • depends_on changed.
  • summary changed from Filters that create new data need to flag it for storage to Filters that convert outgoing data should optionally cache it.

The incoming "related" resources have been implemented (r560), this ticket is now just for a ThingFish::Cache framework.