As a plugin developer, I'd like to configure a maximum size in bytes for my cache via a documented and supported API

Plugin developers create and consume caches as needed via the com.atlassian.cache.CacheFactory interface. In Confluence, this delegates to the Ehcache implementation in a non-clustered environment, defaulting to

    <defaultCache
        maxElementsInMemory="1000"
        eternal="false"
        timeToIdleSeconds="3600"
        timeToLiveSeconds="0"
        overflowToDisk="false"
        />

The maxElementsInMemory configuration is currently adjustable for Confluence via the com.atlassian.confluence.cache.CacheConfigManager service. This service is not exposed to the plugin container, so you'd have to retrieve the instance from the host context via ComponentLocator or some other hacks.

Since enforcing an object count as a quota burdens the client with ensuring that their serialised objects are roughly of the same size in order to calculate a sensible count, I'd suggest spiking an upgrade to Ehcache 2.5 or above (Confluence 5.1 currently uses the 1.6.0-atlassian-4 fork).

The major new capability of Ehcache 2.5 is enhanced cache resource management, specifically the ability to set the maximum size in bytes of a Cache or CacheManager.

This would allow the configuration of a maxBytesLocalHeap attribute for a specific cache, allowing an (approximated) size in byte quota to be enforced. This setting is much more sensible for (plugin) developers, as the serialised objects normally heavily vary in their size. E.g. Team Calendars caches the net.fortuna.ical4j.model.Calendar object, the size of it depends on the events being attached to this calendar.

Setting this attribute should be exposed to the plugin system for Confluence, e.g. by modifying the CacheConfigManager and making it plugin:available="true". Please ensure that the necessary documentation on DAC is updated, e.g. Confluence Caching Architecture.