Recipe 3.31 Creating an Object Cache

Problem

Your application creates many objects that are expensive to create and/or have a large memory footprint-for instance, objects that are populated with data from a database or a web service upon their creation. These objects are used throughout a large portion of the application's lifetime. You need a way to not only enhance the performance of these objects-and as a result, your application-but also to use memory more efficiently.

Solution

Create an object cache to keep these objects in memory as long as possible, without tying up valuable heap space and possibly resources. Since cached objects may be reused at a later time, you also forego the process of having to create similar objects many times.

You can reuse the ASP.NET cache that is located in the System.Web.Caching namespace or you can build your own lightweight caching mechanism. The See Also section at the end of this recipe provides several Microsoft resources that show you how to use the ASP.NET cache to cache your own objects. However, the ASP.NET cache is very complex and may have a nontrivial overhead associated with it, so using a lightweight caching mechanism like the one shown here is a viable alternative.

The following class, ObjCache, represents a type that allows the caching of SomeComplexObj objects:

using System;
using System.Collections;

public class ObjCache
{
    // Constructors
    public ObjCache( )
    {
        Cache = new Hashtable( );
    }

    public ObjCache(int initialCapacity)
    {
        Cache = new Hashtable(initialCapacity);
    }

    // Fields
    private Hashtable cache = null;

    // Methods
    public SomeComplexObj GetObj(object key)
    {
        if (!cache.ContainsKey(key) || !IsObjAlive(key))
        {
            AddObj(key, new SomeComplexObj( ));
        }

        return ((SomeComplexObj)((WeakReference)cache[key]).Target);
    }

    public object GetObj(object key, object obj)
    {
        if (!cache.ContainsKey(key) || !IsObjAlive(key))
        {
            return (null);
        }
        else
        {
            return (((WeakReference)cache[key]).Target);
        }
    }

    public void AddObj(object key, SomeComplexObj item)
    {
        WeakReference WR = new WeakReference(item, false);

        if (cache.ContainsKey(key))
        {
            cache[key] = WR;
        }
        else
        {
            cache.Add(key, WR);
        }
    }

    public void AddObj(object key, object item)
    {
        WeakReference WR = new WeakReference(item, false);

        if (cache.ContainsKey(key))
        {
            cache[key] = WR;
        }
        else
        {
            cache.Add(key, WR);
        }
    }

    public bool IsObjAlive(object key)
    {
        if (cache.ContainsKey(key))                                                
        {
            return (((WeakReference)cache[key]).IsAlive);
        }
        else
        {
            return (false);
        }
    }

    public int AliveObjsInCache( )
    {
        int count = 0;

        foreach (DictionaryEntry item in cache)
        {
            if (((WeakReference)item.Value).IsAlive)
            {
                count++;
            }
        }

        return (count);
    }

    public int ExistsInGeneration(object key)
    {
        int retVal = -1;

        if (cache.ContainsKey(key) && IsObjAlive(key))
        {
            retVal = GC.GetGeneration((WeakReference)cache[key]);
        }

        return (retVal);
    }

    public bool DoesKeyExist(object key)
    {
        return (cache.ContainsKey(key));
    }

    public bool DoesObjExist(object complexObj)
    {
        return (cache.ContainsValue(complexObj));
    }

    public int TotalCacheSlots( )
    {
        return (cache.Count);
    }
}

The SomeComplexObj class can be replaced with any type of class you choose. For this recipe, we will use this class, but for your code, you can change it to whatever class or structure type you need.

The SomeComplexObj is defined here (realistically, this would be a much more complex object to create and use; however, for the sake of brevity, this class is written as simply as possible):

public class SomeComplexObj
{
    public SomeComplexObj( )        {}

    private int idcode = -1;

    public int IDCode
    {
        set{idcode = value;}
        get{return (idcode);}
    }
}

ObjCache, the caching object used in this recipe, makes use of a Hashtable object to hold all cached objects. This Hashtable allows for fast lookup when retrieving objects and generally for fast insertion and removal times. The Hashtable object used by this class is defined as a private field and is initialized through its overloaded constructors.

Developers using this class will mainly be adding and retrieving objects from this object. The GetObj method implements the retrieval mechanism for this class. This method returns a cached object if its key exists in the Hashtable and the WeakReference object is considered to be alive. An object that the WeakReference type refers to has not been garbage collected. The WeakReference type can remain alive long after the object to which it referred is gone. An indication of whether this WeakReference object is alive is obtained through the read-only IsAlive property of the WeakReference object. This property returns a bool indicating whether this object is alive (true) or not (false). When an object is not alive, or when its key does not exist in the Hashtable, this method creates a new object with the same key as the one passed in to the GetObj method and adds it to the Hashtable.

The AddObj method implements the mechanism to add objects to the cache. This method creates a WeakReference object that will hold a weak reference to our object. Each object in the cache is contained within a WeakReference object. This is the core of the caching mechanism used in this recipe. A WeakReference that references an object (its target) allows that object to later be referenced through itself. When the target of the WeakReference object is also referenced by a strong (i.e., normal) reference, the GC cannot collect the target object. But if no references are made to this WeakReference object, the GC can collect this object to make room in the managed heap for new objects.

After creating the WeakReference object, the Hashtable is searched for the same key that we want to add. If an object with that key exists, it is overwritten with the new object; otherwise, the Add method of the Hashtable class is called.

The ObjCache class has been written to cache either a specific object type or multiple object types. To do this, a method called GetAnyTypeObj has been added that returns an object. Additionally, the AddObj method is overloaded to accept an object as its second parameter type. The following code uses the strongly typed GetObj method to return a SomeComplexObj object:

SomeComplexObj SCO2 = OC.GetObj("ID2");

The following code uses the generic GetAnyTypeObj method to return some other type of object:

Obj SCO2 = (Obj)OC.GetAnyTypeObj("ID2");
if (SCO2 == null)
{
    OC.AddObj("ID2", new Obj( ));
    SCO2 = (Obj)OC.GetAnyTypeObj("ID2");
}

where Obj is an object of any type. Notice that it is now the responsibility of the caller to verify that the GetObj method does not return null.

Quite a bit of extra work is required in the calling code to support a cache of heterogeneous objects. More responsibility is placed on the user of this cache object, which can quickly lead to usability and maintenance problems if not written correctly.

The code to exercise the ObjCache class is shown here:

// Create the cache here
ObjCache OC = new ObjCache( );

public void TestObjCache( )
{
    OC.AddObj("ID1", new SomeComplexObj( ));
    OC.AddObj("ID2", new SomeComplexObj( ));
    OC.AddObj("ID3", new SomeComplexObj( ));
    OC.AddObj("ID4", new SomeComplexObj( ));
    OC.AddObj("ID5", new SomeComplexObj( ));

    Console.WriteLine("\r\n--> Add 5 weak references");
    Console.WriteLine("OC.TotalCacheSlots = " + OC.TotalCacheSlots( ));
    Console.WriteLine("OC.AliveObjsInCache = " + OC.AliveObjsInCache( ));
    Console.WriteLine("OC.ExistsInGeneration('ID1') = " + 
      OC.ExistsInGeneration("ID1"));

    ////////////// BEGIN COLLECT //////////////
    GC.Collect( );
    GC.WaitForPendingFinalizers( );
    //////////////  END COLLECT  //////////////

    Console.WriteLine("\r\n--> Collect all weak references");
    Console.WriteLine("OC.TotalCacheSlots = " + OC.TotalCacheSlots( ));
    Console.WriteLine("OC.AliveObjsInCache = " + OC.AliveObjsInCache( ));

    OC.AddObj("ID1", new SomeComplexObj( ));
    OC.AddObj("ID2", new SomeComplexObj( ));
    OC.AddObj("ID3", new SomeComplexObj( ));
    OC.AddObj("ID4", new SomeComplexObj( ));
    OC.AddObj("ID5", new SomeComplexObj( ));

    Console.WriteLine("\r\n--> Add 5 weak references");
    Console.WriteLine("OC.TotalCacheSlots = " + OC.TotalCacheSlots( ));
    Console.WriteLine("OC.AliveObjsInCache = " + OC.AliveObjsInCache( ));

    CreateObjLongMethod( );
    Create135( );
    CollectAll( );
}

private void CreateObjLongMethod( )
{
    Console.WriteLine("\r\n--> Obtain ID1");
    if (OC.IsObjAlive("ID1"))
    {
        SomeComplexObj SCOTemp = OC.GetObj("ID1");
        SCOTemp.IDCode = 100;
        Console.WriteLine("SCOTemp.IDCode = " + SCOTemp.IDCode);
    }
    else
    {
        Console.WriteLine("Object ID1 does not exist...Creating new ID1...");
        OC.AddObj("ID1", new SomeComplexObj( ));                                

        SomeComplexObj SCOTemp = OC.GetObj("ID1");
        SCOTemp.IDCode = 101;
        Console.WriteLine("SCOTemp.IDCode = " + SCOTemp.IDCode);
    }
}

private void Create135( )
{
    Console.WriteLine("OC.ExistsInGeneration('ID1') = " + 
      OC.ExistsInGeneration("ID1"));
    Console.WriteLine("\r\n--> Obtain ID1, ID3, ID5");
    SomeComplexObj SCO1 = OC.GetObj("ID1");
    SomeComplexObj SCO3 = OC.GetObj("ID3");
    SomeComplexObj SCO5 = OC.GetObj("ID5");
    SCO1.IDCode = 1000;
    SCO3.IDCode = 3000;
    SCO5.IDCode = 5000;
    Console.WriteLine("OC.ExistsInGeneration('ID1') = " + 
      OC.ExistsInGeneration("ID1"));

    ////////////// BEGIN COLLECT //////////////
    GC.Collect( );
    GC.WaitForPendingFinalizers( );
    //////////////  END COLLECT  //////////////

    Console.WriteLine("\r\n--> Collect all weak references");
    Console.WriteLine("OC.TotalCacheSlots = " + OC.TotalCacheSlots( ));
    Console.WriteLine("OC.AliveObjsInCache = " + OC.AliveObjsInCache( ));
    Console.WriteLine("OC.ExistsInGeneration('ID1') = " 
      + OC.ExistsInGeneration("ID1"));

    Console.WriteLine("SCO1.IDCode = " + SCO1.IDCode);
    Console.WriteLine("SCO3.IDCode = " + SCO3.IDCode);
    Console.WriteLine("SCO5.IDCode = " + SCO5.IDCode);

    Console.WriteLine("\r\n--> Get ID2, which has been collected.  ID2 Exists ==" +     
      OC.IsObjAlive("ID2"));
    SomeComplexObj SCO2 = OC.GetObj("ID2");
    Console.WriteLine("ID2 has now been re-created.  ID2 Exists == " +
      OC.IsObjAlive("ID2"));
    Console.WriteLine("OC.AliveObjsInCache = " + OC.AliveObjsInCache( ));
    SCO2.IDCode = 2000;
    Console.WriteLine("SCO2.IDCode = " + SCO2.IDCode);

    ////////////// BEGIN COLLECT //////////////
    GC.Collect( );
    GC.WaitForPendingFinalizers( );
    //////////////  END COLLECT  //////////////

    Console.WriteLine("\r\n--> Collect all weak references");
    Console.WriteLine("OC.TotalCacheSlots = " + OC.TotalCacheSlots( ));
    Console.WriteLine("OC.AliveObjsInCache = " + OC.AliveObjsInCache( ));
    Console.WriteLine("OC.ExistsInGeneration('ID1') = " + 
      OC.ExistsInGeneration("ID1"));
    Console.WriteLine("OC.ExistsInGeneration('ID2') = " + 
      OC.ExistsInGeneration("ID2"));
    Console.WriteLine("OC.ExistsInGeneration('ID3') = " + 
      OC.ExistsInGeneration("ID3"));
}

private void CollectAll( )
{
    ////////////// BEGIN COLLECT //////////////
    GC.Collect( );
    GC.WaitForPendingFinalizers( );
    //////////////  END COLLECT  //////////////

    Console.WriteLine("\r\n--> Collect all weak references");
    Console.WriteLine("OC.TotalCacheSlots = " + OC.TotalCacheSlots( ));
    Console.WriteLine("OC.AliveObjsInCache = " + OC.AliveObjsInCache( ));
    Console.WriteLine("OC.ExistsInGeneration('ID1') = " + 
      OC.ExistsInGeneration("ID1"));
    Console.WriteLine("OC.ExistsInGeneration('ID2') = " + 
      OC.ExistsInGeneration("ID2"));
    Console.WriteLine("OC.ExistsInGeneration('ID3') = " + 
      OC.ExistsInGeneration("ID3"));
    Console.WriteLine("OC.ExistsInGeneration('ID5') = " + 
      OC.ExistsInGeneration("ID5"));
}

The output of this test code is shown here:

--> Add 5 weak references
OC.TotalCacheSlots = 5
OC.AliveObjsInCache = 5
OC.ExistsInGeneration('ID1') = 0

--> Collect all weak references
OC.TotalCacheSlots = 5
OC.AliveObjsInCache = 0

--> Add 5 weak references
OC.TotalCacheSlots = 5
OC.AliveObjsInCache = 5

--> Obtain ID1
SCOTemp.IDCode = 100
OC.ExistsInGeneration('ID1') = 0

--> Obtain ID1, ID3, ID5
OC.ExistsInGeneration('ID1') = 0

--> Collect all weak references
OC.TotalCacheSlots = 5
OC.AliveObjsInCache = 3
OC.ExistsInGeneration('ID1') = 1
SCO1.IDCode = 1000
SCO3.IDCode = 3000
SCO5.IDCode = 5000

--> Get ID2, which has been collected.  ID2 Exists == False
ID2 has now been re-created.  ID2 Exists == True
OC.AliveObjsInCache = 4
SCO2.IDCode = 2000

--> Collect all weak references
OC.TotalCacheSlots = 5
OC.AliveObjsInCache = 4
OC.ExistsInGeneration('ID1') = 2
OC.ExistsInGeneration('ID2') = 1
OC.ExistsInGeneration('ID3') = 2

--> Collect all weak references
OC.TotalCacheSlots = 5
OC.AliveObjsInCache = 0
OC.ExistsInGeneration('ID1') = -1
OC.ExistsInGeneration('ID2') = -1
OC.ExistsInGeneration('ID3') = -1
OC.ExistsInGeneration('ID5') = -1

Discussion

Caching involves storing frequently used objects in memory that are expensive to create and recreate for fast access. This technique is in contrast to recreating these objects through some time-consuming mechanism (e.g., from data in a database or from a file on disk) every time they are needed. By storing frequently used objects such as these-so that we do not have to create them nearly as much-we can further improve the performance of the application.

When deciding which types of items can be cached, you should look for objects that take a long time to create and/or initialize. For example, if an object's creation involves one or more calls to a database, to a file on disk, or to a network resource, it can be considered as a candidate for caching. In addition to selecting objects with long creation times, these objects should also be frequently used by the application.Selection depends on a combination of the frequency of use and the average time for which it is used in any given usage. Objects that remain in use for a long time when they are retrieved from the cache may work better in this cache than those that are frequently used but for only a very short period of time.

If you know that the number of cached objects will be equal to or less than 10, you can substitute a ListDictionary for the Hashtable. The ListDictionary is optimized for 10 items or fewer. If you are unsure of whether to pick a ListDictionary or a Hashtable, consider using a HybridDictionary object instead. A HybridDictionary object uses a ListDictionary when the number of items it contains is 10 or fewer. When the number of contained items exceeds 10, a Hashtable object is used. The switch from a ListDictionary to a Hashtable involves copying the elements from the ListDictionary to the Hashtable. This can cause a performance problem if this type of collection will usually contain more than 10 items. In addition, if the initial size of a ListDictionary is set above 10, a Hashtable is used by the HybridDictionary exclusively, again reducing the effectiveness of the HybridDictionary.

If you do not want to overwrite cached items having the same key as the object you are attempting to insert into the cache, the AddObj method must be modified. The code for the AddObj method could be modified to this:

public void AddObj(object key, SomeComplexObj item)
{
    WeakReference WR = new WeakReference(item, false);
    if (!cache.ContainsKey(key))
    {
        cache.Add(key, WR);
    }
    else
    {
        throw (new Exception("Attempt to insert duplicate keys."));
    }
}

We could also add a mechanism to calculate the cache-hit-ratio for this cache. The cache-hit-ratio is the ratio of hits-every time an existing object is requested from the Hashtable-to the total number of calls made to attempt a retrieval of an object. This can give us a good indication of how well our ObjCache is working. The code to add to this class to implement a cache-hit-ratio is shown highlighted here:

private float numberOfGets = 0;
private float numberOfHits = 0;

public float HitMissRatioPcnt( )
{
    if (numberOfGets == 0)
    {
        return (0);
    }
    else
    {
        return ((numberOfHits / numberOfGets) * 100);
    }
}

public SomeComplexObj GetObj(object key)
{
    ++numberOfGets;

    if (!cache.ContainsKey(key) || !IsObjAlive(key))
    {
        AddObj(key, new SomeComplexObj( ));
    }
    else
    {
        ++numberOfHits;
    }

    return ((SomeComplexObj)((WeakReference)cache[key]).Target);
}

The numberOfGets field tracks the number of calls made to the GetObj retrieval method. The numberOfHits field tracks the number of times that an object to be retrieved exists in the cache. The HitMissRatioPcnt method returns the numberOfHits divided by the numberOfGets as a percentage. The higher the percent, the better our cache is operating (100% is equal to a hit every time the GetObj method is called). A lower percentage indicates that this cache object is not working efficiently (0% is equal to a miss every time the GetObj method is called). A very low percentage indicates that the cache object may not be the correct solution to your problem or that you are not caching the correct object(s).

The WeakReference objects created for the ObjCache class do not track objects after they are finalized. This would add much more complexity than is needed by this class. Moreover, we would have the responsibility of dealing with resurrected objects that are in an undefined state. This is a dangerous path to follow.

Remember, a caching scheme adds complexity to your application. The most a caching scheme can do for your application is to enhance performance and possibly place less stress on memory resources. You should consider this when deciding whether to implement a caching scheme such as the one in this recipe.