Geeks With Blogs

News This is the *old* blog. The new one is at blog.sixeyed.com
Elton Stoneman
This is the *old* blog. The new one is at blog.sixeyed.com

The QA lead for an API project that was about to go live asked me if I was sure we were safe against SQL injection attacks. It was nice to be able to answer:

we've had an independent security review. That found no major issues and we fixed all the minor issues. But anyway we're definitely safe against SQL injection - we don't use any SQL databases.

We're fortunate that the transactional data we store isn't heavily inter-related, and the persistent store we use is Azure Blob Storage. Every piece of data is stored as a file in blob storage, and that gives us three very important advantages:

  1. everything is JSON
  2. everything has a unique key
  3. the store is completely dumb

Those three points mean it's trivially easy to swap in a different store, or a different type of store - anything that has basic set and retrieve functionality for strings can be a store. That means we can easily support high load with good performance, and keep the API running even with significant levels of failure in the cloud (like the loss of Azure storage, or all of Azure).

With the core concept that every piece of data is represented as JSON and is accessible from a simple store with a unique key, we can build out an architecture like this:

 

image

 

The API exposes JSON resources from the store, but there are stacked repositories in between the external API and the ultimate source of the data. Those repositories all look the same - get me the JSON with this ID - so although they're stacked, the API response is the same no matter which store does provides it.

The first repository in the stack is a level 1 cache, which will be a memory cache. Memory's limited so there may not be a hit, in which case we try the next repository in the stack, which is a level 2 cache. That could be disk on the local machine (or something more managed like Redis), but level 2 has limited size so again there may not be a hit. Onto the next repo which is Azure blob storage - slower again because of the network call, but with practically unlimited size.

Depending on how we configure the stack for a particular resource, if it needs to go to a lower stack to get a hit, it will (asynchronously) add it to higher stacks on the way back, so when we get the first request for a resource we'll fetch it from blob storage, and add it to the disk and memory caches on the way back up, so the next request for the same resource will be much quicker.

If there's nothing in blob storage, the repository will know where to get it from source - which could be a separate API call, where we do whatever enrichment and transformation we need to do and then save the pre-formed response as a blob.

Performance-wise that gives us good caching for frequently-used resources, while making the most of limited stores like the memory cache. Again, we can configure things differently for different resource types, so for objects which don't change often they may stay in the memory cache for hours before being refreshed, while frequently changing resources may only be cached for one minute (which still saves an awful lot of L2 and L3 access if you have hundreds of requests per second).

It also makes DR very simple. DR in the cloud warrants a post of its own (coming soon), but with this architecture all we need to provide up-to-date data for a separate stack is a background job copying new objects from blob storage to a secondary store – Google Cloud Storage or Amazon S3 or anything else. Because the objects are all JSON, we can inspect them during the copy and fix any links to other blobs, so they use the root path for the secondary store. Then the secondary stack points at the secondary store, and if you lose Azure with your primary stack on it, everything just keeps working.

Blob storage in Azure and other cloud storage options are massively optimised and key-based lookup is quick, even with hundreds of thousands of objects in a container. OpenStack recommend a maximum of 1,000,000 objects per container, which is a pretty fair limit before you need to start sharding between containers.

This architecture doesn't fit every problem, but where you have more reads than writes and your writes are clearly contained, it has a lot of advantages.


Posted on Friday, October 3, 2014 4:30 PM The Cloud , Architecture , Caching , Caching | Back to top


Comments on this post: Databases? Where we're going, we don't need databases

No comments posted yet.
Your comment:
 (will show your gravatar)


Copyright © Elton Stoneman | Powered by: GeeksWithBlogs.net