Caching is your best friend
Disclaimer: This article is based on my current knowledge and understanding. If you have any query or see any improvement, please mention in the comment section
Sumit, we all know caching is good for performance of our API/system. Can you tell us something new? Sure! Imagine cache expires at time T and at same time T, we get 1000 requests, what will happen? cache is expired so all requests needs to be served by API/system without cache which will definitely have performance impact. Such phenomenon is called cache stampede.
What’s the solution here? locking the resource
Below is our solution in .NET using output caching feature
Frame 1: Code, Frame 2: Running API terminal, Frame 3: Testing terminally
We have developed an API which simply increments the counter integer variable and return it in response. We have informed this API endpoint that after every 100 milliseconds if request comes, let it enter our system and execute fully but if request comes within T and T+100 milliseconds, it shouldn’t enter our system and should be served from cache i.e. if counter integer variable at time T is 1, then at T +70 milliseconds, counter variable value should be 1 only but at T + 101 milliseconds , value will be 2.
But what about cache stampede? what if 10K requests comes at T+100 milliseconds. From above testing terminal we can see that in test, we bombarded 10K request to our API concurrently, approximately it takes 1 to 1.3 second to complete this whole test. 1 second = 10 times 100 milliseconds. so at least 10 should be value of counter variable and we see 13 as counter variable value which signifies that we prevented cache stampede otherwise counter variable value would have been closed to 10k.
~ Happy Coding
Note: this 10K request test commands were run manually so we don’t see exact 10 as value for counter variable but at least 10 should be the value because approximately it takes 1 — 1.3 seconds to complete this test.
