How we Scale & Migrate Continue Watching from DynamoDb to Redis Part -1
It’s the fifth part of a series of our cloud migration journey
In any OTT platform Continue watching is an important feature. People want to start watching from where they have left off, No one remembers how much content they have watched. Also, It’s an important feature where the platform displays a list of content that has not been watched completely.
Implementing the “Continue Watching” feature can be challenging, especially when dealing with large amounts of users watching at the same point in time.
We are sending continue watching events every 5 sec and which is updated at our backend for analytic purposes as well as for continue watching.
There are a few more business logic that I am skipping for this article.
Earlier we are using DynamoDb with a simple structure and it also helps us to scale our system easily.
Here is our structure, Where we store user_id, content_id, progress, and date..
If a user tries to play content we check based on user_id and content_id if there is any progress information is there. If it’s so then we start the player with this location if not it starts from the start.
For Continue Watching rail we need the last 25 content for a user it’s easier just to use the last_updated date and get X no of content for the user id.
We try to find out a similar solution in Azure and we found out that we can use Cosmos Database. It was a perfect match but we ran into a few problems, Cost and Read SLA. Since 95% of API returns in 30 ms and cosmos SLA around 11 ms or so.
We started looking at another solution where we don’t want to spend much time and energy, We know Redis and it’s matching our read SLA, It normally takes 1–2ms max.
There are a few more use cases for optimizing our APIS, But I am skipping them as of now.
Here is a small Python code on how we have used different data structures for the different use cases. Production is a bit different since there are few more business logic.
There are various challenges we have faced while scaling this system, Which we will discuss in future posts.
