How to build video streaming platform like Netflix.

Due to covid-19 situation, some colleges/schools quickly adopted online content platforms for lecture videos but many were not able to do that transition. One can completely understand this situation as cost of app development or platform development is high. so we thought of building free video streaming platform for such schools/colleges using minimum resources. Moreover, this platform will run even if they have a single computer with standard ram size.

Let’s define problem statement:

We want to build a platform for schools so that lecture videos recorded by teacher can be streamed to students using web interface. Imagine Netflix but with lecture videos. Simple!

Let’s understand scale here. We don’t want to scale globally like Netflix but at same time we can’t use resources (hardware like no. of servers ) which Netflix uses.

Imagine a school with 5000 kids from Standard 1st to 10th . So no. of users will be 5000. Every school has a computer lab these days, we can pick one computer with standard ram size as server for running our frontend and backend. Sumit, where are you going to keep videos? Either they can be stored in another computer from computer lab or common dropbox account.

so challenge may not look like scaling for million users but working for few thousand users with limited hardware makes it look similar 😉.

Before I go ahead and explain how we solved it. I will like to introduce my teammate for this side project. Gagan. He is very good friend of mine and extremely talented Software Engineer. We broke this entire platform into frontend and backend. I was responsible for backend and Gagan took frontend.

Here is my understanding of how Netflix solved streaming problem. They would break videos into segments,store it accordingly and give it to client. This way entire video needn’t go to client but only chunks/segments are transferred over the wire. They may even have cdn for region specific and all. but heart of solution is breaking videos into chunk and sending it over to client.

In our case, we were pretty sure that storing those fragments, having cdn and all is not possible for sure.

we came with following solution:

Basic architecture

• Client will make request for a lecture video file.
• Files can be stored on other system or dropbox, that’s fine. Backend server will fetch it in chunk.
• Once backend server gets chunks, it will stream those chunks over websocket to client without storing it anywhere(websocket protocol establishes persistent connection between server and client unlike HTTP). New chunk is available, again websocket server will forward chunk to client over websocket (this is the reason why persistent connection is so important in this context).
• Once all chunks are fetched by server and given same to client, server will close connection.

Did you see what happened? using raw websocket server and fetching file in chunks, we were able to somehow replicate segment streaming what Netflix does and without actually storing those video segments anywhere in database or queue.

This way our websocket server or computer labs’ ram was not used that much since we didn’t load entire file in memory(RAM) and increased scalability of the server so that it can server more requests per second. And If you want to know how memory affects performance of server, do comment below ☺.

I highly recommend to check below demo video to really understand and visualise what sending chunk over websocket really looks like and how those chunks are getting stitched together so that end user can play video.

I will definitely push Gagan to write more on how he stitched those raw binary array coming from backend to video content. We both were working on video format for first time and we literally have spend weekend, breaking our head to see where are chunks going and all. In video streaming, format of video, codec of mimetype play vital role. One needs to understand how decoding is done at browser end.

Is this solution silver bullet for all streaming service? Hahaha. NO

Few things we want to inform you:

• In Netflix, you can jump in any section of video like go forward or backward. In our solution, you can go only in backward. There is concept of range headers which tells which portion of video needs to be fetched. Right now with my understanding I feel we can’t do that as we are literally streaming and we are not storing anything so it is tough to see forward and get content. Best way to understand this is can you look forward in live videos?No right? I see same thing happening here. We are thinking whether to solve this or not. According to domain, school wants kids to watch entire lecture and not speed forward and skip.(Another example of business dictating feature is you won’t see NoSql in finance domain because transactions are tough to achieve on NoSql DB. Similarly we can understand for schools or educational domain about this feature.
• We want to fetch chunks based on internet speed of end user. This is doable. Gagan and I have a implementation in our minds. Reason we want to implement this is because we want to optimize platform for all users with different internet speeds.
• Proper login screen and signup flow are left. Right now , our focus was to stream video but we will definitely tske care of this.

Hope you enjoyed reading this. We will be putting code on GitHub. So no worries, you can extend the example with more features. One ask we have from reader is if you know any such school or college, do let us know we can help them through this platform. 🙌🏻😊