Single Threaded Real-Time Saga
December 25, 2024
A short story
This is a rant! I've been building a real time system in Python for some time now, and it has been an adventure, considering the simplicity of the python language. It's necessary for the system to be efficient with very little downtime.
The project - written in FastAPI ❤️ utilizes the good ol’ python Websockets library for real time communication.
Ideally, when a client accesses the system, they initiate a process, and a unique WebSocket connection is established. This WebSocket connection can be intercepted, allowing a new client to join the same connection.
The system enforces a rule that prevents processing any client requests until the client’s connection to the server has been fully established.
At the initial stages of development, when it was just 1/2/3 concurrent requests, the connection time was insane but I overlooked it (12-19seconds)
Skill Issue or Not? 👤
Recently, The connection time increased significantly, rising from 12-19 seconds to 50 seconds to over 1 minute as the number of connected clients grew.
This renders the system unusable. I really do not think anyone would want to use a system that slow. I personally would not
I honestly couldn’t take this anymore, had to find a way out to make this as fast as possible (<7s).
That’s like an 86% performance improvement.
Rewriting this in Go or any other high-performance language could result in better performance as suggested by @lanreadelowo. They have superior concurrency handling and handle asynchronous operations more efficiently, but time constraints do exists.
Ayodeji and I were casually talking about this problem the other day and he said this.
I accepted the reality and decided to try centrifugo.
Centrifugo is a real-time messaging server that helps you build scalable and efficient real time systems. It manages the connections and handling of messages between the backend application and the connected clients Your backend application mainly handles the business logic while all real time processes are on a different server entirely - centrifugo server.
I intended to list the steps on how to setup centrifugo and get it running in a python environment but that would come in the next post.
Adopting centrifugo reduced the overall Websocket connection time to (2-3seconds). An unbelievable speed up in my opinion. It handles all edge cases that you would be required to handle if you had a custom Websocket setup
- A client’s internet connection is interrupted or becomes unstable, causing the Websocket connection to drop unexpectedly. Centrifugo automatically handles reconnection and publishes missed messages upon reconnection.
- Automatically broadcasts to all connected clients with zero setup
- Inbuilt message queuing preventing data loss
- Centrifugo is designed to handle high-scale systems with distributed nodes, making it easier to scale horizontally without need for any extra configuration from you.
My key takeway
If you’re constrained to a specific language because of time or familiarity, focus on optimizing your use of that language. Explore libraries, tools, or external services that enhance its capabilities and help you achieve your goal efficiently.
It’s not about loyalty to a specific tool or language; it’s about pragmatism and efficiency.
These takeaways align with the views of Rob Pike and Linus Torvalds.
Have an amazing day & Merry Christmas ✨