Ready to manage 10 mn concurrent users in IPL: Dream Sports CTO Amit Sharma

'We implemented data-driven engineering with ML, AI and analytics across all our brands at Dream Sports very early in our automation journey', said Sharma

For the $8-billion sports technology company Dream Sports every Indian Premier League season in recent years has been a stepping stone to enhancing scale. Amid a busy schedule at the start of the season, Dream Sports (Dream11’s parent company) Chief Technology Officer Amit Sharma talks to Deepsekhar Choudhury about the challenges of scale, deploying artificial intelligence and machine learning to protect users from bad behaviour, and more. Edited excerpts:

What are currently your biggest tech challenges?

We recently took up the mammoth task of migrating the Dream11 app to a React Native implementation and soon will be rolling it out to our users. With React Native, we can share code between Android and iOS platforms. This will enable us to further scale the platform through quicker development and deployment of features that our users love, while maintaining consistency across platforms. It will also help us in hiring top talent from a much larger talent pool with proficiency in Javascript, which is one of the world’s most popular programming languages.

This shift was initiated in September 2021, backed by in-house learnings from FanCode, our premier sports content and commerce platform, using React Native for the last three years. A special team was deployed to enable the transition and collaboration -- some of them being the core contributors of React Native.

Did you have to execute any tech hacks to handle surges during the IPL? 

In previous seasons, Dream11 saw up to 5.5 million concurrent users and we are seeing that mark getting breached this season. Some of our services get more than 80 million requests per minute. All of this means the system needs to be able to handle unexpected surges.

At our scale, we cannot do hacks to sustain the traffic. We go through a proper process in which load/stress testing is an integral part of any release. To prepare ourselves for massive scale, we have been relying on cloud testing and scaffolding along with advanced tools. Most importantly, we analyse the findings from past IPL seasons, share the results and implement our learnings to make each tournament better than the last. Now, we are ready to manage a projected 10 million concurrent users in the TATA IPL 2022.

How do you manage millions of users playing real money games simultaneously?

Supporting over 5.5 million concurrent users with real-time user experience is definitely a unique scale problem. We have more than 100 micro services, each serving its specific purpose. Additionally, we have more than 20 teams modifying these services every week and shipping changes to help increment many product features in parallel.

To help support our data-driven culture, we are investing in multi-variant (A/B) testing in shipping these changes to different cohorts of similar users, using and validating each hypothesis without overlap. We are also investing a lot in the experience of our developers so they can create and destroy environments to develop, test and ship changes to production faster.

What’s happening at Dream11 on the artificial intelligence (AI) and machine learning (ML) front?

We implemented data-driven engineering with ML, AI and analytics across all our brands at Dream Sports very early in our automation journey. This includes our fan engagement product FanCode, which uses AI and ML backend framework to provide a personalised user experience to sports fans, as well as our payment solution DreamPay, which uses AI and Big Data to make runtime decisions that increase the payment success rate for our users. 

We also build many home-grown solutions that address key product-related challenges such as user experience analysis, scale management, mobile app automation, security and FairPlay. Data Highway, our in-house analytics platform, allows us to have more control over data, heightens safety and flexibility, maximises opportunity costs, and helps us scale our services.

How do you ring-fence the platform from players who might try to game the system? 

Multiple teams are involved in the process of analysing user behaviour and preferences to ensure they get the best possible sports experience in the safest way possible. This includes developing ML models to detect fraud or fake accounts on the platform. For instance, Dream11 has a specialised in-house fraud detection system called FENCE (Fairplay Ensuring Network Chain Entity) in place so that users participating in every contest, including the paid ones, win in a fair, square and transparent manner. It is powered by a graph database that’s responsible for processing and maintaining all models and heuristics, so that fairplay violations are detected in time and efficiently.

We also invest a lot to automate runtime prevention, detection and response for any suspected malicious attacks. We are investing heavily in predictive analytics and anomaly detection using ML capabilities.   

What are some of the tech challenges you have had to overcome for tier-II cities, where devices are cheaper, and bandwidth lower? 

The Indian sports market stands at 130 million users at present and there is no doubt that in a country of a billion sports fans, this number will grow significantly soon. Tier-II cities already contribute much of Dream11’s user base of over 120 million, thanks to growing smartphone penetration and accessibility to the internet.

As we are a data-driven company, we test our application usability and performance on the real-world traffic pattern, which includes low bandwidth networks. We make sure that our client, as well as server applications, are well optimised to let users use our application under all conditions.