Big Data at Betfair

Photo by Igor Ovsyannykov

Photo by Igor Ovsyannykov (https://unsplash.com/ @igorovsyannykov)

Working through Opticore, I worked on a number of Data projects for Betfair. One of the more important ones was a Big Data Operations project.

The public API was taking a beating from account holders and the current process for determining abuse of the API was constantly several weeks behind.

Using a Map Reduce system, we were able to process the daily logs from the large number of API web servers down to an analysis set. This was made available through a dynamic Tableau dashboard and allowed the stakeholder to assess on a daily basis which accounts were genuine and which were guilty of abuse.

The abusing accounts were suspended and blocked within a day. After this developed process was put in place, the pressure on the web servers was drastically reduced resulting in a cost saving from scaling back the infrastructure required to met the previous traffic.


Joe Molloy is a freelance technical consultant, project manager and writer, based in London, UK.

Joe helps start-ups and companies convert their vision and ideas into real world products and services. Joe specialises in helping companies get it done.