Working through Opticore, I worked on a number of Data projects for Betfair. One of the more important ones was a Big Data Operations project.
The public API was taking a beating from account holders and the current process for determining abuse of the API was constantly several weeks behind.
Using a Map Reduce system, we were able to process the daily logs from the large number of API web servers down to an analysis set. This was made available through a dynamic Tableau dashboard and allowed the stakeholder to assess on a daily basis which accounts were genuine and which were guilty of abuse.
The abusing accounts were suspended and blocked within a day. After this developed process was put in place, the pressure on the web servers was drastically reduced resulting in a cost saving from scaling back the infrastructure required to met the previous traffic.
Joe Molloy is a freelance technical consultant, project manager and writer, based in London, UK.
Joe helps start-ups and companies convert their vision and ideas into real world products and services. Joe specialises in helping companies get it done.