Menu Navigation Menu

Mobile mall makes shopping social again

Wanelo is an ecommerce site with over 30 million products curated by stylish people. Its catalog includes 500,000 stores from big brands to tiny independent boutiques and sellers.

  • Scaling
  • DevOps
  • Training
  • Full-Stack Development
  • Staff Augmentation

The Challenge

Wanelo came to us for help getting ready to handle a huge increase in traffic.

They had already seen traffic increase by a factor of about 5x over the previous year, and they knew that their existing systems wouldn’t be able to handle another increase of even 2x, let alone 5x.

Together we scaled their infrastructure to successfully handle a 7x increase in traffic from their previous peak capacity and architecting and implementing a database system with plenty of additional room to grow if needed. To do this we designed, built, and deployed a massively parallel system to process more data every day than they had previously processed in the course of weeks. We also trained junior and mid-level developers to make them more effective and productive team members.

Over the course of almost two years, we saw Wanelo grow their engineering team from about 10 to about 20 and increase the traffic they could handle from about 600 requests/second up to about 4200 requests/sec.

The Solution

Planning to handle dramatic scaling

At the beginning, they were sure they needed help getting ready to handle a huge increase in traffic. They had already seen traffic increase by a factor of about 5x over the previous year, and they knew that their existing systems wouldn’t be able to handle another increase of even 2x, let alone 5x.

We spent a few months discussing the current architecture, examining the strategies used by companies like Twitter to handle dramatic scaling, and planning out a new system that we thought would be able to handle the new traffic.

The biggest problem that we could foresee was that user data was growing so fast it would soon overwhelm the largest database server that we could buy, at any price. After the planning phase was over, we spent a few months implementing.

We built a custom web service with a JSON API to replace their existing database of product lists saved by users. We designed a system for sharding the data, writing a web service with an HTTP API that exposed the data across all shards, and then wrote a client library that acted as a drop-in replacement for ActiveRecord for the newly sharded data. The data set was sharded across hundreds of logical nodes, hosted on dozens of physical nodes, with plenty of room to grow more in the future.

Fast display of diverse product images

Wanelo needed many sizes of images, but Ruby was too slow to resize images on the fly. We had been working around it by slowly resizing images in Ruby to every possible size we might use, even for sizes we never actually used in real life, which was costly both in time and in storage fees.

During a freeform hack day, one of the devs built a proof of concept resize-on-demand image service in Go. We spent a month or two helping fix bugs in that service, writing Chef code to automate deploying it, and reworking the existing systems to use the new service.

Accurate catalog information on over 30 million products from 550,000 stores

Wanelo wanted up-to-date information about whether the products in their database were in stock at the online stores that sold them. We worked with them on a system that could download and update information about hundreds of thousands of products from thousands of stores on a nightly basis. We built a system in Ruby to handle fetching, processing, and updating entire store catalogs in parallel, and launched it successfully.

The Outcome

Pairing and training sets team up for success

As Wanelo grew their team, they hired new and less-experienced developers. We began spending more time pairing and training than implementing.

Software development includes many skills that aren’t taught in CS departments, and web development involves many technologies that aren’t covered by a CS degree. Most of the pairing and training time was focused on how to understand the Ruby and Javascript frameworks that Wanelo was already using, how to plan and implement new features in an environment of rapidly changing priorities, and how to research and debug complex existing systems without getting stuck.

We helped train their team, assisted with researching and debugging problems that were unique to Wanelo’s systems, and helped establish patterns of communication and collaboration between product management, designers, and developers.

Partner with Cloud City

Internal projects can be far more complicated than anticipated, even for software companies. By bringing Cloud City into the picture, Fastly completed a critical project and benefited from the partnership. Cloud City is a human-centered, empathetic software development team that was invested in Fastly’s long-term health.

If you’re interested in learning more about how Cloud City can help you reach your goals and execute your dreams fill out the form to the side, or email us at results@cloudcity.io

Let's chat!