RUN-WithIT and SAM – a Case Study

SAM needed to know they were ready
for the largest news agencies in the world.


"When your customers are the largest news brands in the world, they demand high performance and reliability, which is exactly what the experts at RUN-WithIT helped us achieve and surpass."
James Neufeld, SAM Founder and CEO
SAM (Social Asset Management) called on RUN-WithIT when the SAM team was close to releasing their social media content management system to some of the largest newsrooms and agencies in the world. SAM CTO, Sean Feehan, wanted to know they were ready.


The SAM system enables teams in the news business to quickly gather content from any social network, sort, validate, stylize and present it as an embed in the newsroom site. During breaking news, SAM’s embeds needed to keep up with the traffic on the world’s most visited news sites. Performance during these peaks defines their customers’ business.
To make it in this space, it was critical SAM got it right. Their team of 4 was charged with innovating this rich client data delivery system with world class performance and reliability.

Company

SAM - the world's most trusted social media content management system for global news agencies

Product

SaaS, cloud-based, web portal, pre-release stage

Challenge

Knowing they could handle the activity and scale of the world's largest newsrooms before going live

Solutions

Simulate millions of web media consumers

Push the simulated user metrics to their New Relic® dashboard at scale

Break, fix, hypothesize, experiment and repeat until they had it right

Successes

Dramatically reduced I/O wait times

Solutioned performance bottlenecks

Ensured under 50ms response time required during peak news events, before going live

Optimized server costs and scalability
"RUN-WithIT helped us massively with their Performance Testing and Solutioning to get SAM ready for the real world. Especially in our space, this is no small feat."
James Neufeld, SAM Founder and CEO
 

The Challenge
 

    Right from minimal viable product, SAM needed to demonstrate proven engineering. They had to not only handle 10’s to 100’s of thousands of requests an hour, they needed to be highly performant, with response times under 50ms, during breaking news events. They had to know their Amazon hosted resources could give them the capacity they needed at the speed they required and at a cost they could afford.

    Like many, the SAM team was using available free tools to spin up load and then monitoring system activity using New Relic®. “Even if you think you can hit scale, basic out of the box testing rarely simulates real-world scenarios,” according to SAM’s Founder and CEO, James Neufeld. Realism was what they wanted in order to be proactive about performance.

    There were no tools to recreate the huge scale and spikes they would be facing with real browser fetch activity during news events. Exercising the system from these edges was critical to creating the same load and usage patterns of real phones and browsers.

    SAM reached out to RUN-WithIT who specializes in high volume, high fidelity simulation tools for web app systems.


    The Solutions
     

      It was a fast-paced and intense assignment for RUN-WithIT. We had a focused mission with a team who was contending with speed and delivery and who also realized the importance of scalability and reliability. For SAM, we used our framework to quickly model real browser fetcher activity in a small and lightning fast client so we could reach enormous scale. The fidelity of the fetches was critical to mimic the behaviour of client and server connections. Part of the simulation included real and headless browsers in order to verify the validity of the small and fast clients against real browsers.

      We scaled up the browser activity with ramps, spikes and sustained numbers and then pushed all of the system metrics to the team’s in-place New Relic® monitoring. The combined teams observed and solutioned. We could now script and recreate any scenario of “in the wild” activity. With this unique capability, the SAM team could utilize their simulation to break, fix, hypothesize, experiment, and repeat. The realism ensured they would know when they got it right.


      The Success
       

        SAM dramatically reduced their I/O wait times. Realistic browser connections surfaced connections that overwhelmed the server tier. Architectural improvements were staged using an A-B approach to tuning performance in the simulation. SAM minimized the content and volume of data they were caching and serving by pushing content delivery services to the edge where possible. “Our response times were less than 50 ms. Java script embeds on the internet are known to be slow and to slow down your site. We were now highly performant,” according to Sean Solbak, CTO at SAM.

        SAM chose Amazon’s AWS and Elastic Beanstalk to manage their infrastructure. It is a great way to ensure appropriate capacity is always there; however, they needed to maximize each EC2 instance in anticipation of the spikes. While solutioning the connections created by each user instance, we discovered some Linux EC2 configuration settings that limited the number of connections. Modifying these configurations expanded the capacity of each base Linux instance tenfold and reduced the gross number of connections in the server tier once more. The metrics from the simulation demonstrated the immediate gain.

        At the end of the assignment Solbak described the successful outcome by stating, “We absorbed it and ran with it. By the time we went live, we were ready to handle the load with good performance and good response times.”

        The results have stood the test of time. While they have continued to develop the front-end interface, the backend system remains relatively unchanged. SAM has succeeded in adding 3 of the 5 largest news agencies in the world to their client list (Reuters, Associated Press and AFP). More newsrooms around the world use SAM than any other curation platform. Go SAM go!





         

          RUN-WithIT helps teams bring real world performance into development.
          Find out how our tools and approach could help you get ahead.
          Book a quick ideas session.


          Contact Us