CERN Hybrid Cloud Empowers Breakthrough Research

Scientists tap the world’s largest particle accelerator and Intel® technology to understand the universe’s composition.

Executive Summary
The European laboratory for particle physics, known as CERN, studies the subatomic building blocks comprising everything in the universe. Their Large Hadron Collider (LHC), the largest particle accelerator on earth, forces two beams of subatomic particles in opposite directions around a 27-kilometer ring at rates near the speed of light. When the beams collide, particles shatter into their most elementary components. Detectors capture massive volumes of real-time data resulting from the impacts. That data is then stored and mined for insights with the aid of CERN’s hybrid cloud solution.

Challenge
Ironically, understanding the smallest particles in our universe requires enormous levels of compute power to derive meaning from extensive data volumes. Every second, highly sensitive detectors capture petabytes of data resulting from each experimental run using the LHC. Although a filtering process reduces that volume, tens of gigabytes remain to be captured and stored real-time, on-premise. To supplement their on-site infrastructure, CERN requires a fast, scalable, and budget-friendly compute and storage solution for simulation and modeling of the resulting data.

Solution
Because bandwidth between the Cloud Service Providers (CSPs) and CERN facility has practical limits, the LHC data capture process remains localized to CERN’s own datacenters architected for the task. After the data collection phase, the hosted cloud solutions manage CERN’s data evaluation process. Two CSPs each architected custom solutions for CERN’s storage, security, performance, and application hosting specifications. Both CSPs offered expandable virtual machine (VM) capability and extensive storage to facilitate modeling and simulation of LHC data.

Large Hadron Collider tunnel at CERN (photo courtesy CERN)

Results
Both CSPs prototyped and tested a hybrid cloud solution tailored to CERN’s specialized requirements. Each hosted solution provides the needed storage, processing, and networking speeds CERN needs today, with a streamlined growth path into the future. Plus, because the hosted cloud solution can be repurposed by other research organizations in Europe when not in use by CERN, the financial investment benefits a diverse group of scientists. Hybrid cloud thereby creates broader value and greater price-performance.

“Our team continually seeks ways to optimize and accelerate our infrastructure, so hybrid cloud solutions are extremely important to our work. It is our job to make sure that IT does not limit physics.” – Tim Bell, compute and monitoring group leader, IT department, CERN

Hybrid Cloud Facilitates Large-Scale Simulations and Custom Requirements
CERN’s hybrid cloud proves extremely adept in delivering the compute infrastructure needed for specialized workloads. Also, a hosted hybrid cloud solution meets several other CERN requirements including:

  • Cost-efficiency through a pay-for-use model
  • Compliance with European data protection rules
  • The ability to port relevant applications to the cloud
  • Extensive virtual machine capability
  • Federated identity and single sign-on
  • Scalability with expandable resources to handle growing data sets

A jewel of particle physics, the CMS experiment is a 14 000-tonne detector that aims to solve a wide range of questions about the mysteries around the Higgs boson and dark matter. Now that the Large Hadron Collider (LHC) beam has been switched off for a two-year technical stop, Long Shutdown 2 (LS2), CMS is preparing for significant maintenance work and upgrades. (Photo courtesy CERN)

Solution Details
A hybrid cloud solution gives CERN scientists the ability to place key workloads in the ideal location for each task.

CERN’s onsite server nodes, augmented by their data center facilities in Switzerland and Budapest, are connected by 10 Gb Ethernet. Together, these data center facilities benefit from the compute speed of 15,000 servers and 230,000 Intel® Xeon® processing cores from different generations, including the latest Intel® Xeon® Scalable processors. A combination of hard disk drives (HDD) and solid-state drives (SSD) offer CERN the ideal balance between storage cost-per-gigabyte and the speed needed for real-time data capture.

An optimized OpenStack* software solution underlies CERN’s onsite hardware. OpenStack was chosen for the flexibility and scale it brings to CERN’s work. The CERN team also works closely with other scientists using OpenStack elsewhere in the world. In doing so, teams can share best practices for software optimization, which have the potential to benefit other research institutions.

Each of the CSPs hosting data analysis and simulations for CERN offer the infrastructure flexibility to accommodate additional load as required.

Hybrid Cloud Extends On-Premise Infrastructure for Efficient Workload Placement
Years ago, after the CERN team completed LHC data capture phases, they needed to re-provision and optimize their infrastructure resources for the data analysis process. CERN’s new hosted solutions transform this process by placing simulation workloads into the cloud. The transition eliminates the re-provisioning process, offers greater scale, and alleviates on-site compute demands. Rather than making additional on-premise hardware investments, large numbers of hosted virtual machines handling CERN’s simulation workloads enables a simpler, secure, elastic, and cost-effective mechanism to accomplish the task.

Hybrid Cloud Offers a Highly Scalable, Cost-Effective Solution for Multiple Workloads
CERN has a finite budget to accomplish increasingly complex work. Because their future LHC runs will create even larger volumes of data, the team needed a practical, budget-conscious solution. Hosted solutions not only offer them high available compute resources and storage, but the elasticity to grow in parallel with CERN’s larger data sets anticipated from future LHC runs.

Spotlight on CERN
Based in Geneva, the European Organization for Nuclear Research, commonly known as CERN, conducts groundbreaking scientific work to study the composition of matter in our universe. Doing so requires advanced instruments like the world’s largest particle accelerator, the Large Hadron Collider (LHC). The LHC’s 27-kilometer loop is lined with electromagnets to focus particle beams and accelerate them near the speed of light. When atomic particles collide inside the accelerator, they shatter into their smallest constituent parts. CERN researchers use extremely sensitive detectors to observe and gather data for further technical evaluation.

Event recorded with the Compact Muon Solenoid (CMS) detector in 2012 at a proton-proton center of mass energy of 8 TeV. The event shows characteristics expected from the decay of the SM Higgs boson to a pair of photons (dashed yellow lines and green towers). (Image courtesy CERN)

Lessons Learned
Through the RFP, prototyping, and testing of hosted hybrid cloud solutions the CERN team noted several observations that can help other scientific institutions facing similar needs:

  • Rather than the CERN team building a hybrid cloud solu­tion on their own, CERN presented a list of requirements to CSPs and gave the CSPs freedom to architect the solu­tion that best met those requirements.
  • Bandwidth between CERN facilities and the CSPs remains a gating factor, making on-premise infrastructure an ongoing necessity for real-time capture of LHC data.
  • Scientists need a common interface for data access on-premise, or in the cloud.
  • Because petabytes of LHC data must remain accessible for decades, a long-term, cost-effective archival solution must augment short-term storage needs.
  • Augmenting a purely on-premise solution with a hosted hybrid cloud solution is not just a technological change for IT staff, it requires a cultural shift too.

Solution Ingredients

  • Intel Xeon Scalable processors

Learn More
Hybrid Cloud Combines On-and Off-Premises Services ›
Making Private-Public Cloud Decisions on the Way to a Hybrid Cloud ›
The Road to Hybrid Cloud Success ›

Explore Related Products and Solutions

Intel® Xeon® Scalable Processors

Drive actionable insight, count on hardware-based security, and deploy dynamic service delivery with Intel® Xeon® Scalable processors.

Learn more

Intel® Select Solutions

Deliver a simplified data center infrastructure with workload-optimized configurations for fast and easy deployment.

Learn more

Zastrzeżenia i uwagi prawne

Cechy i zalety technologii Intel® zależą od konfiguracji systemu i mogą wymagać obsługującego je sprzętu, oprogramowania lub aktywacji usług. Wydajność może różnić się od podanej w zależności od konfiguracji systemu. Całkowite zabezpieczenie systemu komputerowego jest niemożliwe. Więcej informacji można uzyskać od sprzedawcy lub producenta systemu albo na stronie https://www.intel.pl. // Oprogramowanie i obciążenia wykorzystane w testach wydajności mogły zostać zoptymalizowane do wydajnego działania tylko na mikroprocesorach Intel®. Testy wydajności, takie jak SYSmark i MobileMark, mierzą wydajność określonych systemów komputerowych, komponentów, oprogramowania, operacji i funkcji. Jakakolwiek zmiana wyżej wymienionych czynników może spowodować uzyskanie innych wyników. Aby wszechstronnie ocenić planowany zakup, w tym wydajność danego produktu w porównaniu z konkurencyjnymi, należy zapoznać się z informacjami z innych źródeł oraz innymi testami wydajności. Dokładniejsze informacje można znaleźć na stronie http://www.intel.pl/benchmarks. // Wyniki wydajności są oparte na testach z dnia wskazanego w konfiguracjach i mogą nie uwzględniać wszystkich publicznie dostępnych aktualizacji zabezpieczeń. Więcej informacji zawiera zastrzeżenie dotyczące konfiguracji. Żaden produkt ani komponent nie jest całkowicie bezpieczny. // Opisane scenariusze obniżenia kosztów mają stanowić przykłady na to, jak dany produkt oparty na technologiach Intel® może w określonych warunkach i konfiguracjach wpłynąć na generowanie kosztów oraz zapewnić oszczędności. Warunki mogą ulec zmianie. Firma Intel nie gwarantuje żadnych poziomów kosztów ani ich obniżenia. // Firma Intel nie sprawdza i nie weryfikuje danych referencyjnych podawanych przez strony trzecie lub dostępnych na stronach internetowych wymienianych w niniejszym dokumencie. Aby potwierdzić dokładność tych danych, należy odwiedzić strony internetowe, do których się odnoszą. // W niektórych scenariuszach testowych wyniki otrzymano na podstawie szacunków lub symulacji przeprowadzonych w oparciu o wewnętrzne procesy analizy, symulacji architektury lub modelowania, przeprowadzone przez firmę Intel i dostarczone wyłącznie w celach informacyjnych. Jakiekolwiek różnice w sprzęcie, oprogramowaniu lub konfiguracji mogą wpłynąć na rzeczywistą wydajność.