2015 Intel Big Data Software Summithttp://goto/bigdatasoftware

Enabling Near-Data Accelerators in DatacentersDave Ojika, Jayson Strayer, Gaurav Kaul, Prashanth Thinakaran, Darin Acosta

• Motivation

• Bring unconventional compute cores (especially

FPGAs) into mainstream big data use

• Abstract software complexity by introducing

efficient accelerator programming model

• Enable a data-oriented framework for near-

memory, distributed processing

• Approach

• Accelerate computation on FPGA; transfer data

over low-latency DDR bus

• Provide in-memory storage using open-source

Tachyon framework

• Offload Spark workload to accelerator

• Method

• Use Compute-Near Memory (CNM) architecture

for design-space exploration

• Map data to cores with affinity to specific memory

regions

• Integrate a Java-OpenCL middleware to support

scheduling of tasks on accelerator

Highlights Accelerator Overview

• Memory-speed data access

• Memory-centric buffer

synchronizes with

underlying file system

Write Method

Read Method

Data

Register 4 TB

Image

Cmd

Register

Interface

Connect Object

Copy Method

W

W

R

R

R

W

write_bit

read_bit

copy_bit

R/W

Workload Analysis

In-Memory Framework

Data and Compute Layer Current Developments

• Boosted Decision Tree (BDT)• Latency-sensitive

• Poor data locality

• Fits in 4TB memory

• 7-fold cross-validation

Hit 1GB 10GB

L1 94.06% 89.75%

DRAM 0.74% 5.74%quad-core i7 CPU, 8 GB RAM

• Fraction of store-bound stalls increases with size of

dataset; memory bandwidth requirement too high for CPU

Workload can be trivially parallelized across DIMMs

• 1st Place ATLAS ‘14 Higgs ML Challenge:• Deep Learning from Oxdata’s H20

• Where do FPGA accelerators stand?• Explore BDT on CNM accelerator

High-energy physics experiment at CERN’s LHC

(collaboration with UF Physics)

• Simics Simulator • Functional model

• Software stack

• Apps & workload

exploration

Task

Task

Host Middleware

Driver FPGA

Queue

Scheduler

Tachyon

File System (Local or HDFS)

• In-memory data exchange

• Reliable file sharing at

memory-speed

• Caching of working set files

in memory

• Fault-tolerant and distributed

API

Tachyon utilizes memory aggressively, leveraging data lineage

• OpenCL driver integration

• Container enablement

• Cloud orchestration

• NVM support and NFV

Compute Near Memory (CNM)

Big Data Framework

Application

API

API

Prototype with PCI, DDR and

Direct I/O interfaces

JOC

JOC: Java-to-OpenCL Component

No-Higgs or Higgs

• BDT on CPU (2nd Place

ATLAS ‘14 Higgs ML Challenge)

Application to architecture transformation

• Utilize parallelism on FPGA

• Leverage low-latency DDR

and 100 GB optical links

100 GB

Transceiver

IP

FIFO Decoder(data filter)

Data

Reassembly

Level 2: FPGA as high-performance accelerator

Level 1: FPGA receives and pre-processes data in real-time

QFSP• Direct I/O

• Real-time

• Low latency

• Low power

• Compute Engine

• Up to 3 TFLOPS

• OpenCL kernel

• BW of host memory

Altera Arria 10 FPGA

Generic implementation

To

Datastore

DRAM

DRAMDRAM

Pre-processed dataset

Datastore

Synchronize

• In-memory data store• Memory-centric distributed storage

• Reliable data sharing at memory speed

Development kit

Cloud Orchestration

Training time for 11 million events: 5 hours!

Xeon E5-2680 @ 2.8 GHZ

BDT on MATLAB

• Prediction time: 370 ms

• Okay for online, real-

time prediction

• Training time: 5 hours

• Grew with increasing

data size

• Data affinity • Cores cooperate with each

other for shared data

accesses

• Shared Virtual Memory (SVM)

Accelerator model

CPU and device both

access shared data using

the same virtual

addresses

No explicit data

marshaling

Dave Ojika: Cloud Infrastructure Jayson Strayer: Platform Silicon

Gaurav Kaul: Health and Life SciencesPrashanth Thinakaran: Big Data

Darin Acosta: Physics Professor, UF