Dr. John Bainbridge, Principal Application Architect, NetSpeed
- 1. May 9, 2016 Configurable, Coherent SoC Interconnect for Heterogeneous Multiprocessing and Storage Applications Dr. John Bainbridge ChipEx 2016 May 9th, 2016
- 2. May 9, 2016 2 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems MARKET TRENDS AND CHALLENGES
- 3. May 9, 2016 3 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems Discontinuities in the Compute Infrastructure New Demand Profile Driving New Architectures MAIN FRAMES 1980s Compute CLIENT / SERVER 1990s Desktop DATA CENTERS 2000s Internet CLOUD COMPUTING Now Software Defined Everything General Purpose > Workload specific Hardware acceleration for Software Customized SKUs
- 4. May 9, 2016 4 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems Cloud Architectures Driving Heterogeneous Computing Increased Efficiency From Hardware Specialization Source: ISSC Proceedings
- 5. May 9, 2016 5 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems Cloud Architectures Driving Heterogeneous Computing Reconfigurable Computing – ASIC + FPGA Source: Hot Chips ’14 Proceedings
- 6. May 9, 2016 6 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems Discontinuities in Chip Design Techniques Changing Abstraction Levels CUSTOM 1980s Sea of xtors ASIC 1990s Sea of Cells SOC 2000s Sea of Blocks NEXT-GEN SOCS Now Sea of IPs Time-to-Market Pressure Performance Analysis from Day #1 Create Differentiated Platforms
- 7. May 9, 2016 7 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems Evolution of Coherency Solutions Coherency Participation Has Increased CPU Memory L1 L2 CPU L1 CPU L1 Memory L3 Memory L2 CPU L1 CPU L1 L2 CPU L1 CPU L1 Last Level Cache Memory L2 CPU L1 CPU L1 L2 CPU L1 CPU L1 GPU Accelerator Number of IP blocks have exploded and More agents are participating in coherency Requirements moving from homogeneous heterogeneous
- 8. May 9, 2016 8 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems The Need for Flexible Coherency Solutions Flexibility Creates Opportunities APPLICATION Improve latency of critical traffic Improve compute cluster latency FLOORPLAN Reduce interconnect congestion Utilize unused die area with additional cache capacity POWER Partition solution to enhance opportunities for power-gating Lower BW requirement
- 9. May 9, 2016 9 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems SOLUTION
- 10. May 9, 2016 10 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems NetSpeed Technology Overview Built From Requirements, Optimized For Specific Applications PERFORMANCE, POWER, AREA WORKLOAD MODELS COMPUTE ENGINES NETSPEED IP AND SOC ARCHITECTURE SYNTHESIS CUSTOMIZED SOC PERFORMANC E OPTIMIZED CORRECT-BY- CONSTRUCTIO N CACHE COHERENT
- 11. May 9, 2016 11 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems NetSpeed Gemini Overview Scalable, Configurable Cache Coherent NoC CONFIGURABLE SOLUTION LATENCY OPTIMIZED SOLUTION SCALABLE SOLUTION
- 12. May 9, 2016 12 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems TECHNOLOGY DETAILS
- 13. May 9, 2016 13 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems Configurable Solution Floorplan Aware Design Flow • Components, Floorplan • Coherency Requirements • PPA, SoC Level Use Cases Step 1: Specify • Rapid Architecture Exploration • Real-time Customization • Power Optimization Step 2: Customize Synthesizable RTL Verification IP Physical Design Collateral IPXACT Performance stats, Programmers Guide, etc. • Customized SoC Interconnect IP Step 3: Generate Configurable Latency Optimized Scalable
- 14. May 9, 2016 14 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems Configurable Solution Enabling Cache Hierarchy Customization Add last level caches to reduce critical latencies Use multiple coherency controllers or caches to increase bandwidth Improve cache utilization by controlling address ranges serviced by each cache Support coherency across a mix of different slaves devices Configurable Latency Optimized Scalable
- 15. May 9, 2016 15 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems Correct-by-Construction Formal Analysis to Build Deadlock-free NoC Formally Proven Formal techniques and graph theory algorithms Correct-by-construction User-driven traffic dependencies Robust Handles complex topologies and routing Configurable Latency Optimized Scalable
- 16. May 9, 2016 16 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems Latency Optimized Solution Traffic Based Rapid Reconfiguration Fully heterogeneous in determining channel & buffer sizes Ring TreeHeterogeneous Mesh NetSpeed Router NetSpeed Auto-generated Links Configurable Latency Optimized Scalable
- 17. May 9, 2016 17 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems Latency Optimized Solution Physically Distributed Coherency GPULast Level Cache CPU CPU CPU CPU Last Level Cache CCC CCC CCC DSP Lower latency by placing coherency controllers and caches where they are accessed the most Reduce congestion by handling requests locally and using caches to reduce traffic to memory Adjust cache hierarchy to support floorplan requirements Improve die utilization by placing caches in empty die space Configurable Latency Optimized Scalable
- 18. May 9, 2016 18 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems Latency Optimized Solution Physically Distributed Coherency FastPath – Dedicated connections for latency sensitive traffic to reduce arbitration, congestion MultiLayer – Isolate traffic with different performance requirements QoS configurability for bandwidth allocation and prioritization ACE Master #0 ACE Master #1 ACE ACE FastPathTM Bridge FastPathTM Bridge ACE CCC NoC Master Bridge ACE Master Bridge FastPathTM Reducing CPU latency Configurable Latency Optimized Scalable
- 19. May 9, 2016 19 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems Scalable Solution Scalable Coherency Bandwidth Configurable Latency Optimized Scalable More coherent lookups per cycle – Increased coherency bandwidth through address-sliced coherency controllers – Automated determination of coherency controllers
- 20. May 9, 2016 20 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems Scalable Solution Importance of Filtering Snoops Configurable Latency Optimized Scalable Scalability requires significant filtering of snoops Directory structure delivers efficient solution in lesser area 1.0 2.0 1.5 0.5 0.0 3.0 2.5 90 % 85 % Filter Rate (%) SizeofDirectory/SnoopFilter(MB) 100 % 95 % 50 % 80 % 70 % 75 % 65 %
- 21. May 9, 2016 21 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems Scalable Solution Directory Scaling Configurable Latency Optimized Scalable NETSPEED SOLUTION: – Superior directory design – Linear growth vs. O(n2) growth in directory Directory size grows O(n2) – Number of Entries & Number of Agents TAG Number of Entries DIRECTORY Ownership Vector Number of Agents
- 22. May 9, 2016 22 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems Scalable Solution Reducing Conflicts Configurable Latency Optimized Scalable Power Performance ONE-TO-ONE DIRECTORY – Directory = sum(cache associativity) – Dynamic power with traffic & number of agents – O(n2) GLOBAL SHARED DIRECTORY – Address conflicts reduce performance – Limited associativity reduces power – Conflict rate increases with cache associativity NETSPEED SOLUTION: – Limits associativity and dynamic power – Advanced techniques to avoid address conflicts
- 23. May 9, 2016 23 John Bainbridge, ChipEx 2016 | © Copyright 2016 NetSpeed Systems Summary PROBLEM New architectures driving need for heterogeneous computing Exploding complexity with shrinking timelines Need configurable coherency solutions SOLUTION NetSpeed Architecture Synthesis Platform NetSpeed Gemini: Configurable and Scalable Coherent NoC IP BENEFITS Flexible, correct-by-construction IP Higher performance with lower latency Build customized application-specific solution