RISC-­‐V

 “Rocket  Chip”  
Tutorial  
Colin  Schmidt  
UC  Berkeley  
[email protected]!
 Outline  
§  What  can  Rocket  Chip  do?  
§  How  do  I  change  what  Rocket  Chip  generates?  
- What  are  chisel  parameters  and  how  do  they  help  me?  
§  How  do  I  use  the    C++  emulator?  
§  How  do  I  get  a  waveform/debug?  
§  How  do  I  add  different  opDons?  
- Where  do  I  put  my  changes?  
§  How  do  I  add  new  instrucDons?  
- How  do  I  “drop-­‐in”  my  accelerator?  
- Where  do  I  put  different  extensions?  
§  How  do  I  use  verilog  generaDon?  
- For  an  ASIC  toolflow  
- For  an  FPGA  target  

2
 What  can  Rocket  Chip  do?  

§  What  can  Rocket  Chip  do?  

§  Rocket  chip  allows  you  to  generate  different  
configuraDons  of  an  SoC,  including  the  soPware  
toolchain  that  would  run  on  this  soPware  
§  These  configuraDons  are  specified  through  chisel  
parameters  most  of  which  can  be  freely  changed  
§  We  can  then  select  what  to  generate  
§  C++  RTL  emulator  
§  Verilog    
- FPGA  
- ASIC  

3
 What  are  all  these  submodules  in  Rocket  Chip?  
§  Chisel  
-  The  HDL  we  use  at  Berkeley  to  develop  our  RTL.  
§  Rocket  
-  Source  code  for  the  Rocket  core  and  caches    
§  Uncore  
-  Logic  outside  the  core:  coherence  agent,  Dle  interface,  host  
interface  
§  Hardfloat  
-  Parameterized  FMAs  and  converters,  see  README  
§  Dramsim2  
-  Simulates  DRAM  Dming  for  simulaDons  
§  Fpga-­‐zync  
-  Code  that  helps  get  rocket-­‐chip  on  FPGAs    
§  Riscv-­‐tools  
-  SoPware  toolchain  used  with  this  version  of  Rocket  Chip  

4
 What  about  the  other  folders?  
§  CSrc  
-  Glue  code  to  be  used  with  the  C++  emulator  
§  Emulator  
-  Build  directory  for  the  C++  emulator,  contains  generated  code  
and  executables  
§  Fsim  
-  Build  directory  for  FPGA  verilog  generaDon  
§  Project  
-  Scala/sbt  configuraDon  files  
§  Src  
-  Chisel  source  code  for  rocket  chip  
§  Vsrc  
-  Verilog  test  harness  for  rocket-­‐chip  
§  Vsim  
-  Build  directory  for  ASIC  verilog  generaDon  
§  Rocc-­‐template  (example  rocc  used  for  this  tutorial)  

5
 Overview  of  Rocket  Chip  Parameters  

§  Located  in    

src/main/scala/PublicConfigs.scala
§  Easily  changed  parameters  are  called  Knobs  
§  Important  configuraDon  opDons  fit  in  a  few  
categories  
- Tile  –  How  many,  what  types,  what  accel?  
- Memory  –  Phys/Virt  Address  bits,  Mem  interface  params  
- Caches  –  Sets,  ways,  width  etc.  for  L1  and  L2;  TLBs  
- Core  –  FPU?,  fma  latency,  etc.  
- Uncore  –  coherence  protocol,  Dlelink  params  

6
 Configs  

§  Parameters  can  be  changed  to  create  different  

configuraDons  
§  Knobs  require  defaults  and  are  parameters  we  expect  
to  be  tunable  via  Design  space  exploraDon  
§  Two  examples  given  at  bocom  of  PublicConfigs.scala  
- DefaultConfig  –  used  when  no  other  configuraDons  are  
specified  
- SmallConfig  –  removes  FPU  and  has  smaller  caches  
§  To  generate  a  different  configuraDon  you  can  simply  
follow  the  SmallConfig  Example,  sedng  parameters  
and  knobs  as  you  want  

7
 SimulaEng  a  ConfiguraEon  

§  C++  RTL  emulator  built  from  emulator  directory  

§  The  default  emulator  has  already  been  built  
make run-asm-tests
§  We  can  also  build  the  small  config  very  easily  
make CONFIG=ExampleSmallConfig
§  And  test  it  too!  
make CONFIG=ExampleSmallConfig run-
asm-tests

8
 Making  and  SimulaEng  a  new  ConfiguraEon  
§  Lets  try  making  a  “medium”  sized  config  
-  Double  the  number  of  ways  in  L1  I  and  D  cache  in  small  config  
class MediumConfig extends SmallConfig{
override val knobValues:Any=>Any = {
case "L1D_WAYS" => 2
case "L1I_WAYS" => 2
}
}
class ExampleMediumConfig extends ChiselConfig(new
MediumConfig ++ new DefaultConfig)
§  All  we  need  to  do  is  specify  it  when  making  the  emulator  
make CONFIG=ExampleMediumConfig
§  We  can  then  test  the  new  config  
make CONFIG=ExampleMediumConfig run-asm-
tests
§  The  power  of  generators!  

9
 More  Complicated  ConfiguraEons  

§  How  would  I  add  a  new  parameter  to  rocket  chip?  

- Widely  used  parameters  for  the  generator  can  be  added  to  
the  DefaultConfig    
- It  is  then  made  available  via  Chisel  parameters  to  the  
implementaDon  
§  How  do  I  add  accelerators?  What  about  their  
parameters?  
- Other  modules  like  accelerators  should  have  their  
parameters  declared  in  their  own  source  folder    
- Default  configuraDon  can  be  added  to  a  new  *Configs.scala  
in  the  rocket-­‐chip  source  
- More  on  this  later  

10
Rocket Custom Coprocessor
ISA  Extensions  and  RoCC  
Extensions
§  Chapter  9  in  the  ISA  manual  
§  4  major  opcodes  set  aside  for  non-­‐standard  
extensions  (Table  8.1)  
Rocket is a particular microarchitectural implementation of RISC-V, which supports addition of
custom accelerators over a standardized coprocessor interface. This chapter describes the instruc-
- Custom  
tion encoding template0used
-­‐3     by Rocket Custom Coprocessors (RoCCs). Each accelerator will
- Custom  2  and  3  are  reserved  for  future  RV128  
define its own instruction subset name, but these should all begin with “X” to signify they are
non-standard ISA extensions.
§  RoCC  interface  uses  this  opcode  space  
- 2  source  operands,  1  desDnaDon,  7bit  funct  field  
This extension format is only one possible way of adding custom instructions to the base in-
struction set, and was designed to simplify the microarchitectural dependencies between a core
- 3  bits(xd,xs1,x2)  
implementing the base ISA and d a etermine   if  this  instrucDon  
coprocessor implementing uses  
custom extensions. the  ex-
Custom
register  
which inoturn
perands,  
build on the and   passes  
floating-point the  value  in  register  rs1/2,  or  
tensions can be more tightly coupled to the core, for example, by building on the packed-SIMD
extensions, extensions.
Of writes   the  is rthe
course, Chisel esponse  
best way to tsculpt
o  rd  a RoCC.
Depending  
The§ Rocket core implements on   all t he  ofatheccelerator  
four major opcodes reserved these   for could  
custom always  
extensions be  
(custom-0,
repurposed  since  the  instrucDon  is  also  sent  over  the  
custom-1, custom-2, custom-3 in Table ??), and treats these as R-type instructions:

RoCC  interface  
31 25 24 20 19 15 14 13 12 11 7 6 0
funct7 rs2 rs1 xd xs1 xs2 rd opcode
7 5 5 1 1 1 5 7
roccinst[6:0] src2 src1 dest custom-0/1/2/3 11
 RoCC  Accelerators  

§  ImplemenDng  the  RoCC  interface  is  probably  the  

simplest  way  to  create  a  RISC-­‐V  extension  
§  Toolchain  already  supports  custom0-­‐3  assembly  
- No  need  to  modify  the  toolchain  at  all  if  you  fit  into  this  
interface  
§  Need  to  implement  the  RoCCIO  interface  
§  Located  in    
rocket/src/main/scala/rocc.scala
 

12
 §  Rocket  sends  coprocessor  
Rocket Decoupled(Cmd) ROCC instrucDon  via  the  Cmd  
Accel. interface  (including  
Decoupled(Resp)
registers)  
CacheIO §  Accelerator  responds  
through  Resp  interface  
busy §  Accelerator  sends  memory  
IRQ requests  to  L1D$  via  
CacheIO  
supervisor bit §  busy  bit  for  fences  
§  IRQ,  S,  excepDon  bit  used  
UncachedTileLinkIO for  virtualizaDon  
§  UncachedTileLinkIO  for  
PTWIO instrucDon  cache  on  
exception
accelerator  
§  PTWIO  for  page-­‐table  
walker  ports  on  accelerator  
13
 Rocc  Accelerator  Example  

§  We  can  now  start  walking  through  an  example  

accelerator  used  in  teaching  CS250  at  Berkeley  
§  (checkout  sha3  branch  of  rocc-­‐template)  
§  This  branch  of  rocc-­‐template  implements  the  SHA3  
cryptographic  hashing  algorithm  
§  It  includes  several  things  
-  C  reference  code  in  rocc-template/src/main/c
-  Chisel  implementaDon  in  rocc-template/src/main/
scala
-  C  test  cases  for  both  SW  and  RoCC  in  rocc-template/
tests
-  FuncDonal  model  for  Spike  in  rocc-template/isa-sim
-  New  Rocket  chip  configuraDon  in  rocc-template/config
-  Script  to  install  symlinks  to  all  these  files  

14
 FuncEonal  Model  of  Accelerator  

§  First  step  to  any  architecture  project  write  a  simulator  

§  Spike  is  designed  to  be  extendable  
vim rocc-template/isa-sim/sha3/sha3.h
§  We  extend  the  rocc_t  class  implemenDng  a  subset  of  
the  custom  opcodes  
§  Describes  a  funcDonal  model  of  the  computaDon  
§  Adheres  to  the  same  interface  as  the  accelerator  
§  InteracDng  with  the  simulated  memory  happens  
through  the  processors  mmu  p->get_mmu()  
§  Now  we  are  ready  to  test  the  model  

15
 FuncEonal  Model  of  Accelerator  

§  Rather  than  moving  the  files  out  of  the  rocc-­‐template  
directory  we  just  symlink  to  them  
./install-symlinks
§  Now  we  can  rebuild  spike  to  be  able  to  model  our  
accelerator  
cd ../riscv-tools && ./build-spike-
only.sh
§  Now  spike  understands  our  extension  
spike --extension=sha3

16
 Accelerator  Tests  
§  A  few  variants  of  a  simple  sha3  test  
sha3-sw[-bm].c
sha3-rocc[-bm].c
§  sw  versions  just  uses  the  reference  C  implementaDon  
§  rocc  versions  use  inline  assembly  to  call  the  
accelerator  
cd ../rocc-template/tests
vim sha3-rocc.c
§  The  operands  are  xd/rd,  xs1/rs1,  xs2/rs2,  and  funct  
§  Pudng  0  for  the  register  operands  marks  them  
unused  
§  Otherwise  you  can  use  standard  assembly  syntax  to  
send  values  to  the  accelerator  
17
 FuncEonal  Model  of  Accelerator  TesEng  

§  Now  we  are  ready  to  test  our  model  

§  First  just  the  soPware  only  version  
spike pk sha3-sw.rv
§  Lets  try  the  accelerator  version  without  the  accel  
spike pk sha3-rocc.rv
§  An  expected  failure  so  now  we  enable  our  model  
spike --extension=sha3 sha3-rocc.rv
§  Success!  

18
 Chisel  Accelerator  

§  Time  to  implement  our  design  in  chisel  and  plug  it  in  
to  Rocket  chip  
§  Luckily  the  implementaDon  is  done  and  rocket  chip  is  
smart  enough  to  pick  up  on  folders  that  look  like  a  
chisel  project  (i.e.  have  a  src/main/scala  directory)  
§  We  can  look  at  how  the  accelerator  is  parameterized  
vim src/main/scala/sha3.scala
§  Looking  at  the  bocom  we  see  it  looks  similar  to  
previous  configs  we  have  looked  at  with  the  addiDon  
of  a  set  of  constraints  
§  The  constraints  help  during  any  design  space  
exploraDon  you  want  to  undertake  
19
 Chisel  Accelerator  Plug-­‐in  

§  Now  lets  setup  rocket  chip  to  include  our  accelerator  
vim config/PrivateConfigs.scala
§  The  important  parameter  is  the  BuildRoCC  parameter  
which  gives  the  constructor  for  the  Sha3  accelerator  
§  Rocket  chip  uses  this  parameter  to  instanDate  the  
accelerator  in  its  datapath  
§  The  clean  interface  allows  this  to  happen  seamlessly  
§  Now  we  can  build  the  accelerated  version    
make CONFIG=Sha3CPPConfig

20
 Chisel  Accelerator  Performance  

§  Time  to  test  this  new  emulator  

§  We  can  even  measure  performance  (pk  “s”  flag)  
./emulator-DefaultCPPConfig pk -s
sha3-sw-bm.rv
./emulator-Sha3CPPConfig pk -s sha3-
sw-bm.rv
./emulator-Sha3CPPConfig pk -s sha3-
rocc-bm.rv
§  Even  on  a  very  short  test  with  a  single  hash  we  see  a  
good  speed  up    

21
 Chisel  RTL  Debugging  
§  What  if  I  had  a  bug?    
§  Chisel  has  support  for  “prins”  in  your  code  but  you  
might  want  to  just  see  a  waveform  
§  C++  emulator  supports  this  too  
make debug
./emulator-DefaultCPPConfig-debug -
vtest.vcd +loadmem=output/
median.riscv.hex
§  This  creates  a  standard  vcd  that  a  program  like  
gtkwave  can  open  
gtkwave test.vcd
§  This  same  setup  works  for  the  accelerator  just  takes  
longer  because  of  the  pk  and  test  length  
22
 Non-­‐RoCC  extensions  

§  What  if  I  want  to  extend  the  ISA  in  a  different  way,  
not  RoCC  
§  This  will  be  more  work  but  could  give  you  more  
freedom  and  a  Dghter  integraDon  
§  Updates  need  to  be  made  in  several  locaDons  
- riscv-­‐opcodes  (define  your  new  encodings)  
- riscv-­‐gnu-­‐toolchain  (add  new  instrucDons  to  assembler)  
- riscv-­‐isa-­‐sim  (update/add  instrucDon  definiDon)  
- rocket  (datapath  and  front-­‐end  updates)  

23
 Non-­‐RoCC  extension  riscv-­‐opcodes  

§  Repository  for  all  encodings  

§  Generates  
- Header  files  gnu-­‐toolchain  
- Header  files  for  isa-­‐sim  
- ISA  manual  tables  
- Chisel  code  to  include  in  rocket  
§  Add  the  instrucDon  to  one  of  the  opcodes  files    
make install
- Generates  all  the  different  files  and  installs  them  in  the  
correct  folders  

24
 Non-­‐RoCC  extension  riscv-­‐gnu-­‐toolchain  

§  Contains  binuDls,  gcc,  newlib  and  gcc  ports  

§  Add  instrucDon  definiDon  to    
binutils/opcodes/riscv-opc.c
§  This  is  all  that’s  needed  for  simple  instrucDons  
§  Rebuild  the  toolchain  and  you  can  assemble  your  new  
instrucDon  

25
 Non-­‐RoCC  extension  riscv-­‐isa-­‐sim  

§  Already  looked  at  this  earlier  for  RoCC  extensions  

§  Standard  riscv  instrucDons  are  defined  in    
riscv/insns
§  Adding  the  instrucDon  to  riscv-­‐opcodes  will  cause  
spike  to  look  for  a  header  file  in  this  folder  with  the  
instrucDons  name  
§  The  header  file  describes  how  the  instrucDon  behaves  
§  Many  examples  of  different  instrucDons  to  start  with  

26
 Non-­‐RoCC  extension  rocket  

§  ModificaDons  to  this  code  will  greatly  depend  on  the  
instrucDon  
§  Simply  adding  a  new  ALU  op  would  require  very  few  
changes  
§  The  complexity  of  the  changes  will  depend  greatly  on  
the  instrucDon  
§  Happy  to  work  through  this  with  you  in  lab  Dme  but  
we’ll  skip  over  it  for  now  

27
 Rocket  Chip  Verilog  

§  The  vsim  directory  contains  build  scripts  to  generate  

verilog  with  an  ASIC  backend  in  mind  
cd ../vsim && make
§  The  generated-­‐src  directory  contains  
- Verilog  source  (Top.$CONFIG.v)  
- Set  of  exported  parameters  (Top.$CONFIG.prm)  
- Memory  parameters  (Top.$CONFIG.conf)  
§  Memory  parameters  are  used  in  our  flow  to  figure  
out  which  SRAMs  to  generate  or  request  
§  vlsi_mem_gen  script  is  used  by  Berkeley  to  
automate  this  process  
§  APer  this  processing  the  verilog  is  ready  for  CAD  tools  

28
 Rocket  Chip  Verilog  for  FPGA  

§  The  fsim  directory  contains  build  scripts  to  generate  

verilog  with  an  FPGA  backend  in  mind  
cd ../fsim && make
§  The  generated-­‐src  directory  contains  
- Verilog  source  (Top.$CONFIG.v)  
- Set  of  exported  parameters  (Top.$CONFIG.prm)  
- Memory  parameters  (Top.$CONFIG.conf)  
§  fpga_mem_gen  handles  the  memory  configuraDons  
§  fpga-­‐zynq  repo  has  build  scripts  aPer  this  point  but  
requires  the  fpga  tools  to  run  
§  Well  documented  repo  so  refer  to  its  README  for  
more  instrucDons  
29
Lab  Time!  

30