Joe Rodriguez, Rambus

Justin Endo, Mixel

MIPI® Sensor Solutions for
Autonomous Driving

© 2022 MIPI Alliance, Inc.

Agenda
• Introduction
• Sensors
• Safety
• Summary

© 2022 MIPI Alliance, Inc. 2

MIPI in Automotive
• Auto industry is being transformed by global
trends
– Growing embrace of electric vehicles
– Increasing vehicle automation
– Tighter safety
– Fuel economy standards
• Automotive industry is now leveraging the
mobile smartphone technology
– High-resolution dashboard displays connected to
back-up rear cameras
– Infotainment displays with GPS navigation
– Multi-wireless Bluetooth, Wi-Fi and 4G/5G
cellular connections
– Advanced Driver Assistance Systems (ADAS)
– Voice recognition + voice commands

© 2022 MIPI Alliance, Inc. 3

 Automotive Sensor Market
• The automotive sensor market is rapidly growing due to the adoption of
Advanced Driver Assist Systems (ADAS)
– Camera, Light Detection and Ranging, Radar and Ultrasound
– Automobiles can be expected to have 8-12 sensors

Radiation Type
Wavelength (m)

About the Size of Atomic Nuclei Atoms Molecules Protozoans Pinpoint Honeybee Humans Buildings

Short wavelength Long wavelength

High energy Low energy
High frequency Low frequency
Source: MIPI Alliance White Paper Driving the Wires of Automotive

© 2022 MIPI Alliance, Inc. 4

Architectures Must Balance Many Requirements To Succeed
• High Throughput
• Low Power
• Low Latency
• Continuous Streaming
– Predictable (no buffering)
• Programable
MIPI CSI-2® Enables trade-offs
based on application requirements
© 2022 MIPI Alliance, Inc. 5
Sensors

© 2022 MIPI Alliance, Inc.

MIPI CSI-2 Specification, the Sensor Workhorse
• Each feature builds on a
solid foundation
• Continual improvement
• Continual flexibility

© 2022 MIPI Alliance, Inc. 7

MIPI D-PHY and MIPI C-PHY
• D-PHYSM and C-PHYSM Feature MIPI D-PHY Specification

are the foundation of

V1.0 V1.1 V1.2 V2.0 V2.1 V2.5 V3.0

Maximum Speed 1.0Gbps 1.5Gbps 2.5Gbps 4.5Gbps 9.0Gbps

low power architecture (Standard Channel)

Deskew Calibration
(Standard Channel)

• Multiple generations HS-TX Halfswing

HS-RX unterminated

with increasing speeds ALP Mode

and additional features Fast Bus Turnaround

Feature MIPI C-PHY Specification

– Some features add V1.0 V1.1 V1.2 V2.0 V2.1 V3.0 (TBD)
power savings Maximum Speed 1.7Gsps 2.8Gsps 3.5Gsps 6.0Gsps 6.0Gsps 5.0Gsps (4 Bits/Sym)
(Standard Channel)

– Other features for IoT LVHS TX

HS-RX unterminated
applications requiring ALP Mode
long channel, reduce #
of wires Fast Bus Turnaround

© 2022 MIPI Alliance, Inc. 8

MIPI D-PHY and MIPI C-PHY in Automotive Applications
• MIPI D-PHY has been traditionally
used when MIPI CSI-2 is needed for
automotive applications, having
been around for many more years D-PHY vs. C-PHY for Automotive Applications
100%
• MIPI C-PHY initial gained traction in 75%

consumer applications 50%

25%
• Increasing use of MIPI C-PHY for 0%
2019 2020 2021 2022 (YTD)
automotive applications (primarily
C/D-PHY or C-PHY D-PHY
for MIPI CSI-2)
Source: Mixel, Inc.
• Seeing increasing use of MIPI for
“mission critical applications”
© 2022 MIPI Alliance, Inc. 9
MIPI PHY Solution for Automotive Application
• In automotive applications, testability is
essential. MIPI IP may require:
– In system testability
• Universal configuration supports loopback and full-
speed production testing.
• Mixel created RX+ and TX+ configurations based on
automotive customers’ request for in-system,
production testing with minimal area penalty
– For mission critical applications, design for higher
sigma and higher junction temperature
– Design to automotive PDK from foundry
• Additional testing and verification required
for automotive IP:
– Stress test (e.g., voltage stress) with ability to screen
out latent defects during testing
– HTOL to ensure lifetime requirement CSI-2 TX+ CD-PHY
– Reliability simulation
• Ageing
• EMIR

© 2022 MIPI Alliance, Inc. 10

Automotive Sensor Subsystem Architecture
• Design a sensor aggregator
chip on ECU side C/D-PHY Bridge

– ECU support 1 or 2 sensors MIPI CSI-2

Front Camera

• System configurable
– Sensors MIPI
CSI-2 RX C/D-PHY Bridge
MIPI
• One high lane rate ECU CSI-2 TX

• Two mid-range lane rate

• How can MIPI C/D-PHY C/D-PHY Bridge
support this with MIPI CSI-2
controller?
MIPI CSI-2
Rear Camera

© 2022 MIPI Alliance, Inc. 11

 Software Configurable Sensor Aggregator Solution
Subsystem

Controller
Controller

C/D-PHY
C/D-PHY
Sensor

CSI-2 Rx
CSI-2 Tx

Subsystem

Controller
Controller

C/D-PHY
C/D-PHY
Sensor

CSI-2 Rx
CSI-2 Tx
sensor
2L/2T sensor aggregator
aggregator 4L/3T
DSC Encoder
Subsystem

Controller
DSC Encoder
C/D-PHY
Controller

C/D-PHY
Sensor

CSI-2 Rx
CSI-2 Tx

SerDes PHY/
2L/2T SerDes PHY/
Controller

Controller

SoC Fabric Interconnect (e.g., NOC/AXI bus)

SoC Fabric Interconnect (e.g., NOC/AXI bus)

© 2022 MIPI Alliance, Inc. 12

Bandwidth Matching: User FIFO Overview
• Main purpose is to handle cases where the
User Application clock does not match the
CSI-2 Tx Controller Clock. System Clock Byte Clock
from PHY

• The User FIFO also handles cases of pixel

bandwidth differences. (16 pixels per clock
into FIFO, 8 pixels per clock out). clk Command ui_clk clk Command clk
Req/Type/Cnt Req/Type/Cnt
CSI-2 TX User FIFO

• The User FIFO must be large enough to User

Command Ack Command Ack
CSI-2 TX
prevent any gaps or stalls in MIPI data during Application Controller

packet transfer Pixel Read Enable Pixel Read Enable

Pixel Data Pixel Data

– User clock is faster or slower than MIPI clock

• The User Application interface sends packet Command request include
requests indicating type and pixel count.
Once the User FIFO acknowledges the packet Data Type
request, pixel read enable requests pixel data Virtual Channel
until pixel count is reached. Number of Pixels

© 2022 MIPI Alliance, Inc. 13

Bandwidth Matching Example
• If MIPI bandwidth is less than User FIFO input bandwidth, then clock can be slower than MIPI clock
• For example:
– 4 lane (32 bits/cycle) D-PHY clock to controller with controller clock at 312.5Mhz
– Input to User FIFO is 4 pixels per clock and pixel width is 16-bit pixels
– 4 * 16 = 64 bits to user interface each cycle User clock can run at 312.5Mhz/2 to keep up with MIPI rate

System Clock Byte Clock

from PHY
156.25Mhz 312.5Mhz

clk Command ui_clk clk Command clk

Req/Type/Cnt Req/Type/Cnt
CSI-2 TX User FIFO
Command Ack Command Ack
User CSI-2 TX PPI Interface to PHY
Application Controller
4 * 8 bits/lane
Pixel Read Enable Pixel Read Enable
32 bits/cycle
Pixel Data Pixel Data
4 pixels, 16 bit wide 4 pixels, 16 bit wide

© 2022 MIPI Alliance, Inc. 14

Safety

© 2022 MIPI Alliance, Inc.

Driving Automation Levels
• Most autopilot falls under SAE
Level 2, partial driving
automation, which includes
hands-free driving, adaptive
cruise control, lane change, etc.
• While Level 2 and higher are
often talked about, SAE Level 0
and SAE Level 1 also include
features that require advanced
sensors and cameras such as:
– Forward Collision Warning (FCW)
– Automatic Emergency Braking
(AEC)
– Blind sport warning
– Lane departure warning

© 2022 MIPI Alliance, Inc. 16

ISO26262 Automotive Functional Safety Standard

• ASIL Ratings go from Quality Management (QM) then ASIL-A to ASIL-D

• ASIL-C and ASIL-D are considered “high-level safety critical”
• Rating is determined by 3 parameters: Colors correspond to QM (Green), ASIL-A (Yellow) through ASID-D (Dark peach)
1. Exposure (E): probability of vehicle being in risky situation that causes damage to person or property (E1 is lowest, E4 is highest)
2. Controllability (C): extent driver can take control of vehicle (C1 is easy to control, C3 is difficult to control)
3. Severity (S): seriousness or intensity of damage or consequences to life of people (S1 is light/moderate injury to S3 is life threatening)

© 2022 MIPI Alliance, Inc. 17

Safety Element out of Context (SEooC)
• IP vendor has no prior knowledge of Assumed Assumed
system Requirements Design of
• Safety Manual to cover from Top-level Top-level
System System
– Assumptions on Safety Functions
– Assumptions of Use (AoU)
• Target ASIL
• Operating Conditions for PHY Safety Element
External
– Safety Mechanisms Assumed out of Context
• Internal Requirements (SEooC)
• External
– Safety lifecycle of the IP

© 2022 MIPI Alliance, Inc. 18

Hardware Safety Analyses
Design failure mode and effect analysis Failure modes, effects, and diagnostic analysis
(DFMEA) (FMEDA)

FMEDA is used to analyze hardware IP

random faults

DFMEA is used for systematic failure analysis

© 2022 MIPI Alliance, Inc. 19

CSI-2 Controller Safety BIST For Autonomous Driving
• CSI-2 Controller Safety Targeted BIST
– Concentrates on fault detection mechanisms
– Tests ECC, CRC, parity and other safety mechanisms on power up
– Can be trigged by system software/firmware for period testing during operation
• Advantages over system level BIST
– Deeper test of error detection mechanisms
– Always run even if BIST is not done at a higher level
– Creates an early confirmation that IP is functional
– Provides confidence that safety mechanisms are functional
– Provides a guaranteed level of safety coverage that system BIST may not provide

BIST does not replace SOC testing:

Interfaces are not targeted for application specific traffic patterns
© 2022 MIPI Alliance, Inc. 20
CSI-2 Controller Protection and Safety Watchdog
• Data Protection
– Parity protection on pixels and pixel buffers
– MIPI protocol header ECC and packet data CRC
• Critical Logic Protection Through Redundancy
– Data Formatting, Packing Logic, critical state machines
– Critical logic blocks compare duplicate outputs
• Data Order Protection
– Packet number field in user interface FIFO stores packet numbers
– Internal error triggered on nonsequential packet numbers
• Watchdog
– Area/Power efficient way to detect faults that stop the Controller from making forward progress
– Follows a sequence of input and output signals and checks internal signals

© 2022 MIPI Alliance, Inc. 21

CSI-2 Controller Fault Recovery
• Detection Reporting to System
– IRQ on error back to the system
• Fault Recovery
– Support soft reset of all state machines and critical logic
• Methodology
– CSI-2 Controller assert IRQ on error
– System software/firmware reads error status enables
• Soft reset only the Rambus CSI-2 Controller
• Soft reset entire system
• Hard reset entire system
• System software/firmware can also execute BIST before beginning
sending/receiving real packets
© 2022 MIPI Alliance, Inc. 22
Customer Use Case: NXP Camera-to-Processor Connection
• Camera sub-assembly has the camera
sensor and supporting circuitry for
image capture and for organizing the
image data for transmission using a D-
PHY Tx macro
• Camera image is serialized and sent
across to an Image Signal Processing
sub-assembly containing the supporting
circuitry for receiving the data – a D-PHY
Rx macro
• Physical connection between the Tx and
Rx side is made using a MIPI interface
• Mixel provided MIPI D-PHY with Rambus
CSI-2 controller to NXP for this
application

© 2022 MIPI Alliance, Inc. 23

Customer Use Case: GEO GW5 ADAS Vision
• ADAS uses CSI-2 as communication link
between the sensors and CVP
– CVP drives processed video through a MIPI
CSI-2 TX interface to an automotive SerDes
link, which drives a high-resolution display
through MIPI D-PHY based DSI interface
– For computer vision applications, the
unprocessed high-resolution video is
transmitted from a CVP to a computer
vision processor through another pair of
MIPI CSI-2 TX/RX interfaces.
• Data sent over MIPI D-PHY physical layer
• GEO GW3 and GEO GW4 also used
Mixel MIPI D-PHY with Rambus CSI-2
controller

© 2022 MIPI Alliance, Inc. 24

Customer Use Case: Renesas (IDT) Automotive SoC
• Multi-channel sensor interface w/ parallel
channel acquisition
– Data acquisition control operating at hundreds of
MHz
– Actuator control and signal generation using GHz
time base (supports cm level resolution)
• CSI-2 Tx 4 Lane D-PHY @2.5GBs/lane
– Up to 10Gbps to HOST-ECU
– Tx FIFO for bandwidth matching
• Safety Requirements
– AEC-Q100 Auto grade 1, supporting Tj=150°C
– ISO26262 ASIL-B supported by
• Internal safety mechanisms for data path, configuration,
and supply monitoring
• External safety mechanisms for module level data path
and supply monitoring

© 2022 MIPI Alliance, Inc. 25

Summary
• MIPI was designed from the ground up for low-power applications
and as a result, MIPI CSI-2 has become the de facto standard for
automotive camera and sensors
• Cars require increasing number of sensors as automotive features
progress from Level 0 to Level 3 and higher
• MIPI PHY and MIPI CSI-2 targeted to automotive applications
need to support many testability features, including in-system
testing
• By leveraging ASIL-B Functional Safety deliverables and/or ASIL-B
certified IP, automotive SoC designers can target higher ASIL levels
for their product, essential for mission critical applications
© 2022 MIPI Alliance, Inc. 26
© 2022 MIPI Alliance, Inc.
© 2022 MIPI Alliance, Inc.