Automated Driving Toolbox User’s Guide

Sensor Configuration and Coordinate System Transformations
	Coordinate Systems in Automated Driving Toolbox
		World Coordinate System
		Vehicle Coordinate System
		Sensor Coordinate System
		Spatial Coordinate System
		Pattern Coordinate System
	Calibrate a Monocular Camera
		Estimate Intrinsic Parameters
		Place Checkerboard for Extrinsic Parameter Estimation
		Estimate Extrinsic Parameters
		Configure Camera Using Intrinsic and Extrinsic Parameters
Ground Truth Labeling and Verification
	Get Started with Ground Truth Labelling
	Load Ground Truth Signals to Label
		Load Timestamps
		Open Ground Truth Labeler App
		Load Signals from Data Sources
		Configure Signal Display
	Label Ground Truth for Multiple Signals
		Create Label Definitions
		Label Video Using Automation
		Label Point Cloud Sequence Using Automation
		Label with Sublabels and Attributes Manually
		Label Scene Manually
		View Label Summary
		Save App Session
	Export and Explore Ground Truth Labels for Multiple Signals
	Sources vs. Signals in Ground Truth Labeling
	Keyboard Shortcuts and Mouse Actions for Ground Truth Labeler
		Label Definitions
		Frame Navigation and Time Interval Settings
		Labeling Window
		Cuboid Resizing and Moving
		Polyline Drawing
		Polygon Drawing
		Zooming, Panning, and Rotating
		App Sessions
	Control Playback of Signal Frames for Labeling
		Signal Frames
		Main Signal
		Change Main Signal
		Display All Timestamps
		Specify Timestamps
		Frame Display and Automation
	Label Lidar Point Clouds for Object Detection
		Set Up Lidar Point Cloud Labeling
		Zoom, Pan, and Rotate Frame
		Hide Ground
		Label Cuboid
		Modify Cuboid Label
		Apply Cuboids to Multiple Frames
		Configure Display
	Create Class for Loading Custom Ground Truth Data Sources
		Custom Class Folder
		Class Definition
		Class Properties
		Method to Customize Load Panel
		Methods to Get Load Panel Data and Load Data Source
		Method to Read Frames
		Use Predefined Data Source Classes
Tracking and Sensor Fusion
	Visualize Sensor Data and Tracks in Bird's-Eye Scope
		Open Model and Scope
		Find Signals
		Run Simulation
		Organize Signal Groups (Optional)
		Update Model and Rerun Simulation
		Save and Close Model
	Linear Kalman Filters
		Motion Model
		Measurement Models
		Filter Loop
		Built-In Motion Models in trackingKF
		Example: Estimate 2-D Target States Using trackingKF
	Extended Kalman Filters
		State Update Model
		Measurement Model
		Extended Kalman Filter Loop
		Predefined Extended Kalman Filter Functions
		Example: Estimate 2-D Target States with Angle and Range Measurements Using trackingEKF
Planning, Mapping, and Control
	Display Data on OpenStreetMap Basemap
	Read and Visualize HERE HD Live Map Data
		Enter Credentials
		Configure Reader to Search Specific Catalog
		Create Reader for Specific Map Tiles
		Read Map Layer Data
		Visualize Map Layer Data
	HERE HD Live Map Layers
		Road Centerline Model
		HD Lane Model
		HD Localization Model
	Rotations, Orientations, and Quaternions for Automated Driving
		Quaternion Format
		Quaternion Creation
		Quaternion Math
		Extract Quaternions from Transformation Matrix
	Control Vehicle Velocity
	Velocity Profile of Straight Path
	Velocity Profile of Path with Curve and Direction Change
	Plan Path Using A-Star Path Planners
	Use ROS Logger App to Save ROS Messages from Simulink
Cuboid Driving Scenario Simulation
	Create Driving Scenario Interactively and Generate Synthetic Sensor Data
		Create Driving Scenario
		Add a Road
		Add Lanes
		Add Barriers
		Add Vehicles
		Add a Pedestrian
		Add Sensors
		Generate Synthetic Sensor Data
		Save Scenario
	Keyboard Shortcuts and Mouse Actions for Driving Scenario Designer
		Canvas Operations
		Road Operations
		Actor Operations
		Preview Actor Times of Arrival
		Barrier Placement Operations
		Sensor Operations
		File Operations
	Prebuilt Driving Scenarios in Driving Scenario Designer
		Choose a Prebuilt Scenario
		Modify Scenario
		Generate Synthetic Sensor Data
		Save Scenario
	Euro NCAP Driving Scenarios in Driving Scenario Designer
		Choose a Euro NCAP Scenario
		Modify Scenario
		Generate Synthetic Detections
		Save Scenario
	Cuboid Versions of 3D Simulation Scenes in Driving Scenario Designer
		Choose 3D Simulation Scenario
		Modify Scenario
		Save Scenario
		Recreate Scenario in Simulink for 3D Environment
	Create Reverse Motion Driving Scenarios Interactively
		Three-Point Turn Scenario
		Add Road
		Add Vehicle
		Add Trajectory
		Run Simulation
		Adjust Trajectory Using Specified Yaw Values
	Generate INS Sensor Measurements from Interactive Driving Scenario
		Import Road Network
		Add Actor and Trajectory
		Smooth the Trajectory
		Add INS Sensor
		Simulate Scenario
		Export to MATLAB and Explore Sensor Data
		Export Scenario and Sensor to a Simulink Model
	Import ASAM OpenDRIVE Roads into Driving Scenario
		Import ASAM OpenDRIVE File
		Inspect Roads
		Add Actors and Sensors to Scenario
		Generate Synthetic Detections
		Save Scenario
	Export Driving Scenario to ASAM OpenDRIVE File
		Load Scenario File
		Export to ASAM OpenDRIVE
		Inspect Exported Scenario
		Limitations
	Import HERE HD Live Map Roads into Driving Scenario
		Set Up HERE HDLM Credentials
		Specify Geographic Coordinates
		Select Region Containing Roads
		Select Roads to Import
		Import Roads
		Compare Imported Roads Against Map Data
		Save Scenario
	Import OpenStreetMap Data into Driving Scenario
		Select OpenStreetMap File
		Select Roads to Import
		Import Roads
		Compare Imported Roads Against Map Data
		Save Scenario
	Import Zenrin Japan Map API 3.0 (Itsumo NAVI API 3.0) into Driving Scenario
		Set Up Zenrin Japan Map API 3.0 (Itsumo NAVI API 3.0) Credentials
		Specify Geographic Coordinates
		Select Region Containing Roads
		Select Roads to Import
		Import Roads
		Compare Imported Roads Against Map Data
		Save Scenario
	Create Driving Scenario Variations Programmatically
	Generate Sensor Blocks Using Driving Scenario Designer
	Test Open-Loop ADAS Algorithm Using Driving Scenario
	Test Closed-Loop ADAS Algorithm Using Driving Scenario
	Automate Control of Intelligent Vehicles by Using Stateflow Charts
	Simulate INS Block
	Generate INS Measurements from Driving Scenario in Simulink
	Create Roads with Multiple Lane Specifications Using Driving Scenario Designer
		Open Driving Scenario Designer
		Add Road
		Define Multiple Lane Specifications
		Next Steps
	Export Driving Scenario to ASAM OpenSCENARIO File
		Load Scenario File
		Export to ASAM OpenSCENARIO
		ASAM OpenSCENARIO Representations
		Limitations
3D Simulation – User's Guide
	Unreal Engine Simulation for Automated Driving
		Unreal Engine Simulation Blocks
		Algorithm Testing and Visualization
	Unreal Engine Simulation Environment Requirements and Limitations
		Software Requirements
		Minimum Hardware Requirements
		Limitations
	How Unreal Engine Simulation for Automated Driving Works
		Communication with 3D Simulation Environment
		Block Execution Order
	Coordinate Systems for Unreal Engine Simulation in Automated Driving Toolbox
		World Coordinate System
		Vehicle Coordinate System
	Choose a Sensor for Unreal Engine Simulation
	Simulate Simple Driving Scenario and Sensor in Unreal Engine Environment
	Depth and Semantic Segmentation Visualization Using Unreal Engine Simulation
	Visualize Sensor Data from Unreal Engine Simulation Environment
	Customize Unreal Engine Scenes for Automated Driving
	Install Support Package for Customizing Scenes
		Verify Software and Hardware Requirements
		Install Support Package
		Set Up Scene Customization Using Support Package
	Migrate Projects Developed Using Prior Support Packages
	Customize Scenes Using Simulink and Unreal Editor
		Open Unreal Editor from Simulink
		Reparent Actor Blueprint
		Create or Modify Scenes in Unreal Editor
		Run Simulation
	Package Custom Scenes into Executable
		Package Scene into Executable Using Unreal Editor
		Simulate Scene from Executable in Simulink
	Apply Labels to Unreal Scene Elements for Semantic Segmentation and Object Detection
	Create Top-Down Map of Unreal Engine Scene
		Capture Screenshot
		Convert Screenshot to Map
	Place Cameras on Actors in the Unreal Editor
		Place Camera on Static Actor
		Place Camera on Vehicle in Custom Project
	Build Light in Unreal Editor
		Use AutoVrtlEnv Project Lighting in Custom Scene
	Create Empty Project in Unreal Engine
	Prepare Custom Vehicle Mesh for the Unreal Editor
		Step 1: Setup Bone Hierarchy
		Step 2: Assign Materials
		Step 3: Export Mesh and Armature
		Step 4: Import Mesh to Unreal Editor
		Step 5: Set Block Parameters
RoadRunner Scenario Scenario Simulation
	Overview of Simulating RoadRunner Scenarios with MATLAB and Simulink
		Author RoadRunner Actor Behavior Using Simulink or MATLAB System Objects
		Associate Actor Behavior in RoadRunner Scenario
		Publish Actor Behavior
		Tune Actor Parameters
		Simulate Scenario in RoadRunner
		Control Scenario Simulation Using MATLAB
		Inspect Simulation Results Using Data Logging
	Simulate RoadRunner Scenarios with Actors Modeled in Simulink
		Author RoadRunner Actor Behavior Using Simulink
		Author RoadRunner Actor Behavior Using User-Defined Actions in Simulink
		Associate Actor Behavior in RoadRunner and Simulate Scenario
	Simulate RoadRunner Scenarios with Actors Modeled in MATLAB
		Build Custom MATLAB System Object Behavior
		Associate Actor Behavior in RoadRunner
	Publish Actor Behavior as Proto File, Package or Action Asset
		Generate Behavior Proto File for Simulink or MATLAB System Object Behavior
		Generate Package from Simulink Model or MATLAB System Object
		Generation Action Asset File from Simulink Model
Featured Examples
	Configure Monocular Fisheye Camera
	Annotate Video Using Detections in Vehicle Coordinates
	Read Data From ADTF DAT Files
	Read Sensor Messages from IDC file
	Automate Ground Truth Labeling Across Multiple Signals
	Automate Ground Truth Labeling of Lane Boundaries
	Automate Ground Truth Labeling for Semantic Segmentation
	Automate Attributes of Labeled Objects
	Evaluate Lane Boundary Detections Against Ground Truth Data
	Evaluate and Visualize Lane Boundary Detections Against Ground Truth
	Visual Perception Using Monocular Camera
	Create 360° Bird's-Eye-View Image Around a Vehicle
	Perception-Based Parking Spot Detection Using Unreal Engine Simulation
	Train a Deep Learning Vehicle Detector
	Ground Plane and Obstacle Detection Using Lidar
	Build Map and Localize Using Segment Matching
	Build a Map with Lidar Odometry and Mapping (LOAM) Using Unreal Engine Simulation
	Code Generation for Tracking and Sensor Fusion
	Forward Collision Warning Using Sensor Fusion
	Adaptive Cruise Control with Sensor Fusion
	Forward Collision Warning Application with CAN FD and TCP/IP
	Multiple Object Tracking Tutorial
	Track Multiple Vehicles Using a Camera
	Track Vehicles Using Lidar: From Point Cloud to Track List
	Sensor Fusion Using Synthetic Radar and Vision Data
	Sensor Fusion Using Synthetic Radar and Vision Data in Simulink
	Autonomous Emergency Braking with Sensor Fusion
	Visualize Sensor Coverage, Detections, and Tracks
	Extended Object Tracking of Highway Vehicles with Radar and Camera
	Track-to-Track Fusion for Automotive Safety Applications
	Track-to-Track Fusion for Automotive Safety Applications in Simulink
	Visual-Inertial Odometry Using Synthetic Data
	Lane Following Control with Sensor Fusion and Lane Detection
	Track-Level Fusion of Radar and Lidar Data
	Track-Level Fusion of Radar and Lidar Data in Simulink
	Track Vehicles Using Lidar Data in Simulink
	Grid-Based Tracking in Urban Environments Using Multiple Lidars
	Track Multiple Lane Boundaries with a Global Nearest Neighbor Tracker
	Generate Code for a Track Fuser with Heterogeneous Source Tracks
	Highway Vehicle Tracking with Multipath Radar Reflections
	Extended Object Tracking of Highway Vehicles with Radar and Camera in Simulink
	Grid-based Tracking in Urban Environments Using Multiple Lidars in Simulink
	Object Tracking and Motion Planning Using Frenet Reference Path
	Asynchronous Sensor Fusion and Tracking with Retrodiction
	Extended Target Tracking with Multipath Radar Reflections in Simulink
	Processor-in-the-Loop Verification of JPDA Tracker for Automotive Applications
	Scenario Generation from Recorded Vehicle Data
	Generate Lane Information from Recorded Data
	Lane Keeping Assist with Lane Detection
	Model Radar Sensor Detections
	Model Vision Sensor Detections
	Radar Signal Simulation and Processing for Automated Driving
	Simulate Radar Ghosts Due to Multipath Return
	Create Driving Scenario Programmatically
	Create Actor and Vehicle Trajectories Programmatically
	Define Road Layouts Programmatically
	Simulate Vehicle Parking Maneuver in Driving Scenario
	Automated Parking Valet
	Automated Parking Valet in Simulink
	Visualize Automated Parking Valet Using Cuboid Simulation
	Highway Trajectory Planning Using Frenet Reference Path
	Motion Planning in Urban Environments Using Dynamic Occupancy Grid Map
	Code Generation for Path Planning and Vehicle Control
	Use HERE HD Live Map Data to Verify Lane Configurations
	Localization Correction Using Traffic Sign Data from HERE HD Maps
	Build a Map from Lidar Data
	Build a Map from Lidar Data Using SLAM
	Create Occupancy Grid Using Monocular Camera and Semantic Segmentation
	Lateral Control Tutorial
	Highway Lane Change
	Visual Localization in a Parking Lot
	Design Lane Marker Detector Using Unreal Engine Simulation Environment
	Select Waypoints for Unreal Engine Simulation
	Visualize Automated Parking Valet Using Unreal Engine Simulation
	Simulate Vision and Radar Sensors in Unreal Engine Environment
	Highway Lane Following
	Automate Testing for Highway Lane Following
	Traffic Light Negotiation
	Design Lidar SLAM Algorithm Using Unreal Engine Simulation Environment
	Lidar Localization with Unreal Engine Simulation
	Develop Visual SLAM Algorithm Using Unreal Engine Simulation
	Automatic Scenario Generation
	Automatic Scenario Variant Generation for Testing AEB Systems
	Generate Scenario from Recorded GPS and Lidar Data
	Highway Lane Following with RoadRunner Scene
	Export Multiple Scenes Using MATLAB
	Convert Scenes Between Formats Using MATLAB Functions
	Simulate a RoadRunner Scenario Using MATLAB Functions
	Build Simple Roads Programatically Using RoadRunner HD Map
	Build Pikes Peak RoadRunner 3D Scene Using RoadRunner HD Map
	Traffic Light Negotiation with Unreal Engine Visualization
	Generate Code for Lane Marker Detector
	Highway Lane Following with Intelligent Vehicles
	Forward Vehicle Sensor Fusion
	Generate Code for Vision Vehicle Detector
	Automate Testing for Lane Marker Detector
	Generate Code for Highway Lane Following Controller
	Automate Testing for Highway Lane Following Controls and Sensor Fusion
	Generate Code for Highway Lane Change Planner
	Surround Vehicle Sensor Fusion
	Build Occupancy Map from 3-D Lidar Data using SLAM
	Automate Testing for Vision Vehicle Detector
	Automate Testing for Forward Vehicle Sensor Fusion
	Automate Testing for Highway Lane Following Controller
	Automate Testing for Highway Lane Change
	Visualize Logged Data from Unreal Engine Simulation
	Automate Real-Time Testing for Highway Lane Following Controller
	Generate C++ Message Interfaces for Lane Following Controls and Sensor Fusion
	Automate Testing for Autonomous Emergency Braking
	Autonomous Emergency Braking with Vehicle Variants
	Automate Real-Time Testing for Forward Vehicle Sensor Fusion
	Highway Lane Change Planner and Controller
	Intersection Movement Assist Using Vehicle-to-Vehicle Communication
	Traffic Light Negotiation Using Vehicle-to-Everything Communication
	Trajectory Follower with RoadRunner Scenario
	Speed Action Follower with RoadRunner Scenario
	Highway Lane Change Planner with RoadRunner Scenario
	Truck Platooning Using Vehicle-to-Vehicle Communication
	Automate PIL Testing for Forward Vehicle Sensor Fusion
	Highway Lane Following with RoadRunner Scenario
	Autonomous Emergency Braking with RoadRunner Scenario
	Automate Testing for Scenario Variants of AEB System
Scenario Generation
	Overview of Scenario Generation from Recorded Sensor Data
		Preprocess Input Data
		Extract Ego Vehicle Information
		Extract Scene Information
		Extract Non-Ego Actor Information
		Create, Simulate, and Export Scenario
	Smooth GPS Waypoints for Ego Localization
	Preprocess Lane Detections for Scenario Generation
	Improve Ego Vehicle Localization
	Extract Lane Information from Recorded Camera Data for Scene Generation
	Generate High Definition Scene from Lane Detections
	Extract Vehicle Track List from Recorded Camera Data for Scenario Generation
	Extract Vehicle Track List from Recorded Lidar Data for Scenario Generation
	Generate Scenario from Actor Track List and GPS Data
	Generate RoadRunner Scene from Recorded Lidar Data
	Generate RoadRunner Scenario from Recorded Sensor Data
Scenario Variant Generation
	Overview of Scenario Variant Generation
		Parameter Extraction
		Parameter Variation
		Scenario Generation
	Generate Scenario Variants by Modifying Actor Dimensions
	Generate Scenario Variants for Testing ACC Systems
	Generate Variants of ACC Target Cut-In Scenario
	Generate Scenario Variants for Testing AEB Pedestrian Systems
	Generate Scenario Variants for Lane Keep Assist Testing