Adding a 3D vision sensor to V-REP

In this section, we will add another sensor to our mobile robot: an rgb-d sensor, like the one already used in Chapter 4, Simulating Robots Using ROS and Gazebo. V-REP already has a pre-built model for this sensor, but, unlike the laser scanner, it is not directly integrated with ROS. For this reason, let's modify the script associated to this sensor to stream its data via topics. Add the sensor selecting the kinect model from the Model Browser panel and drop it in the robot components. Position the kinect in the desired location and the associated script in the following way:

if (sim_call_type==sim_childscriptcall_initialization) then  
depthCam=simGetObjectHandle('kinect_depth') 
depthView=simFloatingViewAdd(0.9,0.9,0.2,0.2,0) 
colorCam=simGetObjectHandle('kinect_rgb') 
colorView=simFloatingViewAdd(0.69,0.9,0.2,0.2,0) 
glass=simGetObjectHandle('kinect_glass') 
kinect=simGetObjectHandle('kinect') 
end  
 
if(sim_call_type==sim_childscriptcall_sensing) then 
    rgbTopicName=simExtROS_enablePublisher('/rgb/image_raw',1,simros_strmcmd_get_vision_sensor_image,colorCam,0,'')      DepthTopicName=simExtROS_enablePublisher('/depth/image_raw',1,simros_strmcmd_get_vision_sensor_image,depthCam,0,'')   
infoTopicName=simExtROS_enablePublisher('/cameraInfo',1,simros_strmcmd_get_vision_sensor_info,colorCam,0,'') 
 
end 
 
if (sim_call_type==sim_childscriptcall_cleanup) then  
end  

Here, in the sim_childscriptcall_sensing section, we add the code to stream data generated by the rgb-d sensor: the colored image and the depth image, and the information concerning the camera calibration. We can run the simulation, as shown here:

$ rosrun image_view image_view image:=/rgb/image_raw

And we get the following image:

We have now simulated a differential wheeled mobile robot equipped with different kinds of sensors fully integrated with ROS. This robot model is saved in the mobile_robot.ttt simulation scene provided with the vrep_demo_pkg package.