MIT Self Driving Race Car Ultra Sonic Sensor Test

Hey Steemians , This is another video in the MIT Race Car Series project.
In this Video we carried out a few preliminary tests to test the movements by using depth sensors and Zed Camera.
If you would like to configure your car just like the one you see in the video i would recommend following this Guide .

Hardware Tested In this Video

The reason why we use the ZED camera is because it enables Real-time 3D Mapping which is essential in identifying objects and thereby enabling the robot in making smart decisions.You may find some other Camera's which may work just as well but this was the equipment provided to us for completing this project.

All the Code written for Testing Depth Sensing is included on the Github Repository here .We used Python for implementing these sensors as there are a collection of default Libraries in python which will make our life easier.

For starter's you will require the following imports to use the PyZedCamera
import pyzed.camera as zcam
import pyzed.defines as sl
import pyzed.types as tp
import pyzed.core as core


After importing all the required libraries you can now create an Object of type PyZEDCamera and use all the additional features of PyZedCamera
zed = zcam.PyZEDCamera()


Now You may initialize the depth mode and and specify the depth measurements required.
init_params = zcam.PyInitParameters()
init_params.depth_mode = sl.PyDEPTH_MODE.PyDEPTH_MODE_PERFORMANCE # Use PERFORMANCE depth mode
init_params.coordinate_units = sl.PyUNIT.PyUNIT_MILLIMETER # Use milliliter units (for depth measurements)


Finally here is the code which will allow your robot to assess depth in various images and overwrite the Electronic speed controller thereby enabling your bot to make smart decisions

    # Open the camera
    err = zed.open(init_params)
    if err != tp.PyERROR_CODE.PySUCCESS:
        exit(1)

    # Create and set PyRuntimeParameters after opening the camera
    runtime_parameters = zcam.PyRuntimeParameters()
    runtime_parameters.sensing_mode = sl.PySENSING_MODE.PySENSING_MODE_STANDARD  # Use STANDARD sensing mode

    i = 0 #counter
    image = core.PyMat()
    depth_for_display = core.PyMat()

    print('Current mode: Capture {} images as fast as possible.\nMerge the images.\nSave to pickle files.'.format(num_images))
    while i < num_images:
        # JOYSTICK
        pygame.event.pump() # keep everything current
        throttle = (j.get_axis(throttle_axis)+1)/2 # left stick
        steering = (j.get_axis(steering_axis)+1)/2 # right stick steering
        left_bumper = j.get_button(left_bumper) # left bumper is deadman/ controls whether you are saving
        exit_button = j.get_button(exit_axis) # Options button exits

        if exit_button:
            print('Exit button (options) pressed. Stopping data collection')
            break

        if left_bumper:
            # A new image is available if grab() returns PySUCCESS
            if zed.grab(runtime_parameters) == tp.PyERROR_CODE.PySUCCESS:
                # Retrieve left image
                zed.retrieve_image(image, sl.PyVIEW.PyVIEW_LEFT)
                # Retrieve left depth
                zed.retrieve_image(depth_for_display,sl.PyVIEW.PyVIEW_DEPTH)

                #convert to arrays
                data=image.get_data()
                depth_data=depth_for_display.get_data()

                # Convert images to smaller square images
                square_image_size=500
                data=cv2.resize(data,(square_image_size,square_image_size))
                depth_data=cv2.resize(depth_data,(square_image_size,square_image_size))

                merged = merge_images(data,depth_data)
                print('writing dataset/image_{0}_{1:.4f}_{2:.4f}.pickle'.format(i,throttle,steering))
                if (i%25==0):
                    print('{} images have been captured'.format(i))

                pickle.dump(merged,open( 'dataset/image_{0}_{1:.4f}_{2:.4f}.pickle'.format(i,throttle,steering), 'wb' ))
            else:
                print('image collection failed')
            # Increment the loop
            i = i + 1

    j.quit()
    print('Image capture complete')
    # Close the camera
    zed.close()

Hope you enjoyed this post, feel free to reach out if you would like to replicate this project or have any questions regarding this project.