- Create the custom detect program which utilizes zetabot camera.
- Within your individual computers, execute the following mission.
Writing Custom segNet Program
---------------------------------
Similar to how we created a new python file in our team assignment, generate a new python file and name it ``02_11-2. tao_dashcamnet_camera.py``.
Create a new python file in the Jupyter Notebook Environment:
- Press the blue plus button on the top left corner of the web.
.. thumbnail:: /_images/ai_training/add_plus.png
|
- Create a new python file by pressing the ``Python File`` button
.. thumbnail:: /_images/ai_training/pick_python.png
|
- Rename the untitiled python file to ``02_11-2. tao_dashcamnet_camera.py``
- On the new python file, import the libraries necessary. For our detect task, we need to import the Jetson inference library modules and jetson utility library modules
- ``argparse``: This library contains modules that are responsbile for bringing and intitializing the flags or parameters set by the user when envoking the program.
- ``sys``: this library allows us to manipulate/ utilize system functions within our python programs.
- ``jetson_inference``: This library contains all the pre-built networks that can be used for inference task and a functions that would allow for custom models to be used for inference tasks.
- ``detectNet``: We are importing detectNet module for our dashcam detection task.
- ``jetson_utils``: This library contains modules that are responsible for processing input and output sources along with output stream methods. We will be importing the following modules:
- ``videoSource``: used to process input source (whether it is a camera, an image, or a video).
- ``videoOutput``: used to process the output stream.
.. code-block:: python
import sys
import argparse
from jetson_inference import detectNet
from jetson_utils import videoSource, videoOutput
- After all the libraries are imported, initialize the parser variable with ``argparse.ArgumentParser`` module.
For our mission, we must receive the network name, and Camera output channel name. Additionally we add our minor functinoality flags.
.. code-block:: python
# parse the command line
# For our mission, We recieve the network name, and Camera name.
# Set up argument parser, so that command line parameters can be read within the program
parser = argparse.ArgumentParser(description="Locate objects in a live camera stream using an object detection DNN.",
formatter_class=argparse.RawTextHelpFormatter,
epilog=detectNet.Usage() + videoSource.Usage() + videoOutput.Usage())
# Major Functionality parameters (required from the user)
parser.add_argument("input_CAMERA", type=str, default="", nargs='?', help="use csi://0 for Raspberry pi Camera")
parser.add_argument("--network", type=str, default="dashcamnet", help="pre-trained model to load")
# Minor Functionality parameters (optional)
parser.add_argument("--overlay", type=str, default="box,labels,conf", help="detection overlay flags (e.g. --overlay=box,labels,conf)\nvalid combinations are: 'box', 'labels', 'conf', 'none'")
parser.add_argument("--threshold", type=float, default=0.5, help="minimum detection threshold to use")
- Initialize opt variable to hold all the user-set flags in a list form. If the user has set no flags, terminate the program:
.. code-block:: python
# If no parameter is given from the user, shut the program down
try:
opt = parser.parse_known_args()[0]
except:
print("")
parser.print_help()
sys.exit(0)
- Initialize the necessary variables. Since we wish to infer a network with a camera and show the results with our output stream we will need:
1. ``net`` variable for holding the nvidia pre-built networks. For this mission we are using fcn-mobilenet network.
2. ``input`` variable for handling the input stream. Using the ``opt`` variable created in our previous step, we will bring in input_CAMERA to set our videoSource.
3. ``display`` variable for handling the output stream. Although we are accessing the code remotely on our remote computer, the zetabot is equipped with a touch screen display. The display is set on ``DISPLAY://0``
4. ``buffer`` variable for managing buffer.
.. code-block:: python
# create video sources and outputs
input = videoSource(opt.input_CAMERA, argv=sys.argv)
output = videoOutput("DISPLAY://0", argv=sys.argv)
# load the object detection network
net = detectNet(opt.network, sys.argv, opt.threshold)
- For this task we are utilizing our camera. On our previous trials, we had to to an inference on a single image. The program could recieve the one image infer it with the network and output a single result.
But with a camera, we need to repeatedly run the inference so that we may capture the incoming frames from the camera and output a constant stream of results.
- We may achieve this by running a while loop until an envoked output stream window is killed by the user.
.. code-block:: python
# process frames until the user exits
while display.IsStreaming():
- Within the while loop:
- Capture the current frame from the camera and infer the image using the trained model.
.. code-block:: python
# capture the next image
img = input.Capture()
if img is None: # timeout
continue
# detect objects in the image (with overlay)
detections = net.Detect(img, overlay=opt.overlay)
- Render the result output and update the title bar of the output window.
.. code-block:: python
# render the output image
output.Render(buffers.output)
# update the title bar
output.SetStatus("{:s} | Network {:.0f} FPS".format(opt.network, net.GetNetworkFPS()))
# exit on input/output EOS
if not input.IsStreaming() or not output.IsStreaming():
break
Executing the Custom Program
-----------------------------
- Open the ``02_11-3. tao_dashcamnet_camera.ipynb`` notebook.
.. thumbnail:: /_images/ai_tao/dashcam_camera.png
|
- Run the cell code which initializes the input/ output stream of the environment as well as the CAMERA variable, which will be the flag that determines the input vairable for the program to be a camera stream.
.. code-block:: python
%env DISPLAY=:0
%env csi=:0
%env CAMERA=csi://0
- Check if your python notebook can read the python code you have written:
.. code-block:: python
!cat /home/zeta/notebook/lecture/'2.AI Training Examples'/'02_11-2. tao_dashcamnet_camera.py'
- Execute the tao_dashcamnet_camera python code.
*Note* that we are setting our major functions,
- ``--network``: to set which networks to use in our detect task.
- ``input_CAMERA``: to set which input stream will be used for our task. It is being set to CAMERA environment variable which holds ``csi://0`` as a string.
.. code-block:: python
!python3 /home/zeta/notebook/lecture/'2.AI Training Examples'/'02_11-2. tao_dashcamnet_camera.py' $CAMERA