JeVoisBase  1.16
JeVois Smart Embedded Machine Vision Toolkit Base Modules
Share this page:
PythonParallel.py
Go to the documentation of this file.
1 import pyjevois
2 if pyjevois.pro: import libjevoispro as jevois
3 else: import libjevois as jevois
4 import cv2
5 import numpy as np
6 import multiprocessing as mp
7 
8 # ###################################################################################################
9 ## Image processing function, several instances will run in parallel
10 # It is just defined as a free function here to emphasize the fact that class member data will not be shared anyway
11 # across the workers. NOTE: Do not attempt to use jevois.sendSerial() or any other functions of module jevois here, it
12 # will not work because computefunc() is running in a completely different process than jevois-daemon is. Just return
13 # any strings you wish to send out, and let your process() or processNoUSB() function do the sendSerial() instead.
14 def computefunc(inimggray, th1, th2):
15  return cv2.Canny(inimggray, threshold1 = th1, threshold2 = th2, apertureSize = 3, L2gradient = False)
16 
17 ## Simple example of parallel image processing using OpenCV in Python on JeVois
18 #
19 # This module by default simply converts the input image to a grayscale OpenCV image, and then applies the Canny edge
20 # detection algorithm, 4 times running in parallel with 4 different edge coarseness settings. The resulting image is
21 # simply the 4 horizontally stacked results from the 4 parallel runs. Try to edit it to do something else!
22 #
23 # True multi-threaded processing is not supported by Python (the python \b threading module does not allow concurrent
24 # execution of several threads of python code). Parallel processing is somewhat feasible using the \b mutiprocessing
25 # python module, which is a process-based multiprocessing approach. Note that there are significant costs to
26 # parallelizing code over multiple processes, the main one being that data needs to be transferred back and forth
27 # between processes, using pipes, sockets, or other mechanisms. For machine vision, this is a significant problem as the
28 # amount of data (streaming video) that needs to be packaged, transferred, and unpacked is high. C++ is the preferred
29 # way of developping multi-threaded JeVois modules, where std::async() makes multi-threaded programming easy.
30 #
31 # \fixme <b>You should consider this module highly experimental and buggy!</b> This module is currently not working well
32 # when running with USB output. There is some internal issue with using the Python \b multiprocessing module in
33 # JeVois. Just creating a python process pool interferes with our USB video output driver, even if we simply create the
34 # pool and destroy it immediately without using it at all. Once the python process pool has been created, any subsequent
35 # attempt to change video format will fail with a video buffer allocation error. This module may still be useful for
36 # robotics applications where parallel python processing is needed but no video output to USB is necessary).
37 #
38 # \fixme This module is hence not enabled by default. You need to edit <b>JEVOIS:/config/videomappings.cfg</b>, and
39 # uncomment the line with \jvmod{PythonParallel} in it, to enable it.
40 #
41 # \fixme Conflated with this problem is the fact that guvcview sometimes, when it starts, turns streaming on, then grabs
42 # only 5 frames, then stream off, then set same video format again, and then stream on again. We are not sure why
43 # guvcview is doing this, however, this breaks this module since the second streamon fails as it is unable to allocate
44 # video buffers.
45 #
46 # Using this module
47 # -----------------
48 #
49 # One way we have been able to use this module with USB video outputs is: start `guvcview -f yuyv -x 352x288` (launches
50 # \jvmod{PythonSandbox}), then use the pull-down menu to select 1280x240 (switches to \jvmod{PythonParallel})
51 #
52 # This module is best used with no USB video outputs. Connect to JeVois over serial and issue:
53 # \verbatim
54 # setpar serout USB # to get text results through serial-over-USB, or use Hard if you want results on the 4-pin serial
55 # setpar serlog USB # to get log messages through serial-over-USB, or use Hard if you want them on the 4-pin serial
56 # setmapping2 YUYV 320 240 25.0 JeVois PythonParallel
57 # streamon
58 # \endverbatim
59 #
60 # As noted above, once you have loaded this module, any later attempts to change format again will fail.
61 #
62 # Creating your own module
63 # ------------------------
64 #
65 # See http://jevois.org/tutorials for tutorials on getting started with programming JeVois in Python without having
66 # to install any development software on your host computer.
67 #
68 # @author Laurent Itti
69 #
70 # @videomapping YUYV 1280 240 25.0 YUYV 320 240 25.0 JeVois PythonParallel # CAUTION: has major issues
71 # @videomapping NONE 0 0 0.0 YUYV 320 240 25.0 JeVois PythonParallel
72 # @email itti\@usc.edu
73 # @address University of Southern California, HNB-07A, 3641 Watt Way, Los Angeles, CA 90089-2520, USA
74 # @copyright Copyright (C) 2018 by Laurent Itti, iLab and the University of Southern California
75 # @mainurl http://jevois.org
76 # @supporturl http://jevois.org/doc
77 # @otherurl http://iLab.usc.edu
78 # @license GPL v3
79 # @distribution Unrestricted
80 # @restrictions None
81 # @ingroup modules
83  # NOTE: Do not use a constructor in python multiprocessing JeVois modules, as it would be executed several times, in
84  # each spawned worker process. Just do everything in the process() and/or processNoUSB() functions instead. Since
85  # python allows runtime creation of new class data members, you can simply check whether they already exist, and, if
86  # not, create them as new data members of the JeVois module.
87 
88  # ###################################################################################################
89  ## Process function with no USB output
90  def processNoUSB(self, inframe):
91  # Create a parallel processing pool and a timer, if needed (on first frame only):
92  if not hasattr(self, 'pool'):
93  # create a multiprocessing pool, not specifying the number of processes, to use the number of cores:
94  self.pool = mp.Pool()
95  # Instantiate a JeVois Timer to measure our processing framerate:
96  self.timer = jevois.Timer("PythonParallel", 100, jevois.LOG_INFO)
97 
98  # Get the next camera image (may block until it is captured) and convert it to OpenCV GRAY:
99  inimggray = inframe.getCvGRAY()
100 
101  # Start measuring image processing time (NOTE: does not account for input conversion time):
102  self.timer.start()
103 
104  # Detect edges using the Canny algorithm from OpenCV, launching 4 instances in parallel:
105  futures = [ self.pool.apply_async(computefunc, args = (inimggray, 10*x, 20*x, )) for x in range(1,5) ]
106 
107  # Collect the results, handling any exception thrown by the workers. Here, we make sure we get() all the results
108  # first, then rethrow the last exception received, if any, so that we do ensure that all results will be
109  # collected before we bail out on an exception:
110  results = []
111  error = 0
112  for ii in range(4):
113  try: results.append(futures[ii].get(timeout = 10))
114  except Exception as e: error = e
115  if error: raise error
116 
117  # In real modules, we would do something with the results... Here, just report their size:
118  str = ""
119  for ii in range(4):
120  h, w = results[ii].shape
121  str += "Canny {}: {}x{} ".format(ii, w, h)
122 
123  # Send a message to serout:
124  jevois.sendSerial(str)
125 
126  # Report frames/s info to serlog:
127  self.timer.stop()
128 
129 
130  # ###################################################################################################
131  ## Process function with USB output
132  def process(self, inframe, outframe):
133  # Create a parallel processing pool and a timer, if needed (on first frame only):
134  if not hasattr(self, 'pool'):
135  # create a multiprocessing pool, not specifying the number of processes, to use the number of cores:
136  self.pool = mp.Pool()
137  # Instantiate a JeVois Timer to measure our processing framerate:
138  self.timer = jevois.Timer("PythonParallel", 100, jevois.LOG_INFO)
139 
140  # Get the next camera image (may block until it is captured) and convert it to OpenCV GRAY:
141  inimggray = inframe.getCvGRAY()
142 
143  # Start measuring image processing time (NOTE: does not account for input conversion time):
144  self.timer.start()
145 
146  # Detect edges using the Canny algorithm from OpenCV, launching 4 instances in parallel:
147  futures = [ self.pool.apply_async(computefunc, args = (inimggray, 10*x, 20*x, )) for x in range(1,5) ]
148 
149  # Collect the results, handling any exception thrown by the workers. Here, we make sure we get() all the results
150  # first, then rethrow the last exception received, if any, so that we do ensure that all results will be
151  # collected before we bail out on an exception:
152  results = []
153  error = 0
154  for ii in range(4):
155  try: results.append(futures[ii].get(timeout = 10))
156  except Exception as e: error = e
157  if error: raise error
158 
159  # Aggregate the worker result images into a single output image:
160  outimggray = np.hstack(results)
161 
162  # Write frames/s info from our timer into the edge map (NOTE: does not account for output conversion time):
163  fps = self.timer.stop()
164  height, width = outimggray.shape
165  cv2.putText(outimggray, fps, (3, height - 6), cv2.FONT_HERSHEY_SIMPLEX, 0.5, 255, 1, cv2.LINE_AA)
166 
167  # Convert our GRAY output image to video output format and send to host over USB:
168  outframe.sendCvGRAY(outimggray)
169 
170  # ###################################################################################################
171  ## Required multiprocessing pool cleanup to avoid hanging on module unload
172  # JeVois engine calls uninit(), if present, before destroying our module. FIXME: python multiprocessing still messes
173  # up the system deeply, we become unable to allocate mmap'd UVC buffers after this module has been loaded.
174  def uninit(self):
175  # Close and join the worker pool if any, so we don't leave lingering processes:
176  if hasattr(self, 'pool'):
177  self.pool.close()
178  self.pool.terminate()
179  del self.pool
180 
181  # ###################################################################################################
182  ## Parse a serial command forwarded to us by the JeVois Engine, return a string
183  #def parseSerial(self, str):
184  # return "ERR: Unsupported command"
185 
186  # ###################################################################################################
187  ## Return a string that describes the custom commands we support, for the JeVois help message
188  #def supportedCommands(self):
189  # return ""
190 
191 
192 
PythonParallel.PythonParallel.uninit
def uninit(self)
Required multiprocessing pool cleanup to avoid hanging on module unload JeVois engine calls uninit(),...
Definition: PythonParallel.py:174
PythonParallel.PythonParallel.processNoUSB
def processNoUSB(self, inframe)
Process function with no USB output.
Definition: PythonParallel.py:90
PythonParallel.PythonParallel.process
def process(self, inframe, outframe)
Process function with USB output.
Definition: PythonParallel.py:132
PythonParallel.PythonParallel
Simple example of parallel image processing using OpenCV in Python on JeVois.
Definition: PythonParallel.py:82
PythonParallel.computefunc
def computefunc(inimggray, th1, th2)
Image processing function, several instances will run in parallel It is just defined as a free functi...
Definition: PythonParallel.py:14
PythonParallel.PythonParallel.pool
pool
Definition: PythonParallel.py:94
PythonParallel.PythonParallel.timer
timer
Definition: PythonParallel.py:96
jevois::Timer