JeVoisBase  1.10
JeVois Smart Embedded Machine Vision Toolkit Base Modules
Share this page:
PythonParallel.py
Go to the documentation of this file.
1 import libjevois as jevois # FIXME: this may be the cause of our troubles noted in the doc below
2 import cv2
3 import numpy as np
4 import multiprocessing as mp
5 
6 # ###################################################################################################
7 ## Image processing function, several instances will run in parallel
8 # It is just defined as a free function here to emphasize the fact that class member data will not be shared anyway
9 # across the workers. NOTE: Do not attempt to use jevois.sendSerial() or any other functions of module jevois here, it
10 # will not work because computefunc() is running in a completely different process than jevois-daemon is. Just return
11 # any strings you wish to send out, and let your process() or processNoUSB() function do the sendSerial() instead.
12 def computefunc(inimggray, th1, th2):
13  return cv2.Canny(inimggray, threshold1 = th1, threshold2 = th2, apertureSize = 3, L2gradient = False)
14 
15 ## Simple example of parallel image processing using OpenCV in Python on JeVois
16 #
17 # This module by default simply converts the input image to a grayscale OpenCV image, and then applies the Canny edge
18 # detection algorithm, 4 times running in parallel with 4 different edge coarseness settings. The resulting image is
19 # simply the 4 horizontally stacked results from the 4 parallel runs. Try to edit it to do something else!
20 #
21 # True multi-threaded processing is not supported by Python (the python \b threading module does not allow concurrent
22 # execution of several threads of python code). Parallel processing is somewhat feasible using the \b mutiprocessing
23 # python module, which is a process-based multiprocessing approach. Note that there are significant costs to
24 # parallelizing code over multiple processes, the main one being that data needs to be transferred back and forth
25 # between processes, using pipes, sockets, or other mechanisms. For machine vision, this is a significant problem as the
26 # amount of data (streaming video) that needs to be packaged, transferred, and unpacked is high. C++ is the preferred
27 # way of developping multi-threaded JeVois modules, where std::async() makes multi-threaded programming easy.
28 #
29 # \fixme <b>You should consider this module highly experimental and buggy!</b> This module is currently not working well
30 # when running with USB output. There is some internal issue with using the Python \b multiprocessing module in
31 # JeVois. Just creating a python process pool interferes with our USB video output driver, even if we simply create the
32 # pool and destroy it immediately without using it at all. Once the python process pool has been created, any subsequent
33 # attempt to change video format will fail with a video buffer allocation error. This module may still be useful for
34 # robotics applications where parallel python processing is needed but no video output to USB is necessary).
35 #
36 # \fixme This module is hence not enabled by default. You need to edit <b>JEVOIS:/config/videomappings.cfg</b>, and
37 # uncomment the line with \jvmod{PythonParallel} in it, to enable it.
38 #
39 # \fixme Conflated with this problem is the fact that guvcview sometimes, when it starts, turns streaming on, then grabs
40 # only 5 frames, then stream off, then set same video format again, and then stream on again. We are not sure why
41 # guvcview is doing this, however, this breaks this module since the second streamon fails as it is unable to allocate
42 # video buffers.
43 #
44 # Using this module
45 # -----------------
46 #
47 # One way we have been able to use this module with USB video outputs is: start `guvcview -f yuyv -x 352x288` (launches
48 # \jvmod{PythonSandbox}), then use the pull-down menu to select 1280x240 (switches to \jvmod{PythonParallel})
49 #
50 # This module is best used with no USB video outputs. Connect to JeVois over serial and issue:
51 # \verbatim
52 # setpar serout USB # to get text results through serial-over-USB, or use Hard if you want results on the 4-pin serial
53 # setpar serlog USB # to get log messages through serial-over-USB, or use Hard if you want them on the 4-pin serial
54 # setmapping2 YUYV 320 240 25.0 JeVois PythonParallel
55 # streamon
56 # \endverbatim
57 #
58 # As noted above, once you have loaded this module, any later attempts to change format again will fail.
59 #
60 # Creating your own module
61 # ------------------------
62 #
63 # See http://jevois.org/tutorials for tutorials on getting started with programming JeVois in Python without having
64 # to install any development software on your host computer.
65 #
66 # @author Laurent Itti
67 #
68 # @videomapping YUYV 1280 240 25.0 YUYV 320 240 25.0 JeVois PythonParallel # CAUTION: has major issues
69 # @videomapping NONE 0 0 0.0 YUYV 320 240 25.0 JeVois PythonParallel
70 # @email itti\@usc.edu
71 # @address University of Southern California, HNB-07A, 3641 Watt Way, Los Angeles, CA 90089-2520, USA
72 # @copyright Copyright (C) 2018 by Laurent Itti, iLab and the University of Southern California
73 # @mainurl http://jevois.org
74 # @supporturl http://jevois.org/doc
75 # @otherurl http://iLab.usc.edu
76 # @license GPL v3
77 # @distribution Unrestricted
78 # @restrictions None
79 # @ingroup modules
81  # NOTE: Do not use a constructor in python multiprocessing JeVois modules, as it would be executed several times, in
82  # each spawned worker process. Just do everything in the process() and/or processNoUSB() functions instead. Since
83  # python allows runtime creation of new class data members, you can simply check whether they already exist, and, if
84  # not, create them as new data members of the JeVois module.
85 
86  # ###################################################################################################
87  ## Process function with no USB output
88  def processNoUSB(self, inframe):
89  # Create a parallel processing pool and a timer, if needed (on first frame only):
90  if not hasattr(self, 'pool'):
91  # create a multiprocessing pool, not specifying the number of processes, to use the number of cores:
92  self.pool = mp.Pool()
93  # Instantiate a JeVois Timer to measure our processing framerate:
94  self.timer = jevois.Timer("PythonParallel", 100, jevois.LOG_INFO)
95 
96  # Get the next camera image (may block until it is captured) and convert it to OpenCV GRAY:
97  inimggray = inframe.getCvGRAY()
98 
99  # Start measuring image processing time (NOTE: does not account for input conversion time):
100  self.timer.start()
101 
102  # Detect edges using the Canny algorithm from OpenCV, launching 4 instances in parallel:
103  futures = [ self.pool.apply_async(computefunc, args = (inimggray, 10*x, 20*x, )) for x in range(1,5) ]
104 
105  # Collect the results, handling any exception thrown by the workers. Here, we make sure we get() all the results
106  # first, then rethrow the last exception received, if any, so that we do ensure that all results will be
107  # collected before we bail out on an exception:
108  results = []
109  error = 0
110  for ii in range(4):
111  try: results.append(futures[ii].get(timeout = 10))
112  except Exception as e: error = e
113  if error: raise error
114 
115  # In real modules, we would do something with the results... Here, just report their size:
116  str = ""
117  for ii in range(4):
118  h, w = results[ii].shape
119  str += "Canny {}: {}x{} ".format(ii, w, h)
120 
121  # Send a message to serout:
122  jevois.sendSerial(str)
123 
124  # Report frames/s info to serlog:
125  self.timer.stop()
126 
127 
128  # ###################################################################################################
129  ## Process function with USB output
130  def process(self, inframe, outframe):
131  # Create a parallel processing pool and a timer, if needed (on first frame only):
132  if not hasattr(self, 'pool'):
133  # create a multiprocessing pool, not specifying the number of processes, to use the number of cores:
134  self.pool = mp.Pool()
135  # Instantiate a JeVois Timer to measure our processing framerate:
136  self.timer = jevois.Timer("PythonParallel", 100, jevois.LOG_INFO)
137 
138  # Get the next camera image (may block until it is captured) and convert it to OpenCV GRAY:
139  inimggray = inframe.getCvGRAY()
140 
141  # Start measuring image processing time (NOTE: does not account for input conversion time):
142  self.timer.start()
143 
144  # Detect edges using the Canny algorithm from OpenCV, launching 4 instances in parallel:
145  futures = [ self.pool.apply_async(computefunc, args = (inimggray, 10*x, 20*x, )) for x in range(1,5) ]
146 
147  # Collect the results, handling any exception thrown by the workers. Here, we make sure we get() all the results
148  # first, then rethrow the last exception received, if any, so that we do ensure that all results will be
149  # collected before we bail out on an exception:
150  results = []
151  error = 0
152  for ii in range(4):
153  try: results.append(futures[ii].get(timeout = 10))
154  except Exception as e: error = e
155  if error: raise error
156 
157  # Aggregate the worker result images into a single output image:
158  outimggray = np.hstack(results)
159 
160  # Write frames/s info from our timer into the edge map (NOTE: does not account for output conversion time):
161  fps = self.timer.stop()
162  height, width = outimggray.shape
163  cv2.putText(outimggray, fps, (3, height - 6), cv2.FONT_HERSHEY_SIMPLEX, 0.5, 255, 1, cv2.LINE_AA)
164 
165  # Convert our GRAY output image to video output format and send to host over USB:
166  outframe.sendCvGRAY(outimggray)
167 
168  # ###################################################################################################
169  ## Required multiprocessing pool cleanup to avoid hanging on module unload
170  # JeVois engine calls uninit(), if present, before destroying our module. FIXME: python multiprocessing still messes
171  # up the system deeply, we become unable to allocate mmap'd UVC buffers after this module has been loaded.
172  def uninit(self):
173  # Close and join the worker pool if any, so we don't leave lingering processes:
174  if hasattr(self, 'pool'):
175  self.pool.close()
176  self.pool.terminate()
177  del self.pool
178 
179  # ###################################################################################################
180  ## Parse a serial command forwarded to us by the JeVois Engine, return a string
181  #def parseSerial(self, str):
182  # return "ERR: Unsupported command"
183 
184  # ###################################################################################################
185  ## Return a string that describes the custom commands we support, for the JeVois help message
186  #def supportedCommands(self):
187  # return ""
188 
189 
190 
def processNoUSB(self, inframe)
Process function with no USB output.
def process(self, inframe, outframe)
Process function with USB output.
def uninit(self)
Required multiprocessing pool cleanup to avoid hanging on module unload JeVois engine calls uninit()...
def computefunc(inimggray, th1, th2)
Image processing function, several instances will run in parallel It is just defined as a free functi...
Simple example of parallel image processing using OpenCV in Python on JeVois.