#include <jevois/DNN/PostProcessorPose.H>

Post-Processor for neural network pipeline, for human/animal/other pose detection (skeleton)

This is the last step in a deep neural network processing Pipeline. It estimates the pose (skeleton) of humans or animals seen by the camera.

Definition at line 35 of file PostProcessorPose.H.

Inheritance diagram for jevois::dnn::PostProcessorPose:

[legend]

Collaboration diagram for jevois::dnn::PostProcessorPose:

[legend]

Public Member Functions
virtual	~PostProcessorPose ()
	Destructor.

void	freeze (bool doit) override
	Freeze/unfreeze parameters that users should not change while running.

void	process (std::vector< cv::Mat > const &outs, PreProcessor *preproc) override
	Process outputs and draw/send some results.

void	report (jevois::StdModule mod, jevois::RawImage outimg=nullptr, jevois::OptGUIhelper *helper=nullptr, bool overlay=true, bool idle=false) override
	Report what happened in last process() to console/output video/GUI.

std::vector< ObjDetect > const &	latestDetections () const
	Get the latest detections, use with caution, not thread-safe.

std::vector< PoseSkeleton > const &	latestSkeletons () const
	Get the latest skeletons, use with caution, not thread-safe.

Public Member Functions inherited from jevois::dnn::PostProcessor
virtual	~PostProcessor ()
	Destructor.

Public Member Functions inherited from jevois::Component
	Component (std::string const &instance)
	Constructor.

virtual	~Component ()
	Virtual destructor for safe inheritance.

template<class Comp , typename... Args>
std::shared_ptr< Comp >	addSubComponent (std::string const &instance, Args &&...args)
	Pseudo-constructor: construct and add another component as a subcomponent of this one.

template<class Comp >
void	removeSubComponent (std::shared_ptr< Comp > &component)
	Remove a sub-Component from this Component, by shared_ptr.

void	removeSubComponent (std::string const &instance, bool warnIfNotFound=true)
	Remove a sub-Component from this Component, by instance name.

template<class Comp = jevois::Component>
std::shared_ptr< Comp >	getSubComponent (std::string const &instance) const
	Get a sub-component by instance name.

bool	isTopLevel () const
	Returns true if this component is top-level, i.e., its parent is jevois::Manager.

bool	initialized () const
	Has this component been initialized yet?

std::string const &	className () const
	The class name of this component.

std::string const &	instanceName () const
	The instance name of this component.

template<typename T >
std::vector< std::string >	setParamVal (std::string const &paramdescriptor, T const &val)
	Set a parameter value.

template<typename T >
void	setParamValUnique (std::string const &paramdescriptor, T const &val)
	Set a parameter value, simple version assuming only one parameter match.

template<typename T >
std::vector< std::pair< std::string, T > >	getParamVal (std::string const &paramdescriptor) const
	Get parameter(s) value(s) by descriptor.

template<typename T >
T	getParamValUnique (std::string const &paramdescriptor) const
	Get a parameter value, simple version assuming only one parameter match.

std::vector< std::string >	setParamString (std::string const &paramdescriptor, std::string const &val)
	Set a parameter value, by string.

void	setParamStringUnique (std::string const &paramdescriptor, std::string const &val)
	Set a parameter value by string, simple version assuming only one parameter match.

std::vector< std::pair< std::string, std::string > >	getParamString (std::string const &paramdescriptor) const
	Get a parameter value, by string.

std::string	getParamStringUnique (std::string const &paramdescriptor) const
	Get a parameter value by string, simple version assuming only one parameter match.

void	freezeParam (std::string const &paramdescriptor, bool doit)
	Freeze/unfreeze a parameter, by name, see ParameterBase::freeze()

void	freezeAllParams (bool doit)
	Freeze all parameters.

std::string	descriptor () const
	Get our full descriptor (including all parents) as [Instancename]:[...]:[...].

void	setParamsFromFile (std::string const &filename)
	Set some parameters from a file.

std::istream &	setParamsFromStream (std::istream &is, std::string const &absfile)
	Set some parameters from an open stream.

virtual void	paramInfo (std::shared_ptr< UserInterface > s, std::map< std::string, std::string > &categs, bool skipFrozen, std::string const &cname="", std::string const &pfx="")
	Get machine-oriented descriptions of all parameters.

void	foreachParam (std::function< void(std::string const &compname, ParameterBase *p)> func, std::string const &cname="")
	Run a function on every param we hold.

template<typename T >
std::shared_ptr< DynamicParameter< T > >	addDynamicParameter (std::string const &name, std::string const &description, T const &defaultValue, ParameterCategory const &category)
	Add a new parameter after the Component has already been constructed.

template<typename T , template< typename > class ValidValuesSpec>
std::shared_ptr< DynamicParameter< T > >	addDynamicParameter (std::string const &name, std::string const &description, T const &defaultValue, ValidValuesSpec< T > const &validValuesSpec, ParameterCategory const &category)
	Add a new parameter after the Component has already been constructed.

template<typename T >
void	setDynamicParameterCallback (std::string const &name, std::function< void(T const &)> cb, bool callnow=true)
	Register a callback with a previously created dynamic parameter.

void	removeDynamicParameter (std::string const &name, bool throw_if_not_found=true)
	Remove a previously added dynamic parameter.

void	setPath (std::string const &path)
	Assign a filesystem path to this component.

std::filesystem::path	absolutePath (std::filesystem::path const &path="")
	If given path is relative (not starting with /), prepend the Component path to it.

Public Member Functions inherited from jevois::ParameterRegistry
virtual	~ParameterRegistry ()
	Virtual destructor for safe inheritance.

Protected Member Functions
void	onParamChange (postprocessor::classes const &param, std::string const &val) override

void	onParamChange (postprocessor::skeleton const &param, std::string const &val) override

Protected Member Functions inherited from jevois::Component
virtual void	preInit ()
	Called before all sub-Components are init()ed.

virtual void	postInit ()
	Called after all sub-Components are init()ed.

virtual void	preUninit ()
	Called before all sub-Components are uninit()ed.

virtual void	postUninit ()
	Called after all sub-Components are uninit()ed.

Protected Member Functions inherited from jevois::ParameterRegistry
void	addParameter (ParameterBase *const param)
	The Parameter class uses this method to register itself on construction with its owning Component.

void	removeParameter (ParameterBase *const param)
	The Parameter class uses this method to un-register itself on destruction with its owning Component.

void	callbackInitCall ()
	For all parameters that have a callback which has never been called, call it with the default param value.

Protected Attributes
std::map< int, std::string >	itsLabels
	Mapping from object ID to class name.

HailoROIPtr	itsROI = std::make_shared<HailoROI>(HailoROI(HailoBBox(0.0f, 0.0f, 1.0f, 1.0f)))

std::pair< std::vector< KeyPt >, std::vector< PairPairs > >	itsKeypointsAndPairs

std::shared_ptr< jevois::PoseSkeletonDefinition >	itsSkeletonDef

cv::Size	itsImageSize

std::vector< ObjDetect >	itsDetections

std::vector< PoseSkeleton >	itsSkeletons

Related Symbols
(Note that these are not member symbols.)
	JEVOIS_DECLARE_PARAMETER (jthresh, float, "Joint confidence threshold for pose detection", 50.0F, jevois::Range< float >(0.0F, 100.0F), ParamCateg)
	Parameter.

	JEVOIS_DEFINE_ENUM_CLASS (PoseType,(YOLOv8HAILO)(YOLOv8)(YOLOv8t))
	Enum.

	JEVOIS_DECLARE_PARAMETER (posetype, PoseType, "Type of pose output format", PoseType::YOLOv8HAILO, PoseType_Values, ParamCateg)
	Parameter.

	JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK (skeleton, std::string, "YAML file name for definition of pose skeleton. " "Defines number of joints and links, their names, and their " "colors for drawing. If file name is relative, it will be with respect " "to " JEVOIS_SHARE_PATH, "dnn/skeletons/Coco17.yml", ParamCateg)
	Parameter.

Constructor & Destructor Documentation

◆ ~PostProcessorPose()

jevois::dnn::PostProcessorPose::~PostProcessorPose ( )

virtual

Destructor.

Definition at line 35 of file PostProcessorPose.C.

Member Function Documentation

◆ freeze()

void jevois::dnn::PostProcessorPose::freeze ( bool doit )

overridevirtual

Freeze/unfreeze parameters that users should not change while running.

Implements jevois::dnn::PostProcessor.

Definition at line 39 of file PostProcessorPose.C.

◆ latestDetections()

std::vector< jevois::ObjDetect > const & jevois::dnn::PostProcessorPose::latestDetections ( ) const

Get the latest detections, use with caution, not thread-safe.

This returns a reference to our internal vector of detections. That vector will get overwritten every time process() is called. It is ok to use this after you have called process() on a given frame, or, if using a Pipeline, after you have called Pipeline::process(). Do not hold this ref past the end of the current video frame. If you need to keep a persistent copy of the data, make a deep copy of the vector.

Definition at line 555 of file PostProcessorPose.C.

◆ latestSkeletons()

std::vector< jevois::PoseSkeleton > const & jevois::dnn::PostProcessorPose::latestSkeletons ( ) const

Get the latest skeletons, use with caution, not thread-safe.

This returns a reference to our internal vector of skeletons. That vector will get overwritten every time process() is called. It is ok to use this after you have called process() on a given frame, or, if using a Pipeline, after you have called Pipeline::process(). Do not hold this ref past the end of the current video frame. If you need to keep a persistent copy of the data, make a deep copy of the vector.

Definition at line 559 of file PostProcessorPose.C.

◆ onParamChange() [1/2]

void jevois::dnn::PostProcessorPose::onParamChange	(	postprocessor::classes const &	param,
		std::string const &	val
	)

overrideprotected

Definition at line 47 of file PostProcessorPose.C.

References jevois::absolutePath(), JEVOIS_SHARE_PATH, and jevois::dnn::readLabelsFile().

◆ onParamChange() [2/2]

void jevois::dnn::PostProcessorPose::onParamChange	(	postprocessor::skeleton const &	param,
		std::string const &	val
	)

overrideprotected

Definition at line 54 of file PostProcessorPose.C.

◆ process()

void jevois::dnn::PostProcessorPose::process	(	std::vector< cv::Mat > const &	outs,
		PreProcessor *	preproc
	)

overridevirtual

Process outputs and draw/send some results.

Implements jevois::dnn::PostProcessor.

Definition at line 60 of file PostProcessorPose.C.

References jevois::dnn::PreProcessor::b2i(), jevois::dnn::PreProcessor::blobsize(), jevois::dnn::clamp(), jevois::Component::descriptor(), filter_keypoints(), jevois::Manager::getComponent(), jevois::dnn::getLabel(), jevois::dnn::PreProcessor::imagesize(), LFATAL, jevois::PoseSkeleton::linkDefinitions(), jevois::PoseSkeleton::links, LTHROW, jevois::PoseSkeleton::nodes, o, jevois::dnn::NetworkHailo::outputInfos(), jevois::dnn::shapestr(), jevois::dnn::sigmoid(), jevois::dnn::softmax_dfl(), jevois::split(), and yolov8pose_postprocess().

◆ report()

void jevois::dnn::PostProcessorPose::report	(	jevois::StdModule *	mod,
		jevois::RawImage *	outimg = `nullptr`,
		jevois::OptGUIhelper *	helper = `nullptr`,
		bool	overlay = `true`,
		bool	idle = `false`
	)

overridevirtual

Report what happened in last process() to console/output video/GUI.

Implements jevois::dnn::PostProcessor.

Definition at line 472 of file PostProcessorPose.C.

References jevois::GUIhelper::drawCircle(), jevois::rawimage::drawCircle(), jevois::GUIhelper::drawLine(), jevois::rawimage::drawLine(), jevois::GUIhelper::drawRect(), jevois::rawimage::drawRect(), jevois::GUIhelper::drawText(), jevois::rawimage::Font10x20, jevois::yuyv::LightGreen, o, jevois::StdModule::sendSerialObjDetImg2D(), jevois::sformat(), jevois::dnn::stringToRGBA(), and jevois::rawimage::writeText().

Friends And Related Symbol Documentation

◆ JEVOIS_DECLARE_PARAMETER() [1/2]

JEVOIS_DECLARE_PARAMETER	(	jthresh	,
		float	,
		"Joint confidence threshold for pose detection"	,
		50.	0F,
		jevois::Range< float >	0.0F, 100.0F,
		ParamCateg
	)

◆ JEVOIS_DECLARE_PARAMETER() [2/2]

JEVOIS_DECLARE_PARAMETER	(	posetype	,
		PoseType	,
		"Type of pose output format"	,
		PoseType::YOLOv8HAILO	,
		PoseType_Values	,
		ParamCateg
	)

◆ JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK()

JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK	(	skeleton	,
		std::string	,
		"YAML file name for definition of pose skeleton. " "Defines number of joints and	links,
		their	names,
		and their " "colors for drawing. If file name is	relative,
		it will be with respect " "to "	JEVOIS_SHARE_PATH,
		"dnn/skeletons/Coco17.yml"	,
		ParamCateg
	)

◆ JEVOIS_DEFINE_ENUM_CLASS()

JEVOIS_DEFINE_ENUM_CLASS	(	PoseType	,
		(YOLOv8HAILO)(YOLOv8)(YOLOv8t)
	)

Member Data Documentation

◆ itsDetections

std::vector<ObjDetect> jevois::dnn::PostProcessorPose::itsDetections

protected

Definition at line 87 of file PostProcessorPose.H.

◆ itsImageSize

cv::Size jevois::dnn::PostProcessorPose::itsImageSize

protected

Definition at line 86 of file PostProcessorPose.H.

◆ itsKeypointsAndPairs

std::pair<std::vector<KeyPt>, std::vector<PairPairs> > jevois::dnn::PostProcessorPose::itsKeypointsAndPairs

protected

Definition at line 83 of file PostProcessorPose.H.

◆ itsLabels

std::map<int, std::string> jevois::dnn::PostProcessorPose::itsLabels

protected

Mapping from object ID to class name.

Definition at line 79 of file PostProcessorPose.H.

◆ itsROI

HailoROIPtr jevois::dnn::PostProcessorPose::itsROI = std::make_shared<HailoROI>(HailoROI(HailoBBox(0.0f, 0.0f, 1.0f, 1.0f)))

protected

Definition at line 82 of file PostProcessorPose.H.

◆ itsSkeletonDef

std::shared_ptr<jevois::PoseSkeletonDefinition> jevois::dnn::PostProcessorPose::itsSkeletonDef

protected

Definition at line 85 of file PostProcessorPose.H.

◆ itsSkeletons

std::vector<PoseSkeleton> jevois::dnn::PostProcessorPose::itsSkeletons

protected

Definition at line 88 of file PostProcessorPose.H.

The documentation for this class was generated from the following files:

include/jevois/DNN/PostProcessorPose.H
include/jevois/DNN/PostProcessor.H
src/jevois/DNN/PostProcessorPose.C

Public Member Functions

Protected Member Functions

Protected Attributes

Related Symbols

Constructor & Destructor Documentation

◆ ~PostProcessorPose()

Member Function Documentation

◆ freeze()

◆ latestDetections()

◆ latestSkeletons()

◆ onParamChange() [1/2]

◆ onParamChange() [2/2]

◆ process()

◆ report()

Friends And Related Symbol Documentation

◆ JEVOIS_DECLARE_PARAMETER() [1/2]

◆ JEVOIS_DECLARE_PARAMETER() [2/2]

◆ JEVOIS_DECLARE_PARAMETER_WITH_CALLBACK()

◆ JEVOIS_DEFINE_ENUM_CLASS()

Member Data Documentation

◆ itsDetections

◆ itsImageSize

◆ itsKeypointsAndPairs

◆ itsLabels

◆ itsROI

◆ itsSkeletonDef

◆ itsSkeletons