Main MRPT website > C++ reference for MRPT 1.4.0
tracking.h
Go to the documentation of this file.
1/* +---------------------------------------------------------------------------+
2 | Mobile Robot Programming Toolkit (MRPT) |
3 | http://www.mrpt.org/ |
4 | |
5 | Copyright (c) 2005-2016, Individual contributors, see AUTHORS file |
6 | See: http://www.mrpt.org/Authors - All rights reserved. |
7 | Released under BSD License. See details in http://www.mrpt.org/License |
8 +---------------------------------------------------------------------------+ */
9
10#ifndef mrpt_vision_tracking_H
11#define mrpt_vision_tracking_H
12
13#include <mrpt/vision/types.h>
15
18#include <mrpt/utils/CImage.h>
21
23
24#include <memory> // for auto_ptr, unique_ptr
25
26namespace mrpt
27{
28 namespace vision
29 {
30 /** \addtogroup vision_tracking Feature detection and tracking
31 * \ingroup mrpt_vision_grp
32 * @{ */
33
34 /** A virtual interface for all feature trackers, implementing the part of feature tracking that is common to any specific tracker implementation.
35 * This class provides a quite robust tracking of features, avoiding as many outliers as possible but not all of them:
36 * more robust tracking would require application-specific information and could be done in a number of very different approaches,
37 * so this class will not try to do any kind of RANSAC or any other advanced outlier rejection; instead, it should
38 * be done by the users or the classes that employ this class.
39 *
40 * The basic usage of this class is as follows:
41 * \code
42 * CFeatureTracker_KL tracker; // Note: CFeatureTracker_KL is the most robust implementation for now.
43 * tracker.extra_params["add_new_features"] = 1; // Enable detection of new features, not only tracking
44 * tracker.extra_params[...] = ...
45 * // ....
46 * CFeatureList theFeats; // The list of features
47 * mrpt::utils::CImage previous_img, current_img;
48 *
49 * while (true) {
50 * current_img = ... // Grab new image.
51 * if ( previous_img_is_ok )
52 * tracker.trackFeatures(previous_img, current_img, theFeats);
53 * previous_img = current_img;
54 * }
55 * \endcode
56 *
57 * Below follows the list of optional parameters for "extra_params" which can be set
58 * and will be understood by this base class for any specific tracker implementation.
59 * Note that all parameters are double's, but boolean flags are emulated by the values 0.0 (false) and 1.0 (true).
60 *
61 * List of parameters:
62 * <table border="1" >
63 * <tr><td align="center" > <b>Parameter name</b> </td> <td align="center" > <b>Default value</b> </td> <td align="center" > <b>Comments</b> </td> </tr>
64 * <tr><td align="center" > add_new_features </td> <td align="center" > 0 </td>
65 * <td> If set to "1", the class will not only track existing features, but will also perform (after doing the actual tracking) an efficient
66 * search for new features with the FAST detector, and will add them to the passed "CFeatureList" if they fulfill a set of restrictions,
67 * as stablished by the other parameters (see <i>add_new_feat_min_separation</i>,<i>add_new_feat_max_features</i>,<i>minimum_KLT_response_to_add</i>).
68 * </td> </tr>
69 * <tr><td align="center" > add_new_feat_min_separation </td> <td align="center" > 15 </td>
70 * <td> If <i>add_new_features</i>==1, this is the minimum separation (in pixels) to any other (old, or new) feature for it
71 * being considered a candidate to be added.
72 * </td> </tr>
73 * <tr><td align="center" > desired_num_features_adapt </td> <td align="center" > (img_width*img_height)/512 </td>
74 * <td> If <i>add_new_features</i>==1, the threshold of the FAST(ER) feature detector is dynamically adapted such as the number of
75 * raw FAST keypoints is around this number. This number should be much higher than the real desired numbre of features, since this
76 * one includes many features concentrated in space which are later discarded for the minimum distance.
77 * </td> </tr>
78 * <tr><td align="center" > desired_num_features </td> <td align="center" > 100 </td>
79 * <td> If <i>add_new_features</i>==1, the target number of the patch associated to each feature will be updated with every N'th frame. </td> </tr>
80 * <tr><td align="center" > add_new_feat_patch_size </td> <td align="center" > 11 </td>
81 * <td> If <i>add_new_features</i>==1, for each new added feature, this is the size of the patch to be extracted around the keypoint (set to 0 if patches are not required at all).
82 * </td> </tr>
83 * <tr><td align="center" > minimum_KLT_response_to_add </td> <td align="center" > 10 </td>
84 * <td> If <i>add_new_features</i>==1, this sets the minimum KLT response of candidate FAST features to be added in each frame, if they also fulfil the other restrictions (e.g. min.distance).
85 * </td> </tr>
86 * <tr><td align="center" > check_KLT_response_every </td> <td align="center" > 0 </td>
87 * <td> If >0, it will compute the KLT response at each feature point every <i>N</i> frames
88 * and those below <i>minimum_KLT_response</i> will be marked as "lost" in their "track_status" field.
89 * </td> </tr>
90 * <tr><td align="center" > minimum_KLT_response </td> <td align="center" > 5 </td>
91 * <td> See explanation of <i>check_KLT_response_every</i>.
92 * </td> </tr>
93 * <tr><td align="center" > KLT_response_half_win </td> <td align="center" > 4 </td>
94 * <td> When computing the KLT response of features (see <i>minimum_KLT_response</i> and <i>minimum_KLT_response_to_add</i>),
95 * the window centered at the point for its estimation will be of size (2*W+1)x(2*W+1), with <i>W</i> being this parameter value.
96 * </td> </tr>
97 * <tr><td align="center" > update_patches_every </td> <td align="center" > 0 </td>
98 * <td> If !=0, the patch associated to each feature will be updated with every N'th frame. </td> </tr>
99 * <tr><td align="center" > remove_lost_features </td> <td align="center" > 0 </td>
100 * <td> If !=0, out-of-bound features or those lost while tracking, will be automatically removed from the list of features.
101 * Otherwise, the user will have to manually remove them by checking the track_status field. </td> </tr>
102 * </table>
103 *
104 * This class also offers a time profiler, disabled by default (see getProfiler and enableTimeLogger).
105 *
106 * \sa CFeatureTracker_KL, the example application "track-video-features".
107 */
109 {
110 /** Optional list of extra parameters to the algorithm. */
112
113 /** Default ctor */
114 inline CGenericFeatureTracker() : m_timlog(false), m_update_patches_counter(0),m_check_KLT_counter(0),m_detector_adaptive_thres(10)
115 { }
116 /** Ctor with extra parameters */
117 inline CGenericFeatureTracker(mrpt::utils::TParametersDouble extraParams) : extra_params(extraParams), m_timlog(false), m_update_patches_counter(0),m_check_KLT_counter(0),m_detector_adaptive_thres(10)
118 { }
119 /** Dtor */
121 { }
122
123 /** Perform feature tracking from "old_img" to "new_img", with a (possibly empty) list of previously tracked features "inout_featureList".
124 * This is a list of parameters (in "extraParams") accepted by ALL implementations of feature tracker (see each derived class for more specific parameters).
125 * - "add_new_features" (Default=0). If set to "1", new features will be also added to the existing ones in areas of the image poor of features.
126 * This method does:
127 * - Convert old and new images to grayscale, if they're in color.
128 * - Call the pure virtual "trackFeatures_impl" method.
129 * - Implement the optional detection of new features if "add_new_features"!=0.
130 */
131 void trackFeatures(const mrpt::utils::CImage &old_img,const mrpt::utils::CImage &new_img,TSimpleFeatureList &inout_featureList );
132
133 /** \overload with subpixel precision */
134 void trackFeatures(const mrpt::utils::CImage &old_img,const mrpt::utils::CImage &new_img,TSimpleFeaturefList &inout_featureList );
135
136 /** \overload This overload version uses the old (and much slower) CFeatureList */
137 void trackFeatures(const mrpt::utils::CImage &old_img,const mrpt::utils::CImage &new_img,CFeatureList &inout_featureList );
138
139 /** A wrapper around the basic trackFeatures() method, but keeping the original list of features unmodified and returns the tracked ones in a new list. */
141 const mrpt::utils::CImage &old_img,
142 const mrpt::utils::CImage &new_img,
143 const vision::CFeatureList &in_featureList,
144 vision::CFeatureList &out_featureList
145 )
146 {
147 out_featureList = in_featureList;
148 std::for_each(
149 out_featureList.begin(),out_featureList.end(),
151 this->trackFeatures(old_img, new_img, out_featureList);
152 }
153
154 /** Returns a read-only reference to the internal time logger */
155 inline const mrpt::utils::CTimeLogger & getProfiler() const { return m_timlog; }
156 /** Returns a reference to the internal time logger */
157 inline mrpt::utils::CTimeLogger & getProfiler() { return m_timlog; }
158
159 /** Returns a read-only reference to the internal time logger */
160 inline void enableTimeLogger(bool enable=true) { m_timlog.enable(enable); }
161
162 /** Returns the current adaptive threshold used by the FAST(ER) detector to find out new features in empty areas */
163 inline int getDetectorAdaptiveThreshold() const { return m_detector_adaptive_thres; }
164
166 {
167 size_t raw_FAST_feats_detected; //!< In the new_img with the last adaptive threshold
168 size_t num_deleted_feats; //!< The number of features which were deleted due to OOB, bad tracking, etc... (only if "remove_lost_features" is enabled)
169 };
170
171 TExtraOutputInfo last_execution_extra_info; //!< Updated with each call to trackFeatures()
172
173 protected:
174 /** The tracking method implementation, to be implemented in children classes. */
175 virtual void trackFeatures_impl(const mrpt::utils::CImage &old_img,const mrpt::utils::CImage &new_img,TSimpleFeaturefList &inout_featureList );
176
177 /** The tracking method implementation, to be implemented in children classes. */
178 virtual void trackFeatures_impl(const mrpt::utils::CImage &old_img,const mrpt::utils::CImage &new_img,TSimpleFeatureList &inout_featureList ) = 0;
179
180 /** This version falls back to the version with TSimpleFeatureList if the derived class does not implement it. */
181 virtual void trackFeatures_impl(const mrpt::utils::CImage &old_img,const mrpt::utils::CImage &new_img,CFeatureList &inout_featureList ) = 0;
182
183 mrpt::utils::CTimeLogger m_timlog; //!< the internal time logger, disabled by default.
184
185 /** This field is clared by \a trackFeatures() before calling \a trackFeatures_impl(), and
186 * can be filled out with newly defected FAST(ER) features in the latter.
187 * If it's not the case, feats will be computed anyway if the user enabled the "add_new_features" option.
188 */
190
191 /** Adapts the threshold \a m_detector_adaptive_thres according to the real and desired number of features just detected */
193 const size_t nNewlyDetectedFeats,
194 const size_t desired_num_features);
195
196 private:
197 size_t m_update_patches_counter; //!< for use when "update_patches_every">=1
198 size_t m_check_KLT_counter; //!< For use when "check_KLT_response_every">=1
199 int m_detector_adaptive_thres; //!< For use in "add_new_features" == true
200
201 template <typename FEATLIST>
203 const mrpt::utils::CImage &old_img,
204 const mrpt::utils::CImage &new_img,
205 FEATLIST &inout_featureList );
206 };
207
208#if MRPT_HAS_CXX11
209 typedef std::unique_ptr<CGenericFeatureTracker> CGenericFeatureTrackerAutoPtr;
210#else
211 typedef std::auto_ptr<CGenericFeatureTracker> CGenericFeatureTrackerAutoPtr;
212#endif
213
214 /** Track a set of features from old_img -> new_img using sparse optimal flow (classic KL method).
215 *
216 * See CGenericFeatureTracker for a more detailed explanation on how to use this class.
217 *
218 * List of additional parameters in "extra_params" (apart from those in CGenericFeatureTracker) accepted by this class:
219 * - "window_width" (Default=15)
220 * - "window_height" (Default=15)
221 * - "LK_levels" (Default=3) Number of pyramids to build for LK tracking (this parameter only has effects when tracking with CImage's, not with CImagePyramid's).
222 * - "LK_max_iters" (Default=10) Max. number of iterations in LK tracking.
223 * - "LK_epsilon" (Default=0.1) Minimum epsilon step in interations of LK_tracking.
224 * - "LK_max_tracking_error" (Default=150.0) The maximum "tracking error" of LK tracking such as a feature is marked as "lost".
225 *
226 * \sa OpenCV's method cvCalcOpticalFlowPyrLK
227 */
229 {
230 /** Default ctor */
232 /** Ctor with extra parameters */
234
235 protected:
236 virtual void trackFeatures_impl(const mrpt::utils::CImage &old_img,const mrpt::utils::CImage &new_img,vision::CFeatureList &inout_featureList ) MRPT_OVERRIDE;
237 virtual void trackFeatures_impl(const mrpt::utils::CImage &old_img,const mrpt::utils::CImage &new_img,TSimpleFeatureList &inout_featureList ) MRPT_OVERRIDE;
238 virtual void trackFeatures_impl(const mrpt::utils::CImage &old_img,const mrpt::utils::CImage &new_img,TSimpleFeaturefList &inout_featureList ) MRPT_OVERRIDE;
239
240 private:
241 template <typename FEATLIST>
243 const mrpt::utils::CImage &old_img,
244 const mrpt::utils::CImage &new_img,
245 FEATLIST &inout_featureList );
246
247 };
248
249
250 /** Search for correspondences which are not in the same row and deletes them
251 * ...
252 */
254 CFeatureList &rightList,
256
257
258 /** Filter bad correspondences by distance
259 * ...
260 */
262 unsigned int numberOfSigmas ); // Threshold
263
264
265
266 /** @} */ // end of grouping
267 }
268}
269
270
271#endif
A class for storing images as grayscale or RGB bitmaps.
Definition CImage.h:102
A versatile "profiler" that logs the time spent within each pair of calls to enter(X)-leave(X),...
Definition CTimeLogger.h:36
A list of TMatchingPair.
A list of visual features, to be used as output by detectors, as input/output by trackers,...
Definition CFeature.h:212
void VISION_IMPEXP checkTrackedFeatures(CFeatureList &leftList, CFeatureList &rightList, vision::TMatchingOptions options)
Search for correspondences which are not in the same row and deletes them ...
void VISION_IMPEXP filterBadCorrsByDistance(mrpt::utils::TMatchingPairList &list, unsigned int numberOfSigmas)
Filter bad correspondences by distance ...
std::auto_ptr< CGenericFeatureTracker > CGenericFeatureTrackerAutoPtr
Definition tracking.h:211
#define MRPT_OVERRIDE
C++11 "override" for virtuals:
Definition mrpt_macros.h:28
This is the global namespace for all Mobile Robot Programming Toolkit (MRPT) libraries.
For usage when passing a dynamic number of (numeric) arguments to a function, by name.
Definition TParameters.h:47
An object for making smart pointers unique (ie, making copies if necessary), intended for being used ...
Track a set of features from old_img -> new_img using sparse optimal flow (classic KL method).
Definition tracking.h:229
virtual void trackFeatures_impl(const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, TSimpleFeaturefList &inout_featureList) MRPT_OVERRIDE
The tracking method implementation, to be implemented in children classes.
void trackFeatures_impl_templ(const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, FEATLIST &inout_featureList)
virtual void trackFeatures_impl(const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, TSimpleFeatureList &inout_featureList) MRPT_OVERRIDE
The tracking method implementation, to be implemented in children classes.
CFeatureTracker_KL()
Default ctor.
Definition tracking.h:231
virtual void trackFeatures_impl(const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, vision::CFeatureList &inout_featureList) MRPT_OVERRIDE
This version falls back to the version with TSimpleFeatureList if the derived class does not implemen...
CFeatureTracker_KL(mrpt::utils::TParametersDouble extraParams)
Ctor with extra parameters.
Definition tracking.h:233
size_t num_deleted_feats
The number of features which were deleted due to OOB, bad tracking, etc... (only if "remove_lost_feat...
Definition tracking.h:168
size_t raw_FAST_feats_detected
In the new_img with the last adaptive threshold.
Definition tracking.h:167
A virtual interface for all feature trackers, implementing the part of feature tracking that is commo...
Definition tracking.h:109
virtual void trackFeatures_impl(const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, TSimpleFeatureList &inout_featureList)=0
The tracking method implementation, to be implemented in children classes.
mrpt::utils::TParametersDouble extra_params
Optional list of extra parameters to the algorithm.
Definition tracking.h:111
int m_detector_adaptive_thres
For use in "add_new_features" == true.
Definition tracking.h:199
size_t m_check_KLT_counter
For use when "check_KLT_response_every">=1.
Definition tracking.h:198
void trackFeatures(const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, CFeatureList &inout_featureList)
void trackFeaturesNewList(const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, const vision::CFeatureList &in_featureList, vision::CFeatureList &out_featureList)
A wrapper around the basic trackFeatures() method, but keeping the original list of features unmodifi...
Definition tracking.h:140
void internal_trackFeatures(const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, FEATLIST &inout_featureList)
virtual void trackFeatures_impl(const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, TSimpleFeaturefList &inout_featureList)
The tracking method implementation, to be implemented in children classes.
void trackFeatures(const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, TSimpleFeaturefList &inout_featureList)
TExtraOutputInfo last_execution_extra_info
Updated with each call to trackFeatures()
Definition tracking.h:171
void trackFeatures(const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, TSimpleFeatureList &inout_featureList)
Perform feature tracking from "old_img" to "new_img", with a (possibly empty) list of previously trac...
mrpt::utils::CTimeLogger & getProfiler()
Returns a reference to the internal time logger.
Definition tracking.h:157
void updateAdaptiveNewFeatsThreshold(const size_t nNewlyDetectedFeats, const size_t desired_num_features)
Adapts the threshold m_detector_adaptive_thres according to the real and desired number of features j...
mrpt::vision::TSimpleFeatureList m_newly_detected_feats
This field is clared by trackFeatures() before calling trackFeatures_impl(), and can be filled out wi...
Definition tracking.h:189
void enableTimeLogger(bool enable=true)
Returns a read-only reference to the internal time logger.
Definition tracking.h:160
virtual void trackFeatures_impl(const mrpt::utils::CImage &old_img, const mrpt::utils::CImage &new_img, CFeatureList &inout_featureList)=0
This version falls back to the version with TSimpleFeatureList if the derived class does not implemen...
size_t m_update_patches_counter
for use when "update_patches_every">=1
Definition tracking.h:197
const mrpt::utils::CTimeLogger & getProfiler() const
Returns a read-only reference to the internal time logger.
Definition tracking.h:155
int getDetectorAdaptiveThreshold() const
Returns the current adaptive threshold used by the FAST(ER) detector to find out new features in empt...
Definition tracking.h:163
CGenericFeatureTracker(mrpt::utils::TParametersDouble extraParams)
Ctor with extra parameters.
Definition tracking.h:117
mrpt::utils::CTimeLogger m_timlog
the internal time logger, disabled by default.
Definition tracking.h:183
A structure containing options for the matching.



Page generated by Doxygen 1.9.7 for MRPT 1.4.0 SVN: at Tue Jun 13 13:45:58 UTC 2023