Into

Modules

Documentation

classPiiPointMatchingOperation

#include <PiiPointMatchingOperation.h>

An abstract superclass for feature point matching operations.

Inherits PiiClassifierOperation

Inherited by PiiRansacPointMatcher

Description

This operation maintains a database of object models as feature points and quickly retrieves matching models for unknown objects. Each feature point is described by point coordinates and a feature vector (a.k.a "descriptor"). The point coordinates specify the location of the feature point with respect to the model. The operation can handle N-dimensional point coordinates, but when it comes to images, two-dimensional points are used.

There can be an unlimited number of models, each containing an unlimited number of feature points. Furthermore, each model may have an associated label. This makes it possible to have multiple models of the same object class.

In training, the operation collects all incoming data to a model database. Each incoming model may optionally be associated with a label and a model location. In classification, incoming descriptors are matched against the collected database, and the most likely matching models will be found.

Inputs

featuresa N-by-M matrix in which each row stores a feature point descriptor. All numeric matrix types will be accepted, but the operation internally uses float for calculations. Inherited from PiiClassifierOperation. Note that PiiFeaturePointMatcher uses matrices instead of row vectors as the feature type.
labelan optional input that gives a label to the model described by points and features. This makes it possible to have many models for a single class. Any numeric type will be accepted. This input is used only in when collecting the model samples. Inherited from PiiClassifierOperation.
pointsthe locations of feature points in image coordinates. A N-by-D matrix in which each row contains D-dimensional point coordinates. All numeric matrix types will be accepted, but the operation internally uses float for calculations. Each point must have a corresponding feature vector. Otherwise, a run-time error will be generated.
locationthe location of the model (or any interesting part) with respect to the points. The location is described with 2*D parameters as a hypercube. In the most typical two-dimensional case, the hypercube is a rectangle whose parameters are stored into a 1-by-4 as (x,y,width,height). In three-dimensional case, the matrix must be 1-by-6 (x,y,z,width,height,depth) etc. Any numeric matrix type will be accepted, but float is used internally. This input is optional. If it is not connected, the operation will use the minimum bounding hypercube of the feature points as the model location. Note that it is not necessary that the feature points are inside the location. The location can frame any area of the point space with respect to the feature points. This input is used only when collecting model samples.

Outputs The number of objects emitted through the outputs depends on

matchingMode. In MatchOneModel mode, only the best matching model will be emitted. If no model matches, -1 will be emitted as the model index. In MatchAllModels and MatchDifferentModels modes, 0-N matching models will be emitted for each input.

classificationthe label associated with the matched model (double). If no model matches in MatchOneModel mode, NaN will be emitted. Inherited from PiiClassifierOperation.
model indexthe index of the model that matches the query (int). If no model matches in MatchOneModel mode, -1 will be emitted. In training, this output will always emit the current size of the database minus one; the first training sample has a model index of 0 etc.
locationthe location of the best matching model in model coordinates, represented as a hypercube. The matrix emitted from the transform output can be used to map the corners of the hypercube back to the input space. In training, the received or automatically created location will be passed here. If no model matches in MatchOneModel mode, a 1-by-2*D zero matrix will be emitted. PiiMatrix<float>.
transformthe transform that relates the matched model location to the input space. Usually, the input space is two-dimensional image coordinates, and the transform is represented as a 3-by-3 matrix A that maps homogeneous model coordinates m to image coordinates x so that x = Am. The corresponding point for a point m in the model can be found in the image by applying the transform and reading the pixel at x. If the input space is D-dimensional, the size of the transformation matrix will be D+1 by D+1. In training, and if no model matches in MatchOneModel mode, this output will emit a D+1 by D+1 identity matrix. PiiMatrix<double>.
query pointsthe subset of the input points that was matched against a model. An N-by-D PiiMatrix<float>. May be an empty matrix.
model pointsthe corresponding model points. This output together with query points specifies the pairs of points that were successfully matched. Applying the transform matrix to these points should place them pretty close to the query points.
See also

Properties

PiiMatching::ModelMatchingMode

Matching mode.

Public types

typedef PiiFeaturePointMatcher< float, PiiMatrix< float > >

Constructors and destructor

(
  • int pointDimensions
)

Constructs a new PiiPointMatchingOperation.

Public member functions

virtual int

Returns the number of samples currently in buffer.

virtual void
(
  • bool reset
)

If reset is true and the learning thread is running, this function stops it.

( )

Creates a clone of this operation.

virtual int

Returns the number of features the classifier/learning algorithm is expecting.

template<class Archive>
void
(
  • Archive & archive
  • const unsigned int
)

Protected member functions

virtual double

Reads a feature vector from the features input and emits its classification to the classification output.

virtual void
(
  • double label
  • double weight
)

Reads a feature vector from the features input and stores it into a batch of samples that will be used as the training samples when the training thread is started.

virtual bool

Trains a learning algorithm with the collected set of samples.

( )  = 0

Matches the given points to the database stored in matcher using features as the feature descriptors.

PiiMatching::ModelMatchingMode
(
  • int pointDimensions
)

Constructs a new PiiPointMatchingOperation.

virtual void

Removes duplicate matches to the same model from matchedModels.

virtual void

Replaces the current classifier with a newly trained one.

virtual void

Resets the classifier.

virtual void
(
  • int newSize
)

Resizes the batch of buffered samples.

void
(
  • PiiMatching::ModelMatchingMode matchingMode
)
virtual PiiMatrix< double >
(
  • const PiiMatrix< double > & transformParams
)  = 0

Converts the parameters of the transformation model into a square transformation matrix for homogeneous coordinates.

Friends

friend struct

Property details

  • PiiMatching::ModelMatchingMode matchingMode

    [read, write]

    Matching mode.

    The default is MatchAllModels: every query may result in 0-N matching results, which may include multiple matches to the same model. This mode is suitable if objects may overlap and many similar objects may be present. If the operation is used for database retrieval rather than object detection, MatchDifferentModels is usually used. In this mode, only one match is allowed for each model. In MatchOneModel mode each query will be matched to exactly one model.

Function details

  • PiiPointMatchingOperation

    (
    • int pointDimensions
    )
    [protected]

    Constructs a new PiiPointMatchingOperation.

    Parameters
    pointDimensions

    the number of dimensions in the feature point locations. In images, the number of dimensions is two.

  • ~PiiPointMatchingOperation

    ()
  • virtual int bufferedSampleCount

    ()
    [virtual]

    Returns the number of samples currently in buffer.

    Must be implemented by subclasses to return the number of samples currently in buffer.

    Reimplemented from PiiClassifierOperation.

  • virtual void check

    (
    • bool reset
    )
    [virtual]

    If reset is true and the learning thread is running, this function stops it.

    Otherwise just passes the call to the superclass.

    Reimplemented from PiiClassifierOperation.

  • virtual PiiPointMatchingOperation * clone

    ()
    [virtual]

    Creates a clone of this operation.

    The default implementation uses the serialization library to find a factory object for the class. Once an instance is created it iterates over all properties and copies their values to the new operation instance.

    Returns

    a deep copy of the operation, or 0 if no factory was found.

    Reimplemented from PiiOperation.

  • virtual int featureCount

    ()
    [virtual]

    Returns the number of features the classifier/learning algorithm is expecting.

    If no feature vectors have been seen so far, zero will be returned. Note that once the first sample have been received, the number of features in subsequent feature vectors must be the same.

    Reimplemented from PiiClassifierOperation.

  • template<class Archive>

    void serialize

    (
    • Archive & archive
    • const unsigned int
    )
    [inline]
  • virtual double classify

    ()
    [protected, virtual]

    Reads a feature vector from the features input and emits its classification to the classification output.

    May also send additional objects through other output sockets. This function is called by process() when an incoming sample must be classified.

    Returns

    the classification

    Reimplemented from PiiClassifierOperation.

  • virtual void collectSample

    (
    • double label
    • double weight
    )
    [protected, virtual]

    Reads a feature vector from the features input and stores it into a batch of samples that will be used as the training samples when the training thread is started.

    This function is called by process() during learning if the learning algorithm is not capable of on-line learning or if batch-based learning is requested by the user. Subclasses should respect the value of learningBatchSize. If the current number of samples in the batch is larger than learningBatchSize (and the batch size is not -1), the sample must be either be discarded or one of the older samples must be replaced, depending on fullBufferBehavior.

    Parameters
    label

    the classification of the training sample, or NaN if not applicable.

    weight

    the importance of the sample. If the weight input is not connected, this value will always be 1.0. Learning algorithms that are not capable of weighted learning will ignore this value.

    Reimplemented from PiiClassifierOperation.

  • virtual bool learnBatch

    ()
    [protected, virtual]

    Trains a learning algorithm with the collected set of samples.

    This function is called by the learning thread and must be overridden by subclasses to feed the buffered samples to the learning algorithm. Typically, subclasses call the learnBatch(PiiLearningAlgorithm<SampleSet>*, const SampleSet&, const QVector<double>&, const QVector<double>&) function template.

    This function should not modify the currently operating classifier. If the learning thread is started while the operation is running, the normal functioning should not be changed. The old classifier must be replaced with the newly trained one in replaceClassifier().

    The default implementation returns false.

    Returns

    true if the training succeeded, false otherwise.

    Reimplemented from PiiClassifierOperation.

  • virtual PiiMatching::MatchList match

    ( )
    [protected, pure virtual]

    Matches the given points to the database stored in matcher using features as the feature descriptors.

    Subclasses override this function to implement the actual matching strategy.

    Returns

    a (possibly empty) list of matching models

  • PiiMatching::ModelMatchingMode matchingMode

    ()
    [protected]
  • virtual void removeDuplicates

    ( )
    [protected, virtual]

    Removes duplicate matches to the same model from matchedModels.

    This function is called if there are more than one matched model. Subclasses may implement any strategy for pruning duplicates. The default implementation does nothing.

  • virtual void replaceClassifier

    ()
    [protected, virtual]

    Replaces the current classifier with a newly trained one.

    This function is called if learnBatch() returns true. If the classifier provides information about itself as properties (such as the code book of an NN classifier), these property values need to be changed here.

    Reimplemented from PiiClassifierOperation.

  • virtual void resetClassifier

    ()
    [protected, virtual]

    Resets the classifier.

    This function is called by reset() after ensuring mutual exclusion with the learning thread.

    Reimplemented from PiiClassifierOperation.

  • virtual void resizeBatch

    (
    • int newSize
    )
    [protected, virtual]

    Resizes the batch of buffered samples.

    This function is called by #setLearningBatchSize() after ensuring mutual exclusion with the learning thread. The function will only be called if needed. If the new size is not smaller the current number of buffered samples, nothing needs to be done. The buffer will grow automatically to the target size when new samples are read.

    Parameters
    newSize

    the new size of the batch. The batch must be truncated to the requrested size.

    Reimplemented from PiiClassifierOperation.

  • void setMatchingMode

    (
    • PiiMatching::ModelMatchingMode matchingMode
    )
    [protected]
  • virtual PiiMatrix< double > toTransformMatrix

    (
    • const PiiMatrix< double > & transformParams
    )
    [protected, pure virtual]

    Converts the parameters of the transformation model into a square transformation matrix for homogeneous coordinates.

    Subclasses override this function to convert the model parameters returned by PiiFeaturePointMatcher to a transformation matrix.

Notes (0)

Add a note

Not a single note added yet. Be the first, add yours.