it.uniroma2.sag.kelp.data.dataset

Class SimpleDataset

java.lang.Object

it.uniroma2.sag.kelp.data.dataset.SimpleDataset

All Implemented Interfaces:

Dataset

Direct Known Subclasses:

SequenceDataset

public class SimpleDataset
extends Object
implements Dataset

A SimpleDataset that represent a whole dataset in memory.

Author:

Giuseppe Castellucci, Simone Filice

Constructor Summary

Constructors

Constructor and Description
SimpleDataset() Initializes an empty dataset

Method Summary

Methods

Modifier and Type	Method and Description
void	addExample(Example example) Add an example to the dataset
void	addExamples(Dataset datasetToBeAdded) Add all the examples contained in datasetToBeAdded
static Dataset	extractExamplesOfClasses(Dataset dataset, List<Label> labels) This method extracts examples of given labels from dataset
List<Label>	getClassificationLabels() Returns all the classification labels in the dataset.
Example	getExample(int exampleIndex) Return the example stored in the exampleIndex position
List<Example>	getExamples() Returns an array containing all the stored examples
Example	getNextExample() Returns the next n Examples stored in the Dataset or a fewer number if n examples are not available.
List<Example>	getNextExamples(int n) Returns the next Example stored in the Dataset
int	getNumberOfExamples() Returns the number of Examples in the dataset
int	getNumberOfNegativeExamples(Label positiveClass) Returns the number of negative Examples of a given class
int	getNumberOfPositiveExamples(Label positiveClass) Returns the number of positive Examples of a given class
Example	getRandExample()
List<Example>	getRandExamples(int k)
List<Label>	getRegressionProperties() Returns all the regression properties in the dataset.
SimpleDataset	getShuffledDataset()
Vector	getZeroVector(String representationIdentifier) Returns a zero vector compliant with the representation identifier by representationIdentifier containing all zeros
boolean	hasNextExample() Returns a boolean declaring whether there are other Examples in the dataset
boolean	isConsistent() Evaluates whether the examples included in this dataset are compatible with each other.
void	manipulate(Manipulator... manipulators) Manipulates all the examples in the dataset accordingly to the strategies defined by the given manipulators.
SimpleDataset[]	nFolding(int n) Returns n datasets.
SimpleDataset[]	nFoldingClassDistributionInvariant(int n) Returns n datasets.
void	populate(DatasetReader reader) Populate the dataset using the provided reader
void	populate(String filename) Populate the dataset by reading it from a KeLP compliant file.
void	reset() Reset the reading pointer
void	save(String outputFilePath) Save the dataset in a file.
void	setSeed(long seed) Sets the seed of the random generator used to shuffling examples and getting random examples
void	shuffleExamples(Random randomGenerator) Shuffles the examples in the dataset
SimpleDataset[]	split(float percentage) Returns two datasets created by splitting this dataset accordingly to percentage.
SimpleDataset[]	splitClassDistributionInvariant(float percentage) Returns two datasets created by splitting this dataset accordingly to percentage.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SimpleDataset

public SimpleDataset()

Initializes an empty dataset

Method Detail

addExample

public void addExample(Example example)

Add an example to the dataset

Specified by:

addExample in interface Dataset

Parameters:

example - the example to be added

addExamples

public void addExamples(Dataset datasetToBeAdded)

Add all the examples contained in datasetToBeAdded

Parameters:

datasetToBeAdded - the dataset containing all the examples to be added

getExample

public Example getExample(int exampleIndex)

Return the example stored in the exampleIndex position

Parameters:

exampleIndex - the index of the example to return

Returns:

the example stored in the exampleIndex position

hasNextExample

public boolean hasNextExample()

Description copied from interface: Dataset

Returns a boolean declaring whether there are other Examples in the dataset

Specified by:

hasNextExample in interface Dataset

Returns:

true if and only if there is at least another Example in the dataset

getNextExample

public Example getNextExample()

Description copied from interface: Dataset

Returns the next n Examples stored in the Dataset or a fewer number if n examples are not available.

Specified by:

getNextExample in interface Dataset

Returns:

the next n Examples

getNextExamples

public List<Example> getNextExamples(int n)

Description copied from interface: Dataset

Returns the next Example stored in the Dataset

Specified by:

getNextExamples in interface Dataset

Parameters:

n - the number of examples to be returned

Returns:

the next Example

reset

public void reset()

Description copied from interface: Dataset

Reset the reading pointer

Specified by:

reset in interface Dataset

getNumberOfPositiveExamples

public int getNumberOfPositiveExamples(Label positiveClass)

Description copied from interface: Dataset

Returns the number of positive Examples of a given class

Specified by:

getNumberOfPositiveExamples in interface Dataset

Parameters:

positiveClass - the class whose number of positive Examples are required

Returns:

the number of positive Examples of positiveClass

getNumberOfNegativeExamples

public int getNumberOfNegativeExamples(Label positiveClass)

Description copied from interface: Dataset

Returns the number of negative Examples of a given class

Specified by:

getNumberOfNegativeExamples in interface Dataset

Parameters:

positiveClass - the class whose number of negative Examples are required

Returns:

the number of negative Examples of positiveClass

getNumberOfExamples

public int getNumberOfExamples()

Description copied from interface: Dataset

Returns the number of Examples in the dataset

Specified by:

getNumberOfExamples in interface Dataset

Returns:

the number of Examples in the dataset

getClassificationLabels

public List<Label> getClassificationLabels()

Description copied from interface: Dataset

Returns all the classification labels in the dataset.

Specified by:

getClassificationLabels in interface Dataset

Returns:

the classification labels in the dataset

getRegressionProperties

public List<Label> getRegressionProperties()

Description copied from interface: Dataset

Returns all the regression properties in the dataset.

Specified by:

getRegressionProperties in interface Dataset

Returns:

the regression properties in the dataset

shuffleExamples

public void shuffleExamples(Random randomGenerator)

Shuffles the examples in the dataset

Parameters:

randomGenerator - a random number generator

splitClassDistributionInvariant

public SimpleDataset[] splitClassDistributionInvariant(float percentage)

Returns two datasets created by splitting this dataset accordingly to percentage. The original distribution of the examples among the classes is maintained in the two datasets. The examples are split accordingly to their order. Thus the first dataset consists of the first percentage% of examples of each class, while the second dataset consists in all the remaining examples

Parameters:

percentage - should be a number in [0,1]

Returns:

two datasets generated by splitting this one

split

public SimpleDataset[] split(float percentage)

Returns two datasets created by splitting this dataset accordingly to percentage. The examples are split accordingly to their order without maintaining the original data distribution among the classes. Thus the first dataset consists of the first percentage% of examples, while the second dataset consists in all the remaining examples

Parameters:

percentage - should be a number in [0,1]

Returns:

two datasets generated by splitting this one

nFoldingClassDistributionInvariant

public SimpleDataset[] nFoldingClassDistributionInvariant(int n)

Returns n datasets. Each dataset is a fold storing 1/n of the total examples. The folds are not overlapped and maintain the original distribution of the examples among the classes. The example in this dataset are split into n folds accordingly to their order, so that for instance the first folds has all the first examples of each class

Parameters:

n - the number of folds to create

Returns:

n datasets each one consisting of 1/n% of the examples

nFolding

public SimpleDataset[] nFolding(int n)

Returns n datasets. Each dataset is a fold storing 1/n of the total examples. The folds are not overlapped and do not maintain the original distribution of the examples among the classes. The example in this dataset are split into n folds accordingly to their order, so that for instance the first folds has all the first examples.

Parameters:

n - the number of folds to create

Returns:

n datasets each one consisting of 1/n% of the examples

getExamples

public List<Example> getExamples()

Description copied from interface: Dataset

Returns an array containing all the stored examples

Specified by:

getExamples in interface Dataset

Returns:

the stored examples

extractExamplesOfClasses

public static Dataset extractExamplesOfClasses(Dataset dataset,
                               List<Label> labels)
                                        throws InstantiationException,
                                               IllegalAccessException

This method extracts examples of given labels from dataset

Parameters:

dataset - original dataset

labels - labels of interest

Returns:

new dataset with only examples of labels of interest

Throws:

IllegalAccessException

InstantiationException

populate

public void populate(String filename)
              throws Exception

Populate the dataset by reading it from a KeLP compliant file.

Parameters:

filename - the path of the file to be read

Throws:

Exception

populate

public void populate(DatasetReader reader)
              throws Exception

Populate the dataset using the provided reader

Parameters:

datasetReader - the reader

Throws:

Exception

getRandExample

public Example getRandExample()

Specified by:

getRandExample in interface Dataset

Returns:

a random example

getRandExamples

public List<Example> getRandExamples(int k)

Specified by:

getRandExamples in interface Dataset

Parameters:

k - the number of examples to be returned

Returns:

a list containing k random examples.

NOTE: Duplicates are allowed

getShuffledDataset

public SimpleDataset getShuffledDataset()

Specified by:

getShuffledDataset in interface Dataset

Returns:

a Dataset containing all the examples in this Dataset in a shuffled order

setSeed

public void setSeed(long seed)

Description copied from interface: Dataset

Sets the seed of the random generator used to shuffling examples and getting random examples

Specified by:

setSeed in interface Dataset

Parameters:

seed - the seed of the random generator

getZeroVector

public Vector getZeroVector(String representationIdentifier)

Description copied from interface: Dataset

Returns a zero vector compliant with the representation identifier by representationIdentifier containing all zeros

NOTE: it assumes that there is at least an example in the dataset and that the representation is directly available on the example using the getRepresentation method (i.e., the example is not an ExamplePair storing the representation in its left or right element)

Specified by:

getZeroVector in interface Dataset

Parameters:

representationIdentifier - the identifier of the representation

Returns:

a zero vector compliant with the representation identifier by representationIdentifier containing all zeros

manipulate

public void manipulate(Manipulator... manipulators)

Description copied from interface: Dataset

Manipulates all the examples in the dataset accordingly to the strategies defined by the given manipulators.

NOTE: If more than one manipulator is adopted, they will be applied in the same order of the manipulator in the array

Specified by:

manipulate in interface Dataset

Parameters:

manipulators - the manipulators that must be applied to all the examples in the dataset

save

public void save(String outputFilePath)
          throws FileNotFoundException,
                 IOException

Save the dataset in a file.
NOTE: if the filename ends with ".gz" the file will be compressed through GZIP.

Parameters:

outputFilePath - the file path

Throws:

FileNotFoundException

IOException

isConsistent

public boolean isConsistent()

Evaluates whether the examples included in this dataset are compatible with each other. In particular it compares the first example with all the others

Returns:

whether this dataset is consistent, i.e., its examples are compatible with each others