{"id":217,"date":"2017-02-13T18:13:51","date_gmt":"2017-02-13T18:13:51","guid":{"rendered":"http:\/\/sag.art.uniroma2.it\/kelp_wordpress\/?page_id=217"},"modified":"2017-04-05T15:19:41","modified_gmt":"2017-04-05T15:19:41","slug":"extending-kelp","status":"publish","type":"page","link":"http:\/\/www.kelp-ml.org\/?page_id=217","title":{"rendered":"Extending KeLP"},"content":{"rendered":"<h1><span style=\"font-weight: 400;\">Adding a New Representation<\/span><\/h1>\n<p>A new representation can be added by creating a class that implements <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/data\/representation\/Representation.html\">Representation<\/a>.<br \/>\nThe programmer will be asked for implementing two specific methods, <em>getTextFromData<\/em>\u00a0and <em>setDataFromText<\/em>, which are necessary for serializing and deserializing the <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/data\/representation\/Representation.html\">Representation<\/a>\u00a0in a string format and deserializing it. The programmer can decide the textual format describing the new representation. This means that a specific JSON format is not imposed. For instance, we use the text format of libSVM for sparse vectors and the Penn Treebank notation for the tree representation, as shown in the page, <a href=\"http:\/\/www.kelp-ml.org\/?page_id=39\">Input Data Format<\/a>.<br \/>\nAn empty constructor must be included. Optionally, the class can be annotated with @JsonTypeName in order to specify an alternative type name to be used during the serialization\/deserialization mechanism, otherwise the class name will be automatically used.<br \/>\nIf a normalization function need to be defined on the representation (e.g., the representation is a vector, a matrix or a higher order tensor), then the <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/data\/representation\/Normalizable.html\" target=\"_blank\">Normalizable<\/a> interface can be implemented, enabling some useful preprocessing operations on datasets such as feature scaling or data normalization. In this case the following methods must be implemented:<\/p>\n<ul>\n<li><em>getSquaredNorm<\/em>: returns the squared norm of this vector<\/li>\n<li><em>normalize<\/em>: scales the representation in order to have a unit norm in the explicit feature space<\/li>\n<li><em>scale<\/em>: multiplies each element of the representation by a given coefficient<\/li>\n<\/ul>\n<hr \/>\n<h1><span style=\"font-weight: 400;\">Adding a New Kernel<\/span><\/h1>\n<p>As discussed in <a title=\"Kernels\" href=\"http:\/\/www.kelp-ml.org\/?page_id=185\">Kernels<\/a>, kernels are organized into four main abstractions, i.e.,\u00a0<a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/kernel\/DirectKernel.html\">DirectKernel<\/a>, <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/kernel\/KernelComposition.html\">KernelComposition<\/a>, <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/kernel\/KernelCombination.html\">KernelCombination<\/a>, and <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/kernel\/pairs\/KernelOnPairs.html\">KernelOnPairs<\/a>. Therefore, when implementing a new kernel, the first step is understanding which abstract class we must extends. In this guide, we will describe how to implement a new <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/kernel\/DirectKernel.html\">DirectKernel<\/a> and a new <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/kernel\/KernelComposition.html\">KernelComposition<\/a>; the extensions to the other kernel types is straightforward.<\/p>\n<p>In describing how to implement a new <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/kernel\/Kernel.html\">Kernel<\/a>\u00a0we will use the a linear kernel and a radial basis function (RBF) kernel as practical examples.<\/p>\n<h3><strong>IMPLEMENTING A DIRECT KERNEL: THE LINEAR KERNEL EXAMPLE<\/strong><\/h3>\n<p>The linear kernel is simply the dot product between two vectors <img decoding=\"async\" loading=\"lazy\" src=\"http:\/\/www.kelp-ml.org\/wp-content\/ql-cache\/quicklatex.com-c8700e0258243116de0d4f288e2e3b44_l3.png\" class=\"ql-img-inline-formula quicklatex-auto-format\" alt=\"&#120;&#95;&#105;\" title=\"Rendered by QuickLaTeX.com\" height=\"11\" width=\"15\" style=\"vertical-align: -3px;\"\/> and <img decoding=\"async\" loading=\"lazy\" src=\"http:\/\/www.kelp-ml.org\/wp-content\/ql-cache\/quicklatex.com-43ac7b8c02ebbeedbdb09ce539a0abb9_l3.png\" class=\"ql-img-inline-formula quicklatex-auto-format\" alt=\"&#120;&#95;&#106;\" title=\"Rendered by QuickLaTeX.com\" height=\"14\" width=\"16\" style=\"vertical-align: -6px;\"\/>, i.e., <img decoding=\"async\" loading=\"lazy\" src=\"http:\/\/www.kelp-ml.org\/wp-content\/ql-cache\/quicklatex.com-1059e8f0444ff0881f6b3de26332327a_l3.png\" class=\"ql-img-inline-formula quicklatex-auto-format\" alt=\"&#107;&#40;&#120;&#95;&#105;&#44;&#120;&#95;&#106;&#41;&#32;&#61;&#32;&#120;&#95;&#105;&middot;&#120;&#95;&#106;\" title=\"Rendered by QuickLaTeX.com\" height=\"20\" width=\"120\" style=\"vertical-align: -6px;\"\/>. Then, in implementing <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/kernel\/vector\/LinearKernel.html\">LinearKernel<\/a>, we need to extend a <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/kernel\/DirectKernel.html\">DirectKernel<\/a>\u00a0on\u00a0<a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/data\/representation\/Vector.html\">Vector<\/a>. Optionally the class can be annotated with <em>@JsonTypeName<\/em>\u00a0in order to specify an alternative type name to be used during the serialization\/deserialization mechanism.<\/p>\n<pre> @JsonTypeName(\"linear\")\r\npublic class LinearKernel extends DirectKernel {<\/pre>\n<p>To make the JSON serialization\/deserialization mechanism work, an empty constructor must be defined for the LinearKernel:<\/p>\n<pre>\tpublic LinearKernel() {\r\n\r\n\t}<\/pre>\n<p>Finally the <em>kernelComputation<\/em> method must be implemented:<\/p>\n<pre>\t@Override\r\n\tprotected float kernelComputation(Vector repA, Vector repB) {\r\n\t\treturn repA.innerProduct(repB);\r\n\t}<\/pre>\n<h3><strong>IMPLEMENTING A KERNEL COMPOSITION: THE RBF KERNEL EXAMPLE<\/strong><\/h3>\n<p>The RBF kernel function is <img decoding=\"async\" loading=\"lazy\" src=\"http:\/\/www.kelp-ml.org\/wp-content\/ql-cache\/quicklatex.com-cb9d258b0a994c634f3ef60c19e66bd1_l3.png\" class=\"ql-img-inline-formula quicklatex-auto-format\" alt=\"&#107;&#40;&#120;&#95;&#105;&#44;&#120;&#95;&#106;&#41;&#61;&#101;&#94;&#123;&#45;&#92;&#103;&#97;&#109;&#109;&#97;&#32;&#92;&#108;&#101;&#102;&#116;&#92;&#108;&#86;&#101;&#114;&#116;&#32;&#120;&#95;&#105;&#45;&#120;&#95;&#106;&#32;&#92;&#114;&#105;&#103;&#104;&#116;&#92;&#114;&#86;&#101;&#114;&#116;&#95;&#92;&#109;&#97;&#116;&#104;&#99;&#97;&#108;&#123;&#72;&#125;&#32;&#94;&#50;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"24\" width=\"176\" style=\"vertical-align: -6px;\"\/> where <img decoding=\"async\" loading=\"lazy\" src=\"http:\/\/www.kelp-ml.org\/wp-content\/ql-cache\/quicklatex.com-00462e1ef6a5feabba4ec771176fd4fb_l3.png\" class=\"ql-img-inline-formula quicklatex-auto-format\" alt=\"&#92;&#108;&#101;&#102;&#116;&#92;&#108;&#86;&#101;&#114;&#116;&#32;&#120;&#95;&#105;&#45;&#120;&#95;&#106;&#32;&#92;&#114;&#105;&#103;&#104;&#116;&#92;&#114;&#86;&#101;&#114;&#116;&#95;&#92;&#109;&#97;&#116;&#104;&#99;&#97;&#108;&#123;&#72;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"20\" width=\"82\" style=\"vertical-align: -6px;\"\/> is the distance between two examples <img decoding=\"async\" loading=\"lazy\" src=\"http:\/\/www.kelp-ml.org\/wp-content\/ql-cache\/quicklatex.com-c8700e0258243116de0d4f288e2e3b44_l3.png\" class=\"ql-img-inline-formula quicklatex-auto-format\" alt=\"&#120;&#95;&#105;\" title=\"Rendered by QuickLaTeX.com\" height=\"11\" width=\"15\" style=\"vertical-align: -3px;\"\/> and <img decoding=\"async\" loading=\"lazy\" src=\"http:\/\/www.kelp-ml.org\/wp-content\/ql-cache\/quicklatex.com-43ac7b8c02ebbeedbdb09ce539a0abb9_l3.png\" class=\"ql-img-inline-formula quicklatex-auto-format\" alt=\"&#120;&#95;&#106;\" title=\"Rendered by QuickLaTeX.com\" height=\"14\" width=\"16\" style=\"vertical-align: -6px;\"\/> in a Reproducing Kernel Hilbert Space <img decoding=\"async\" loading=\"lazy\" src=\"http:\/\/www.kelp-ml.org\/wp-content\/ql-cache\/quicklatex.com-173d4d318ade4e23f184a26004cc3766_l3.png\" class=\"ql-img-inline-formula quicklatex-auto-format\" alt=\"&#92;&#109;&#97;&#116;&#104;&#99;&#97;&#108;&#123;&#72;&#125;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"15\" style=\"vertical-align: 0px;\"\/>. Considering a generic RKHS, the\u00a0simple euclidean distance in <img decoding=\"async\" loading=\"lazy\" src=\"http:\/\/www.kelp-ml.org\/wp-content\/ql-cache\/quicklatex.com-f9868b4451c5811a288f7fdd10be5558_l3.png\" class=\"ql-img-inline-formula quicklatex-auto-format\" alt=\"&#92;&#109;&#97;&#116;&#104;&#98;&#98;&#123;&#82;&#125;&#94;&#110;\" title=\"Rendered by QuickLaTeX.com\" height=\"12\" width=\"21\" style=\"vertical-align: 0px;\"\/> can be\u00a0generalized in order to define a distance measure\u00a0applicable to any underlying kernel, operating on any representation. Then <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/kernel\/standard\/RbfKernel.html\">RbfKernel<\/a> must extend <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/kernel\/KernelComposition.html\">KernelComposition<\/a>. Optionally, the class can be annotated with <em>@JsonTypeName<\/em>\u00a0in order to specify an alternative type name to be used during the serialization\/deserialization mechanism.<\/p>\n<pre>@JsonTypeName(\"rbf\")\r\npublic class RbfKernel extends KernelComposition {<\/pre>\n<p>To make the JSON serialization\/deserialization mechanism work, an empty constructor must be defined for the RbfKernel, and all the kernel parameters must be associated with the corresponding getter and setter methods. In this case <em>gamma<\/em> is the only parameter to be serialized and the corresponding <em>getGamma<\/em>\u00a0and <em>setGamma<\/em>\u00a0methods must be implemented.<\/p>\n<pre>\tprivate float gamma;\r\n\r\n\tpublic RbfKernel() {\r\n\r\n\t}\r\n\r\n\t\/**\r\n\t * @return the gamma\r\n\t *\/\r\n\tpublic float getGamma() {\r\n\t\treturn gamma;\r\n\t}\r\n\r\n\t\/**\r\n\t * @param gamma\r\n\t *            the gamma to set\r\n\t *\/\r\n\tpublic void setGamma(float gamma) {\r\n\t\tthis.gamma = gamma;\r\n\t}<\/pre>\n<p>Finally the <em>kernelComputation<\/em> method must be implemented containing the specific kernel similarity logic.<\/p>\n<pre class=\"\">\t@Override\r\n\tprotected float kernelComputation(Example exA, Example exB) {\r\n\t\tfloat innerProductOfTheDiff = this.baseKernel\r\n\t\t\t\t.squaredNormOfTheDifference(exA, exB);\r\n\t\treturn (float) Math.exp(-gamma * innerProductOfTheDiff);\r\n\t}<\/pre>\n<hr \/>\n<h1><span style=\"font-weight: 400;\">Adding a New Learning Algorithm<\/span><\/h1>\n<p>As discussed in <a title=\"Learning Algorithms\" href=\"http:\/\/www.kelp-ml.org?page_id=184\">Learning Algorithms<\/a>, KeLP provides several learning algorithms, which can be used when implementing new others. In this guide, we describe the implementation of a new linear algorithm for binary classification. A similar procedure can be easily used to implement other different types of learning algorithms.<\/p>\n<h3>IMPLEMENTING NEW LEARNING ALGORITHMS: THE PEGASOS EXAMPLE<\/h3>\n<p>We are now pretending that the <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/learningalgorithm\/classification\/pegasos\/PegasosLearningAlgorithm.html\">PegasosLearningAlgorithm<\/a> is not available in KeLP, and that we need to implement it from scratch.<br \/>\nPegasos (Shalev-Shwartz et al., 2007) is an efficient solver for linear SVM for binary classification tasks that uses a mini-batch approach; therefore, we need to implement <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/learningalgorithm\/classification\/ClassificationLearningAlgorithm.html\">ClassificationLearningAlgorithm<\/a>, <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/learningalgorithm\/BinaryLearningAlgorithm.html\">BinaryLearningAlgorithm<\/a>\u00a0and <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/learningalgorithm\/LinearMethod.html\">LinearMethod<\/a>. Optionally, the class can be annotated with @<em>JsonTypeName<\/em> in order to specify an alternative name to be used during the JSON\/XML serialization\/deserialization mechanism.<\/p>\n<pre>@JsonTypeName(\"pegasos\")\r\npublic class PegasosLearningAlgorithm implements LinearMethod, ClassificationLearningAlgorithm, BinaryLearningAlgorithm{<\/pre>\n<p>KeLP decouples the learning from the prediction phase. Regarding <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/learningalgorithm\/classification\/pegasos\/PegasosLearningAlgorithm.html\">PegasosLearningAlgorithm<\/a>, the prediction function is common to other classification algorithms. Thus, it is possible to use the class <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/predictionfunction\/classifier\/BinaryLinearClassifier.html\">BinaryLinearClassifier<\/a>, which is already implemented in the platform.  However, we need to add a new corresponding parameter, i.e., classifier. Thus, we need to add an empty constructor along with all the learning parameters associated with the corresponding getter and setter methods. These are required by the JSON serialization\/deserialization mechanism. The learning parameters of Pegasos are the regularization coefficient lambda, the number of iterations T and the number of examples k exploited during each mini-batch iteration.<\/p>\n<pre>\tprivate BinaryLinearClassifier classifier;\r\n\r\n\tprivate int k = 1;\r\n\tprivate int iterations = 1000;\r\n\tprivate float lambda = 0.01f;\r\n\r\n\t\/**\r\n\t * Returns the number of examples k that Pegasos exploits in its \r\n\t * mini-batch learning approach\r\n\t * \r\n\t * @return k\r\n\t *\/\r\n\tpublic int getK() {\r\n\t\treturn k;\r\n\t}\r\n\r\n\t\/**\r\n\t * Sets the number of examples k that Pegasos exploits in its \r\n\t * mini-batch learning approach\r\n\t * \r\n\t * @param k the k to set\r\n\t *\/\r\n\tpublic void setK(int k) {\r\n\t\tthis.k = k;\r\n\t}\r\n\r\n\t\/**\r\n\t * Returns the number of iterations\r\n\t * \r\n\t * @return the number of iterations \r\n\t *\/\r\n\tpublic int getIterations() {\r\n\t\treturn iterations;\r\n\t}\r\n\r\n\t\/**\r\n\t * Sets the number of iterations\r\n\t * \r\n\t * @param T the number of iterations to set\r\n\t *\/\r\n\tpublic void setIterations(int T) {\r\n\t\tthis.iterations = T;\r\n\t}\r\n\r\n\t\/**\r\n\t * Returns the regularization coefficient\r\n\t * \r\n\t * @return the lambda\r\n\t *\/\r\n\tpublic float getLambda() {\r\n\t\treturn lambda;\r\n\t}\r\n\r\n\t\/**\r\n\t * Sets the regularization coefficient\r\n\t * \r\n\t * @param lambda the lambda to set\r\n\t *\/\r\n\tpublic void setLambda(float lambda) {\r\n\t\tthis.lambda = lambda;\r\n\t}\r\n\r\n\tpublic PegasosLearningAlgorithm(){\r\n\t\tthis.classifier = new BinaryLinearClassifier();\r\n\t\tthis.classifier.setModel(new BinaryLinearModel());\r\n\t}<\/pre>\n<p>According to the selected interfaces some specific methods have to be implemented. As any <em>LinearMethod<\/em> we need to implement <em>setRepresentation<\/em> and <em>getRepresentation<\/em> which refer to the <em>String<\/em> identifier for the specific <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/data\/representation\/Representation.html\">Representation<\/a>\u00a0object the algorithm must exploit. Obviously a corresponding parameter must created, i.e.,\u00a0<em>representation<\/em><\/p>\n<pre>\tprivate String representation;\r\n\r\n\t@Override\r\n\tpublic String getRepresentation() {\r\n\t\treturn representation;\r\n\t}\r\n\r\n\t@Override\r\n\tpublic void setRepresentation(String representation) {\r\n\t\tthis.representation = representation;\r\n\t\tBinaryLinearModel model = this.classifier.getModel();\r\n\t\tmodel.setRepresentation(representation);\r\n\t}<\/pre>\n<p>As any <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/learningalgorithm\/BinaryLearningAlgorithm.html\">BinaryLearningAlgorithm<\/a>\u00a0we need to implement <em>getLabel<\/em> and <em>setLabel<\/em> which refer to the <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/data\/label\/Label.html\">Label<\/a> that must considered as positive class. Obviously a corresponding parameter must created, i.e. <em>label<\/em>. Moreover, to be compliant with the <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/learningalgorithm\/LearningAlgorithm.html\">LearningAlgorithm<\/a> interface, the methods <em>getLabels<\/em> and <em>setLabels<\/em> must be implemented. For the special case of <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/learningalgorithm\/BinaryLearningAlgorithm.html\">BinaryLearningAlgorithm<\/a>\u00a0they must operate on a single entry <strong>List<\/strong>:<\/p>\n<pre>\tprivate Label label;\r\n\r\n\t@Override\r\n\tpublic void setLabels(List<Label> labels){\r\n\t\tif(labels.size()!=1){\r\n\t\t\tthrow new IllegalArgumentException(\"Pegasos algorithm is a binary method which can learn a single Label\");\r\n\t\t}\r\n\t\telse{\r\n\t\t\tthis.label=labels.get(0);\r\n\t\t\tthis.classifier.setLabels(labels);\r\n\t\t}\r\n\t}\r\n\r\n\t@Override\r\n\tpublic List<Label> getLabels() {\r\n\t\treturn Arrays.asList(label);\r\n\t}\r\n\r\n\t@Override\r\n\tpublic Label getLabel(){\r\n\t\treturn this.label;\r\n\t}\r\n\r\n\t@Override\r\n\tpublic void setLabel(Label label){\r\n\t\tthis.setLabels(Arrays.asList(label));\r\n\t}\r\n<\/pre>\n<p>Finally, as any <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/learningalgorithm\/classification\/ClassificationLearningAlgorithm.html\">ClassificationLearningAlgorithm<\/a> we need to implement:<\/p>\n<ul>\n<li><em>getPredictionFunction<\/em>: the proper prediction function must be returned, in this case the\u00a0<em>classifier object<\/em>. It is a good practice to specialize the returning type, i.e., the method must return a <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/predictionfunction\/classifier\/BinaryLinearClassifier.html\">BinaryLinearClassifier<\/a> instead of a generic <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/predictionfunction\/classifier\/Classifier.html\">Classifier<\/a>;<\/li>\n<li><em>duplicate<\/em>: the instance of an algorithm must be created and returned, copying all the learning parameters (the state variables must not be set to their default value). It is a good practice to specialize the returning type, i.e., the method must return a <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/learningalgorithm\/classification\/pegasos\/PegasosLearningAlgorithm.html\">PegasosLearningAlgorithm<\/a>\u00a0instead of a generic <a href=\"http:\/\/www.kelp-ml.org\/kelp-javadoc\/current-version\/it\/uniroma2\/sag\/kelp\/learningalgorithm\/LearningAlgorithm.html\">LearningAlgorithm<\/a>;<\/li>\n<li><em>reset<\/em>: it forces the algorithm to it default state, forgetting the learning process already conducted;<\/li>\n<li><em>learn<\/em>: this method must implement the learning process of Pegasos.<\/li>\n<\/ul>\n<pre class=\"\">\t\r\n\t@Override\r\n\tpublic BinaryLinearClassifier getPredictionFunction(){\r\n\t\treturn this.classifier;\r\n\t}\r\n\r\n\t@Override\r\n\tpublic PegasosLearningAlgorithm duplicate() {\r\n\t\tPegasosLearningAlgorithm copy = new PegasosLearningAlgorithm();\r\n\t\tcopy.setK(k);\r\n\t\tcopy.setLambda(lambda);\r\n\t\tcopy.setIterations(iterations);\r\n\t\tcopy.setRepresentation(representation);\r\n\t\treturn copy;\r\n\t}\r\n\r\n\t@Override\r\n\tpublic void reset() {\r\n\t\tthis.classifier.reset();\t\t\r\n\t}\r\n\r\n\t\t@Override\r\n\tpublic void learn(Dataset dataset) {\r\n\t\tif(this.getPredictionFunction().getModel().getHyperplane()==null){\t\t\t\r\n\t\t\tthis.getPredictionFunction().getModel().setHyperplane(dataset.getZeroVector(representation));\r\n\t\t}\r\n\r\n\t\tfor(int t=1;t<=iterations;t++){\r\n\r\n\t\t\tList A_t = dataset.getRandExamples(k);\r\n\t\t\tList A_tp = new ArrayList();\r\n\t\t\tList signA_tp = new ArrayList();\r\n\t\t\tfloat eta_t = ((float)1)\/(lambda*t);\r\n\t\t\tVector w_t = this.getPredictionFunction().getModel().getHyperplane();\r\n\r\n\t\t\t\/\/creating A_tp\r\n\t\t\tfor(Example example: A_t){\r\n\t\t\t\tBinaryMarginClassifierOutput prediction = this.classifier.predict(example);\r\n\t\t\t\tfloat y = -1;\r\n\t\t\t\tif(example.isExampleOf(label)){\r\n\t\t\t\t\ty=1;\r\n\t\t\t\t}\r\n\r\n\t\t\t\tif(prediction.getScore(label)*y<1){\r\n\t\t\t\t\tA_tp.add(example);\r\n\t\t\t\t\tsignA_tp.add(y);\r\n\t\t\t\t}\t\t\t\t\t\r\n\t\t\t}\r\n\r\n\t\t\t\/\/creating w_(t+1\/2)\r\n\t\t\tw_t.scale(1-eta_t*lambda);\r\n\t\t\tfloat miscassificationFactor = eta_t\/k;\r\n\t\t\tfor(int i=0; i < A_tp.size(); i++){\r\n\t\t\t\tExample example = A_tp.get(i);\r\n\t\t\t\tfloat y = signA_tp.get(i);\r\n\t\t\t\tthis.getPredictionFunction().getModel().addExample(y*miscassificationFactor, example);\r\n\t\t\t}\r\n\r\n\t\t\t\/\/creating w_(t+1)\r\n\t\t\tfloat factor = (float) (1.0\/Math.sqrt(lambda)\/Math.sqrt(w_t.getSquaredNorm()));\r\n\t\t\tif(factor < 1){\r\n\t\t\t\tw_t.scale(factor);\r\n\t\t\t}\r\n\t\t}\r\n\t\t\t\t\r\n\t}\t\t\r\n\r\n<\/pre>\n<hr \/>\n<h3>References<\/h3>\n<p>S. Shalev-Shwartz, Y. Singer, and N. Srebro.\u00a0<em>Pegasos: Primal estimated sub\u2013gradient solver for SVM<\/em>. In Proceedings of the International Conference on Machine Learning, 2007.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Adding a New Representation A new representation can be added by creating a class that implements Representation. The programmer will be asked for implementing two specific methods, getTextFromData\u00a0and setDataFromText, which are necessary for serializing and deserializing the Representation\u00a0in a string format and deserializing it. The programmer can decide the textual format describing the new representation. <a href=\"http:\/\/www.kelp-ml.org\/?page_id=217\" rel=\"nofollow\"><span class=\"sr-only\">Read more about Extending KeLP<\/span>[&hellip;]<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":27,"menu_order":16,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"http:\/\/www.kelp-ml.org\/index.php?rest_route=\/wp\/v2\/pages\/217"}],"collection":[{"href":"http:\/\/www.kelp-ml.org\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"http:\/\/www.kelp-ml.org\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"http:\/\/www.kelp-ml.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.kelp-ml.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=217"}],"version-history":[{"count":30,"href":"http:\/\/www.kelp-ml.org\/index.php?rest_route=\/wp\/v2\/pages\/217\/revisions"}],"predecessor-version":[{"id":1005,"href":"http:\/\/www.kelp-ml.org\/index.php?rest_route=\/wp\/v2\/pages\/217\/revisions\/1005"}],"up":[{"embeddable":true,"href":"http:\/\/www.kelp-ml.org\/index.php?rest_route=\/wp\/v2\/pages\/27"}],"wp:attachment":[{"href":"http:\/\/www.kelp-ml.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=217"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}