{"id":48,"date":"2016-03-29T15:48:25","date_gmt":"2016-03-29T15:48:25","guid":{"rendered":"http:\/\/kelp.unidevel.it\/?page_id=48"},"modified":"2017-03-15T10:46:00","modified_gmt":"2017-03-15T10:46:00","slug":"multiple-representations-in-kelp","status":"publish","type":"page","link":"http:\/\/www.kelp-ml.org\/?page_id=48","title":{"rendered":"Multiple Representations in KeLP"},"content":{"rendered":"

KeLP supports natively a multiple representation formalism. It is useful, for example, when the same data can be represented by different observable properties. For example, in NLP one can decide to derive features of a sentence for different syntactic levels (e.g., part-of-speech, chunk, dependency) and treat them in a learning algorithms with different kernel functions.<\/p>\n

As an example, consider the following representation:<\/p>\n

service |BV| _.:1.0 _and:1.0 _good:1.0 _is:1.0 _look:1.0 _sharp:1.0 _staff:1.0 _the:1.0 _they:1.0 _too:1.0 _very:1.0 |EV||BDV| 0.37651452,0.32109955,0.07726285,0.053550426,-0.06682896,-0.20111458,-0.14017934,... |EDV| |BS| The staff is very sharp and they look good too . |ES| |BS| 35820984#608922#3 |ES|<\/pre>\n
It is composed by<\/p>\n
    \n
  1. a label (i.e., the class to be learned, service<\/em>).<\/li>\n
  2. a Sparse vector, whose boundaries are delimited by the special tokens |BV| |EV|; in this example, a bag of word is used. Note that features can be strings!<\/li>\n
  3. a\u00a0Dense vector, whose boundaries are delimited by the special tokens |BDV| |EDV|.<\/li>\n
  4. Two String representations, delimited by |BS| |ES|; in this case they are\u00a0used for comments.<\/li>\n<\/ol>\n
    On this representation a multiple kernel learning algorithm can be applied. Let’s look at an example of code (the full class can be found on github, here<\/a>):<\/p>\n
    The first part load a dataset, print some statistics and define the basic objects for our learning procedure.<\/p>\n
    \/\/ Read a dataset into a trainingSet variable\r\nSimpleDataset trainingSet = new SimpleDataset();\r\ntrainingSet.populate(\"src\/main\/resources\/multiplerepresentation\/train.dat\");\r\n\/\/ Read a dataset into a test variable\r\nSimpleDataset testSet = new SimpleDataset();\r\ntestSet.populate(\"src\/main\/resources\/multiplerepresentation\/test.dat\");\r\n\r\n\/\/ define the positive class\r\nStringLabel positiveClass = new StringLabel(\"food\");\r\n\r\n\/\/ print some statistics\r\nSystem.out.println(\"Training set statistics\");\r\nSystem.out.print(\"Examples number \");\r\nSystem.out.println(trainingSet.getNumberOfExamples());\r\nSystem.out.print(\"Positive examples \");\r\nSystem.out.println(trainingSet .getNumberOfPositiveExamples(positiveClass));\r\nSystem.out.print(\"Negative examples \");\r\nSystem.out.println(trainingSet.getNumberOfNegativeExamples(positiveClass));\r\n\r\nSystem.out.println(\"Test set statistics\");\r\nSystem.out.print(\"Examples number \");\r\nSystem.out.println(testSet.getNumberOfExamples());\r\nSystem.out.print(\"Positive examples \");\r\nSystem.out.println(testSet.getNumberOfPositiveExamples(positiveClass));\r\nSystem.out.print(\"Negative examples \");\r\nSystem.out.println(testSet.getNumberOfNegativeExamples(positiveClass));\r\n\r\n\/\/ instantiate a passive aggressive algorithm\r\nKernelizedPassiveAggressiveClassification kPA = new KernelizedPassiveAggressiveClassification();\r\n\/\/ indicate to the learner what is the positive class\r\nkPA.setLabel(positiveClass);\r\n\/\/ set an aggressiveness parameter\r\nkPA.setC(2f);<\/pre>\n
    The kernel function is the only that has knowledge about the representation on which it will operate. To use multiple representations, each with a specific kernel function, we must specify for each kernel what representation to use. Note that to have comparable scores with different kernels, we normalize each kernel, by applying a NormalizationKernel<\/a>.<\/p>\n
    \/\/ Kernel for the first representation (0-index)\r\nKernel linear = new LinearKernel(\"0\");\r\n\/\/ Normalize the linear kernel\r\nNormalizationKernel normalizedKernel = new NormalizationKernel(linear);\r\n\/\/ Apply a Polynomial kernel on the score (normalized) computed by\r\n\/\/ the linear kernel\r\nKernel polyKernel = new PolynomialKernel(2f, normalizedKernel);\r\n\r\n\/\/ Kernel for the second representation (1-index)\r\nKernel linear1 = new LinearKernel(\"1\");\r\n\/\/ Normalize the linear kernel\r\nNormalizationKernel normalizedKernel1 = new NormalizationKernel(linear1);\r\n\/\/ Apply a Polynomial kernel on the score (normalized) computed by\r\n\/\/ the linear kernel\r\nKernel rbfKernel = new RbfKernel(1f, normalizedKernel1);\r\n\/\/ tell the algorithm that the kernel we want to use in learning is\r\n\/\/ the polynomial kernel<\/pre>\n
    A weighted linear combination of kernel contribution is simply obtained by instantiating a LinearKernelCombination<\/a>, and by using the add\u00a0<\/em>method on it. Finally we set the kernel on the passive aggressive algorithm.<\/p>\n
    LinearKernelCombination linearCombination = new LinearKernelCombination();\r\nlinearCombination.addKernel(1f, polyKernel);\r\nlinearCombination.addKernel(1f, rbfKernel);\r\n\/\/ normalize the weights such that their sum is 1\r\nlinearCombination.normalizeWeights();\r\n\r\n\/\/ set the kernel for the PA algorithm\r\nkPA.setKernel(linearCombination);<\/pre>\n
    Then, we learn a prediction function, and we apply it on the test data.<\/p>\n
    \/\/ learn and get the prediction function\r\nkPA.learn(trainingSet);\r\nClassifier f = kPA.getPredictionFunction();\r\n\r\n\/\/ classify examples and compute some statistics\r\nint correct = 0;\r\nfor (Example e : testSet.getExamples()) {\r\n\tClassificationOutput p = f.predict(testSet.getNextExample());\r\n\tif (p.getScore(positiveClass) > 0 && e.isExampleOf(positiveClass))\r\n\t\tcorrect++;\r\n\telse if (p.getScore(positiveClass) < 0 && !e.isExampleOf(positiveClass))\r\n\t\tcorrect++;\r\n}\r\n\r\nSystem.out.println(\"Accuracy: \" + ((float) correct \/ (float) testSet.getNumberOfExamples()));<\/pre>\n","protected":false},"excerpt":{"rendered":"
    KeLP supports natively a multiple representation formalism. It is useful, for example, when the same data can be represented by different observable properties. For example, in NLP one can decide to derive features of a sentence for different syntactic levels (e.g., part-of-speech, chunk, dependency) and treat them in a learning algorithms with different kernel functions. Read more about Multiple Representations in KeLP<\/span>[…]<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"parent":27,"menu_order":11,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"http:\/\/www.kelp-ml.org\/index.php?rest_route=\/wp\/v2\/pages\/48"}],"collection":[{"href":"http:\/\/www.kelp-ml.org\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"http:\/\/www.kelp-ml.org\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"http:\/\/www.kelp-ml.org\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/www.kelp-ml.org\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=48"}],"version-history":[{"count":9,"href":"http:\/\/www.kelp-ml.org\/index.php?rest_route=\/wp\/v2\/pages\/48\/revisions"}],"predecessor-version":[{"id":903,"href":"http:\/\/www.kelp-ml.org\/index.php?rest_route=\/wp\/v2\/pages\/48\/revisions\/903"}],"up":[{"embeddable":true,"href":"http:\/\/www.kelp-ml.org\/index.php?rest_route=\/wp\/v2\/pages\/27"}],"wp:attachment":[{"href":"http:\/\/www.kelp-ml.org\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=48"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}