Bringing state-of-the-art machine learning into real-world solutions

Research statement

Data coming from real world processes rarely respects the carefully engineered requirements of most machine learning methods. Partial observability, noise, redundancy, missing values, unclear or under-defined objectives, real-time performance, limited resources, incremental discovery, high dimensionality, continuous spaces, sophisticated tasks, constitute but few of the challenges.

My research interest is in bridging state-of-the-art machine learning and real-world, data-driven problems. On the machine learning side, I advance the state of the art towards robustness, resiliency and performance. On the data analysis side, I devise novel techniques and practices aimed at producing optimal training data out of real world collections.

I specialize in the following approaches: reinforcement learning, neural networks, evolutionary algorithms, black-box optimization, autonomous control, predictive and prescriptive analytics.

Bibliography

1. Jan Koutník, Giuseppe Cuccu, Jürgen Schmidhuber, and Faustino Gomez. “Evolving Large-Scale Neural Networks for Vision-Based Reinforcement Learning.” In Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, 1061–68. GECCO ’13. ACM, 2013. Bibtex PDF

   Evolving Large-scale Neural Networks for Vision-based Reinforcement Learning

   Jan Koutník, Giuseppe Cuccu, Jürgen Schmidhuber, and Faustino Gomez. “Evolving Large-Scale Neural Networks for Vision-Based Reinforcement Learning.” In Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, 1061–68. GECCO ’13. ACM, 2013.

   @inproceedings{koutnik_evolving_rl_2013,
     location = {New York, {NY}, {USA}},
     title = {Evolving Large-scale Neural Networks for Vision-based Reinforcement Learning},
     isbn = {978-1-4503-1963-8},
     url = {https://pdfs.semanticscholar.org/fe4d/de1ea2fb09e65b990ce0c661ef151301dae4.pdf},
     doi = {10.1145/2463372.2463509},
     series = {{GECCO} '13},
     pages = {1061--1068},
     booktitle = {Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation},
     publisher = {{ACM}},
     author = {Koutník, Jan and Cuccu, Giuseppe and Schmidhuber, Jürgen and Gomez, Faustino},
     year = {2013},
     keywords = {Computer vision, neuroevolution, recurrent neural networks, reinforcement learning}
   }
   

   The idea of using evolutionary computation to train artificial neural networks, or neuroevolution (NE), for reinforcement learning (RL) tasks has now been around for over 20 years. However, as RL tasks become more challenging, the networks required become larger, as do their genomes. But, scaling NE to large nets (i.e. tens of thousands of weights) is infeasible using direct encodings that map genes one-to-one to network components. In this paper, we scale-up our compressed network encoding where network weight matrices are represented indirectly as a set of Fourier-type coefficients, to tasks that require very-large networks due to the high-dimensionality of their input space. The approach is demonstrated successfully on two reinforcement learning tasks in which the control networks receive visual input: (1) a vision-based version of the octopus control task requiring networks with over 3 thousand weights, and (2) a version of the TORCS driving game where networks with over 1 million weights are evolved to drive a car around a track using video images from the driver’s perspective.

   ×
2. Jan Koutník, Giuseppe Cuccu, Jürgen Schmidhuber, and Faustino Gomez. “Evolving Large-Scale Neural Networks for Vision-Based TORCS.” In Foundations of Digital Games (FDG), 206–12, 2013. Bibtex PDF

   Evolving Large-Scale Neural Networks for Vision-Based TORCS

   Jan Koutník, Giuseppe Cuccu, Jürgen Schmidhuber, and Faustino Gomez. “Evolving Large-Scale Neural Networks for Vision-Based TORCS.” In Foundations of Digital Games (FDG), 206–12, 2013.

   @inproceedings{koutnik_evolving_torcs_2013,
     location = {Chania, Crete, {GR}},
     title = {Evolving Large-Scale Neural Networks for Vision-Based {TORCS}},
     isbn = {{ISBN} 978-0-9913982-0-1},
     url = {https://pdfs.semanticscholar.org/0def/2a43a183a9579f67c7170867ee10917e6115.pdf},
     pages = {206--212},
     booktitle = {Foundations of Digital Games (FDG)},
     author = {Koutník, Jan and Cuccu, Giuseppe and Schmidhuber, Jürgen and Gomez, Faustino},
     year = {2013}
   }
   

   ×
3. Giuseppe Cuccu, and Faustino Gomez. “Block Diagonal Natural Evolution Strategies.” In Parallel Problem Solving from Nature - PPSN XII, 488–97. Springer, Berlin, Heidelberg, 2012. Bibtex PDF

   Block Diagonal Natural Evolution Strategies

   Giuseppe Cuccu, and Faustino Gomez. “Block Diagonal Natural Evolution Strategies.” In Parallel Problem Solving from Nature - PPSN XII, 488–97. Springer, Berlin, Heidelberg, 2012.

   @inproceedings{cuccu_block_2012,
     title = {Block Diagonal Natural Evolution Strategies},
     url = {https://pdfs.semanticscholar.org/7188/787665b3f79216716061510cb8185d711524.pdf},
     doi = {10.1007/978-3-642-32964-7_49},
     eventtitle = {International Conference on Parallel Problem Solving from Nature},
     pages = {488--497},
     booktitle = {Parallel Problem Solving from Nature - {PPSN} {XII}},
     publisher = {Springer, Berlin, Heidelberg},
     author = {Cuccu, Giuseppe and Gomez, Faustino},
     year = {2012},
     langid = {english}
   }
   

   The Natural Evolution Strategies (NES) family of search algorithms have been shown to be efficient black-box optimizers, but the most powerful version xNES does not scale to problems with more than a few hundred dimensions. And the scalable variant, SNES, potentially ignores important correlations between parameters. This paper introduces Block Diagonal NES (BD-NES), a variant of NES which uses a block diagonal covariance matrix. The resulting update equations are computationally effective on problems with much higher dimensionality than their full-covariance counterparts, while retaining faster convergence speed than methods that ignore covariance information altogether. The algorithm has been tested on the Octopus-arm benchmark, and the experiments section presents performance statistics showing that BD-NES achieves better performance than SNES on networks that are too large to be optimized by xNES.

   ×
4. Giuseppe Cuccu, Faustino Gomez, and Tobias Glasmachers. “Novelty-Based Restarts for Evolution Strategies.” In Evolutionary Computation (CEC), 2011 IEEE Congress On, 158–63. IEEE, 2011. Bibtex PDF

   Novelty-based Restarts for Evolution Strategies

   Giuseppe Cuccu, Faustino Gomez, and Tobias Glasmachers. “Novelty-Based Restarts for Evolution Strategies.” In Evolutionary Computation (CEC), 2011 IEEE Congress On, 158–63. IEEE, 2011.

   @inproceedings{cuccu_novelty-based_2011,
     title = {Novelty-based Restarts for Evolution Strategies},
     url = {https://pdfs.semanticscholar.org/94a7/20b35e96eaa41e0b38656f5db94f583ba8e5.pdf},
     pages = {158--163},
     booktitle = {Evolutionary Computation ({CEC}), 2011 {IEEE} Congress on},
     publisher = {{IEEE}},
     author = {Cuccu, Giuseppe and Gomez, Faustino and Glasmachers, Tobias},
     year = {2011},
     keywords = {black box optimization, black-box optimization, black box search algorithm, Convergence, Covariance matrix, Equations, evolutionary computation, evolution strategies, Monte Carlo methods, novelty-based restarts, novelty search, optimisation, Optimization, restart strategies, search problems, search space, Switches}
   }
   

   ×
5. Giuseppe Cuccu, and Faustino Gomez. “When Novelty Is Not Enough.” In Applications of Evolutionary Computation, 234–43. Springer, Berlin, Heidelberg, 2011. Bibtex PDF

   When Novelty Is Not Enough

   Giuseppe Cuccu, and Faustino Gomez. “When Novelty Is Not Enough.” In Applications of Evolutionary Computation, 234–43. Springer, Berlin, Heidelberg, 2011.

   @inproceedings{cuccu_when_2011,
     title = {When Novelty Is Not Enough},
     url = {https://pdfs.semanticscholar.org/5030/2f1693273fa64aab40fd9732381767b3337e.pdf},
     doi = {10.1007/978-3-642-20525-5_24},
     eventtitle = {European Conference on the Applications of Evolutionary Computation},
     pages = {234--243},
     booktitle = {Applications of Evolutionary Computation},
     publisher = {Springer, Berlin, Heidelberg},
     author = {Cuccu, Giuseppe and Gomez, Faustino},
     year = {2011},
     langid = {english}
   }
   

   The idea of evolving novel rather than fit solutions has recently been offered as a way to automatically discover the kind of complex solutions that exhibit truly intelligent behavior. So far, novelty search has only been studied in the context of problems where the number of possible “different” solutions has been limited. In this paper, we show, using a task with a much larger solution space, that selecting for novelty alone does not offer an advantage over fitness-based selection. In addition, we examine how the idea of novelty search can be used to sustain diversity and improve the performance of standard, fitness-based search.

   ×
6. Giuseppe Cuccu, Matthew Luciw, Jürgen Schmidhuber, and Faustino Gomez. “Intrinsically Motivated Neuroevolution for Vision-Based Reinforcement Learning.” In Development and Learning (ICDL), 2011 IEEE International Conference On, 2:1–7. IEEE, 2011. Bibtex PDF

   Intrinsically Motivated Neuroevolution for Vision-based Reinforcement Learning

   Giuseppe Cuccu, Matthew Luciw, Jürgen Schmidhuber, and Faustino Gomez. “Intrinsically Motivated Neuroevolution for Vision-Based Reinforcement Learning.” In Development and Learning (ICDL), 2011 IEEE International Conference On, 2:1–7. IEEE, 2011.

   @inproceedings{cuccu_intrinsically_2011,
     title = {Intrinsically Motivated Neuroevolution for Vision-based Reinforcement Learning},
     volume = {2},
     url = {https://pdfs.semanticscholar.org/92ab/fe027e903c94683c1c751a6e854744303b84.pdf},
     pages = {1--7},
     booktitle = {Development and Learning ({ICDL}), 2011 {IEEE} International Conference on},
     publisher = {{IEEE}},
     author = {Cuccu, Giuseppe and Luciw, Matthew and Schmidhuber, Jürgen and Gomez, Faustino},
     year = {2011},
     keywords = {artificial evolution, Computer vision, continuous reinforcement learning, embedded agents, high-dimensional visual images, intrinsically motivated neuroevolution, learning (artificial intelligence), neural networks, recurrent neural nets, recurrent neural network controllers, Tin, unsupervised sensory preprocessor, vision-based reinforcement learning, Yttrium}
   }
   

   ×
7. Vincent Graziano, Tobias Glasmachers, Tom Schaul, Leo Pape, Giuseppe Cuccu, Jürgen Leitner, and Jürgen Schmidhuber. “Artificial Curiosity for Autonomous Space Exploration.” Acta Futura 4 (2011): 41–51. Bibtex PDF

   Artificial Curiosity for Autonomous Space Exploration

   Vincent Graziano, Tobias Glasmachers, Tom Schaul, Leo Pape, Giuseppe Cuccu, Jürgen Leitner, and Jürgen Schmidhuber. “Artificial Curiosity for Autonomous Space Exploration.” Acta Futura 4 (2011): 41–51.

   @article{graziano_artificial_2011,
     title = {Artificial Curiosity for Autonomous Space Exploration},
     volume = {4},
     url = {https://pdfs.semanticscholar.org/2809/ea3e24f3b07b3b613929d8517a381956e7de.pdf},
     pages = {41--51},
     journaltitle = {Acta Futura},
     author = {Graziano, Vincent and Glasmachers, Tobias and Schaul, Tom and Pape, Leo and Cuccu, Giuseppe and Leitner, Jürgen and Schmidhuber, Jürgen},
     year = {2011}
   }
   

   ×
8. Leonardo Vanneschi, and Giuseppe Cuccu. “Reconstructing Dynamic Target Functions by Means of Genetic Programming Using Variable Population Size.” In Computational Intelligence, 343:121–34. Springer, Berlin, Heidelberg, 2011. Bibtex PDF

   Reconstructing Dynamic Target Functions by Means of Genetic Programming Using Variable Population Size

   Leonardo Vanneschi, and Giuseppe Cuccu. “Reconstructing Dynamic Target Functions by Means of Genetic Programming Using Variable Population Size.” In Computational Intelligence, 343:121–34. Springer, Berlin, Heidelberg, 2011.

   @incollection{vanneschi_reconstructing_2011,
     title = {Reconstructing Dynamic Target Functions by Means of Genetic Programming Using Variable Population Size},
     volume = {343},
     url = {https://link.springer.com/chapter/10.1007/978-3-642-20206-3_8},
     pages = {121--134},
     booktitle = {Computational Intelligence},
     publisher = {Springer, Berlin, Heidelberg},
     author = {Vanneschi, Leonardo and Cuccu, Giuseppe},
     year = {2011},
     langid = {english},
     doi = {10.1007/978-3-642-20206-3_8}
   }
   

   Dynamic environments are becoming more and more popular in many applicative domains. A large amount of literature has appeared to date dealing with the problem of tracking the extrema of dynamically changing target functions, but relatively few material has been produced on the problem of reconstructing the shape, or more generally finding the equation, of dynamically changing target functions. Nevertheless, in many applicative domains, reaching this goal would have an extremely important impact. It is the case, for instance, of complex systems modelling, like for instance biological systems or systems of biochemical reactions, where one is generally interested in understanding what’s going on in the system over time, rather than following the extrema of some target functions. Last but not least, we also believe that being able to reach this goal would help researchers to have a useful insight on the reasons that cause the change in the system over time, or at least the pattern of this modification. This paper is intended as a first preliminary step in the attempt to fill this gap. We show that genetic programming with variable population size is able to adapt to the environment modifications much faster (i.e. using a noteworthy smaller amount of computational effort) than standard genetic programming using fixed population size. The suitability of this model is tested on a set of benchmarks based on some well known symbolic regression problems.

   ×
9. Leonardo Vanneschi, and Giuseppe Cuccu. “Variable Size Population for Dynamic Optimization with Genetic Programming.” In Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, 1895–96. GECCO ’09. ACM, 2009. Bibtex PDF

   Variable Size Population for Dynamic Optimization with Genetic Programming

   Leonardo Vanneschi, and Giuseppe Cuccu. “Variable Size Population for Dynamic Optimization with Genetic Programming.” In Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, 1895–96. GECCO ’09. ACM, 2009.

   @inproceedings{vanneschi_variable_2009,
     author = {Vanneschi, Leonardo and Cuccu, Giuseppe},
     location = {New York, {NY}, {USA}},
     title = {Variable Size Population for Dynamic Optimization with Genetic Programming},
     isbn = {978-1-60558-325-9},
     url = {https://pdfs.semanticscholar.org/ed3a/0e0ddad1bb698f2e8a04fa74d10177662703.pdf},
     doi = {10.1145/1569901.1570222},
     series = {{GECCO} '09},
     pages = {1895--1896},
     booktitle = {Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation},
     publisher = {{ACM}},
     year = {2009},
     keywords = {dynamic optimization, genetic algorithms, genetic programming, genetic programming, Poster, variable size populations}
   }
   

   A new model of Genetic Programming with variable size population is presented in this paper and applied to the reconstruction of target functions in dynamic environments (i.e. problems where target functions change with time). The suitability of this model is tested on a set of benchmarks based on some well known symbolic regression problems. Experimental results confirm that our variable size population model finds solutions of the same quality as the ones found by standard Genetic Programming, but with a smaller amount of computat ional effort.

   ×
10. Leonardo Vanneschi, and Giuseppe Cuccu. “A Study of Genetic Programming Variable Population Size for Dynamic Optimization Problems.” In International Conference on Evolutionary Computation (ICEC), 119–26, 2009. Bibtex PDF

    A Study of Genetic Programming Variable Population Size for Dynamic Optimization Problems

    Leonardo Vanneschi, and Giuseppe Cuccu. “A Study of Genetic Programming Variable Population Size for Dynamic Optimization Problems.” In International Conference on Evolutionary Computation (ICEC), 119–26, 2009.

    @inproceedings{vanneschi_study_2009,
      author = {Vanneschi, Leonardo and Cuccu, Giuseppe},
      location = {Madeira, Portugal},
      title = {A Study of Genetic Programming Variable Population Size for Dynamic Optimization Problems},
      url = {https://pdfs.semanticscholar.org/f0e0/aff77a512fbddaf66caa6916785aa79adc07.pdf},
      pages = {119--126},
      booktitle = {International Conference on Evolutionary Computation (ICEC)},
      year = {2009},
      keywords = {genetic algorithms, genetic programming}
    }
    

    A new model of Genetic Programming with variable size population is presented in this paper and applied to the reconstruction of target functions in dynamic environments (i.e. problems where target functions change with time). The suitability of this model is tested on a set of benchmarks based on some well known symbolic regression problems. Experimental results confirm that our variable size population model finds solutions of similar quality to the ones found by standard Genetic Programming, but with a smaller amount of computational effort.

    ×