Giuseppe Cuccu
Machine learning is the art of adapting generic solvers to address specific problems.
Short Bio
I am a Senior Researcher and Lecturer at the eXascale Infolab, University of Fribourg, Switzerland.
I obtained my Ph.D. in 2018 from the University of Fribourg under the supervision of Philippe Cudré-Mauroux.
Prior to that I worked for different start-ups (and founded one), and as a Research Assistant with the Dalle Molle Institute for Artificial Intelligence in Lugano, Switzerland.
I completed both my bachelor and master (summa cum laude) at the University of Milano-Bicocca, Italy.
I have been involved with Machine Learning research since 2009 – mostly on
Neural Networks,
Evolutionary Computation and
Reinforcement Learning.
Neural Networks
Parametrized generic function approximators.
Depending on their parametrization, neural networks are capable of approximating (almost) any computable function.
Their graph representation makes them easy to design and extend.
A network's size (and thus number of parameters) correlates with the maximum functional complexity that can be attained.
This model has gained the spotlight over the past years through the results achieved by Deep Learning, which trains deep (i.e. large, multi-layer) networks using gradient descent supervised learning algorithms.
×
Evolutionary Computation
Field studying a class of randomized black-box optimization algorithm originally inspired by natural evolution.
Evolutionary algorithms search for the values that optimize an objective function, without knowing (nor needing access to) its internal workings.
Progress is based on (i) sampling the parameters space, (ii) computing the value of the objective function over the samples, then (iii) approximating the objective function's gradient over the samples.
New samples are then generated to maximize the expected value of the objective function.
×
Reinforcement Learning
A learning paradigm where the only direction for training is given by a score function over candidate solutions.
While supervised learning expects a set of examples to copy from (e.g. classification), and unsupervised learning partitions the data based on similarity (e.g. clustering), reinforcement learning leaves the algorithm to figure out a way to optimize candidate solutions towards improving scores.
Somewhat confusingly, the term is also used interchangeably to indicate "classical" reinforcement learning algorithms such as Q-Learning, SARSA or TD-Learning.
×
Teaching
Research Interests
My work extends the applicability of Machine Learning towards a broad range of real-world problems, with special focus on high-performance
Continuous Control.
My main expertise is in
Neuroevolution,
combining Neural Networks and Evolutionary Computation, a technique capable of
direct policy search in reinforcement learning control problems.
My research interests include:
- High parallelization and scalability for sophisticated Evolutionary Algorithms
- Subtask specialization as an alternative to end-to-end network training
- Hybridization of Deep Learning and Neuroevolution
Continuous Control
A class of problems requiring sequential decision making over continuous time.
At any given moment, the system provides an observation based on its internal (typically hidden) status.
A controller is then tasked with deciding the next action based on the given observation and possibly its own internal state (memory).
Most real-world interactive systems belong to this category, such as robotic control, autonomous driving and videogame playing.
It is typically hard to approach these problems from a supervised learning perspective, as "correct" examples of control are hard to obtain and limiting in the range of skills they can teach.
A reinforcement learning approach can be more natural in this context, as it is more straightforward to describe a "correct behavior" in terms of e.g. touching a target with a robotic arm, drive further in a given time, or reach a high score in a videogame.
×
Neuroevolution
A technique to train Neural Networks using Evolutionary Algorithms.
EAs search in the space of network parameters to optimize the network performance on a task.
This corresponds to directly search in the space of neural networks, optimizing the scoring function.
This makes neuroevolution directly applicable to reinforcement learning problems, i.e. direct policy search.
×
Keywords
Neuroevolution, Neural Networks, Evolutionary Algorithms, Reinforcement Learning.
Selected Awards
- Best Paper Award. International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2019).
- Best Paper Award Nomination. Foundation of Digital Games (FDG 2013).
- Best Paper Award. International Conference on Evolutionary Computation (ICEC 2009).
- Undergraduate Fellowship (Master). University of Milano-Bicocca (2006–2009)
- Undergraduate Fellowship (Bachelor). University of Milano-Bicocca (2003–2006)
Publications
- Declan Oller, Tobias Glasmachers, and Giuseppe Cuccu. “Analyzing Reinforcement Learning Benchmarks with Random Weight Guessing.” In Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems. International Foundation for Autonomous Agents and Multiagent Systems, 2020. Bibtex PDF
Analyzing Reinforcement Learning Benchmarks with Random Weight Guessing
Declan Oller, Tobias Glasmachers, and Giuseppe Cuccu. “Analyzing Reinforcement Learning Benchmarks with Random Weight Guessing.” In Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems. International Foundation for Autonomous Agents and Multiagent Systems, 2020.
@inproceedings{oller2020aamas,
title = {Analyzing Reinforcement Learning Benchmarks with Random Weight Guessing},
author = {Oller, Declan and Glasmachers, Tobias and Cuccu, Giuseppe},
booktitle = {Proceedings of the 19th International Conference on Autonomous Agents and MultiAgent Systems},
year = {2020},
organization = {International Foundation for Autonomous Agents and Multiagent Systems},
url = {https://exascale.info/assets/pdf/oller2020aamas.pdf}
}
×
- Giuseppe Cuccu, Julian Togelius, and Philippe Cudré-Mauroux. “Playing Atari with Six Neurons.” In Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, 998–1006. International Foundation for Autonomous Agents and Multiagent Systems, 2019. Bibtex PDF
Playing Atari with six neurons
Giuseppe Cuccu, Julian Togelius, and Philippe Cudré-Mauroux. “Playing Atari with Six Neurons.” In Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, 998–1006. International Foundation for Autonomous Agents and Multiagent Systems, 2019.
@inproceedings{cuccu2019aamas,
title = {Playing {A}tari with six neurons},
author = {Cuccu, Giuseppe and Togelius, Julian and Cudr{\'e}-Mauroux, Philippe},
booktitle = {Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems},
pages = {998--1006},
year = {2019},
organization = {International Foundation for Autonomous Agents and Multiagent Systems},
url = {https://exascale.info/assets/pdf/cuccu2019aamas.pdf}
}
×
- Giuseppe Cuccu. “Extending the Applicability of Neuroevolution.” PhD thesis, Université de Fribourg, 2018. Bibtex PDF
Extending the Applicability of Neuroevolution
Giuseppe Cuccu. “Extending the Applicability of Neuroevolution.” PhD thesis, Université de Fribourg, 2018.
@phdthesis{cuccu2018phd,
author = {Cuccu, Giuseppe},
title = {Extending the Applicability of Neuroevolution},
school = {Universit{\'e} de Fribourg},
address = {Fribourg, Switzerland},
year = {2018},
url = {https://exascale.info/assets/pdf/cuccu2018phd.pdf}
}
×
- Giuseppe Cuccu, Somayeh Danafar, Philippe Cudré-Mauroux, Martin Gassner, Stefano Bernero, and Krzysztof Kryszczuk. “A Data-Driven Approach to Predict NOx-Emissions of Gas Turbines.” In 2017 IEEE International Conference on Big Data, BigData 2017, Boston,
MA, USA, December 11-14, 2017, 1283–88, 2017. Bibtex PDF
A data-driven approach to predict NOx-emissions of gas turbines
Giuseppe Cuccu, Somayeh Danafar, Philippe Cudré-Mauroux, Martin Gassner, Stefano Bernero, and Krzysztof Kryszczuk. “A Data-Driven Approach to Predict NOx-Emissions of Gas Turbines.” In 2017 IEEE International Conference on Big Data, BigData 2017, Boston,
MA, USA, December 11-14, 2017, 1283–88, 2017.
@inproceedings{cuccu2017bigdata,
author = {Cuccu, Giuseppe and Danafar, Somayeh and Cudr{\'e}-Mauroux, Philippe and Gassner, Martin and Bernero, Stefano and Kryszczuk, Krzysztof},
title = {A data-driven approach to predict NOx-emissions of gas turbines},
booktitle = {2017 {IEEE} International Conference on Big Data, BigData 2017, Boston,
MA, USA, December 11-14, 2017},
pages = {1283--1288},
year = {2017},
doi = {10.1109/BigData.2017.8258056},
url = {https://exascale.info/assets/pdf/cuccu2017bigdata.pdf}
}
×
- Jan Koutník, Giuseppe Cuccu, Jürgen Schmidhuber, and Faustino Gomez. “Evolving Large-Scale Neural Networks for Vision-Based Reinforcement Learning.” In Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, 1061–68. GECCO ’13. ACM, 2013. Bibtex PDF
Evolving Large-scale Neural Networks for Vision-based Reinforcement Learning
Jan Koutník, Giuseppe Cuccu, Jürgen Schmidhuber, and Faustino Gomez. “Evolving Large-Scale Neural Networks for Vision-Based Reinforcement Learning.” In Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation, 1061–68. GECCO ’13. ACM, 2013.
@inproceedings{koutnik2013gecco,
location = {New York, {NY}, {USA}},
title = {Evolving Large-scale Neural Networks for Vision-based Reinforcement Learning},
isbn = {978-1-4503-1963-8},
url = {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.451.760&rep=rep1&type=pdf},
doi = {10.1145/2463372.2463509},
series = {{GECCO} '13},
pages = {1061--1068},
booktitle = {Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation},
publisher = {{ACM}},
author = {Koutník, Jan and Cuccu, Giuseppe and Schmidhuber, Jürgen and Gomez, Faustino},
year = {2013},
keywords = {Computer vision, neuroevolution, recurrent neural networks, reinforcement learning}
}
×
- Jan Koutník, Giuseppe Cuccu, Jürgen Schmidhuber, and Faustino Gomez. “Evolving Large-Scale Neural Networks for Vision-Based TORCS.” In Foundations of Digital Games (FDG), 206–12, 2013. Bibtex PDF
Evolving Large-Scale Neural Networks for Vision-Based TORCS
Jan Koutník, Giuseppe Cuccu, Jürgen Schmidhuber, and Faustino Gomez. “Evolving Large-Scale Neural Networks for Vision-Based TORCS.” In Foundations of Digital Games (FDG), 206–12, 2013.
@inproceedings{koutnik2013fdg,
location = {Chania, Crete, {GR}},
title = {Evolving Large-Scale Neural Networks for Vision-Based {TORCS}},
isbn = {{ISBN} 978-0-9913982-0-1},
url = {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.654.9135&rep=rep1&type=pdf},
pages = {206--212},
booktitle = {Foundations of Digital Games (FDG)},
author = {Koutník, Jan and Cuccu, Giuseppe and Schmidhuber, Jürgen and Gomez, Faustino},
year = {2013}
}
×
- Giuseppe Cuccu, and Faustino Gomez. “Block Diagonal Natural Evolution Strategies.” In Parallel Problem Solving from Nature - PPSN XII, 488–97. Springer, Berlin, Heidelberg, 2012. Bibtex PDF
Block Diagonal Natural Evolution Strategies
Giuseppe Cuccu, and Faustino Gomez. “Block Diagonal Natural Evolution Strategies.” In Parallel Problem Solving from Nature - PPSN XII, 488–97. Springer, Berlin, Heidelberg, 2012.
@inproceedings{cuccu2012ppsn,
title = {Block Diagonal Natural Evolution Strategies},
url = {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.308.655&rep=rep1&type=pdf},
doi = {10.1007/978-3-642-32964-7_49},
eventtitle = {International Conference on Parallel Problem Solving from Nature},
pages = {488--497},
booktitle = {Parallel Problem Solving from Nature - {PPSN} {XII}},
publisher = {Springer, Berlin, Heidelberg},
author = {Cuccu, Giuseppe and Gomez, Faustino},
year = {2012},
langid = {english}
}
×
- Vincent Graziano, Tobias Glasmachers, Tom Schaul, Leo Pape, Giuseppe Cuccu, Jürgen Leitner, and Jürgen Schmidhuber. “Artificial Curiosity for Autonomous Space Exploration.” Acta Futura 4 (2011): 41–51. Bibtex PDF
Artificial Curiosity for Autonomous Space Exploration
Vincent Graziano, Tobias Glasmachers, Tom Schaul, Leo Pape, Giuseppe Cuccu, Jürgen Leitner, and Jürgen Schmidhuber. “Artificial Curiosity for Autonomous Space Exploration.” Acta Futura 4 (2011): 41–51.
@article{graziano2011acta,
title = {Artificial Curiosity for Autonomous Space Exploration},
volume = {4},
url = {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.225.3031&rep=rep1&type=pdf},
pages = {41--51},
journaltitle = {Acta Futura},
author = {Graziano, Vincent and Glasmachers, Tobias and Schaul, Tom and Pape, Leo and Cuccu, Giuseppe and Leitner, Jürgen and Schmidhuber, Jürgen},
year = {2011}
}
×
- Leonardo Vanneschi, and Giuseppe Cuccu. “Reconstructing Dynamic Target Functions by Means of Genetic Programming Using Variable Population Size.” In Computational Intelligence, 343:121–34. Springer, Berlin, Heidelberg, 2011. Bibtex PDF
Reconstructing Dynamic Target Functions by Means of Genetic Programming Using Variable Population Size
Leonardo Vanneschi, and Giuseppe Cuccu. “Reconstructing Dynamic Target Functions by Means of Genetic Programming Using Variable Population Size.” In Computational Intelligence, 343:121–34. Springer, Berlin, Heidelberg, 2011.
@incollection{vanneschi2011computational,
title = {Reconstructing Dynamic Target Functions by Means of Genetic Programming Using Variable Population Size},
volume = {343},
url = {https://link.springer.com/chapter/10.1007/978-3-642-20206-3_8},
pages = {121--134},
booktitle = {Computational Intelligence},
publisher = {Springer, Berlin, Heidelberg},
author = {Vanneschi, Leonardo and Cuccu, Giuseppe},
year = {2011},
langid = {english},
doi = {10.1007/978-3-642-20206-3_8}
}
×
- Giuseppe Cuccu, Matthew Luciw, Jürgen Schmidhuber, and Faustino Gomez. “Intrinsically Motivated Neuroevolution for Vision-Based Reinforcement Learning.” In Development and Learning (ICDL), 2011 IEEE International Conference On, 2:1–7. IEEE, 2011. Bibtex PDF
Intrinsically Motivated Neuroevolution for Vision-based Reinforcement Learning
Giuseppe Cuccu, Matthew Luciw, Jürgen Schmidhuber, and Faustino Gomez. “Intrinsically Motivated Neuroevolution for Vision-Based Reinforcement Learning.” In Development and Learning (ICDL), 2011 IEEE International Conference On, 2:1–7. IEEE, 2011.
@inproceedings{cuccu2011icdl,
title = {Intrinsically Motivated Neuroevolution for Vision-based Reinforcement Learning},
volume = {2},
url = {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.308.3917&rep=rep1&type=pdf},
pages = {1--7},
booktitle = {Development and Learning ({ICDL}), 2011 {IEEE} International Conference on},
publisher = {{IEEE}},
author = {Cuccu, Giuseppe and Luciw, Matthew and Schmidhuber, Jürgen and Gomez, Faustino},
year = {2011},
keywords = {artificial evolution, Computer vision, continuous reinforcement learning, embedded agents, high-dimensional visual images, intrinsically motivated neuroevolution, learning (artificial intelligence), neural networks, recurrent neural nets, recurrent neural network controllers, Tin, unsupervised sensory preprocessor, vision-based reinforcement learning, Yttrium}
}
×
- Giuseppe Cuccu, Faustino Gomez, and Tobias Glasmachers. “Novelty-Based Restarts for Evolution Strategies.” In Evolutionary Computation (CEC), 2011 IEEE Congress On, 158–63. IEEE, 2011. Bibtex PDF
Novelty-based Restarts for Evolution Strategies
Giuseppe Cuccu, Faustino Gomez, and Tobias Glasmachers. “Novelty-Based Restarts for Evolution Strategies.” In Evolutionary Computation (CEC), 2011 IEEE Congress On, 158–63. IEEE, 2011.
@inproceedings{cuccu2011cec,
title = {Novelty-based Restarts for Evolution Strategies},
url = {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.308.240&rep=rep1&type=pdf},
pages = {158--163},
booktitle = {Evolutionary Computation ({CEC}), 2011 {IEEE} Congress on},
publisher = {{IEEE}},
author = {Cuccu, Giuseppe and Gomez, Faustino and Glasmachers, Tobias},
year = {2011},
keywords = {black box optimization, black-box optimization, black box search algorithm, Convergence, Covariance matrix, Equations, evolutionary computation, evolution strategies, Monte Carlo methods, novelty-based restarts, novelty search, optimisation, Optimization, restart strategies, search problems, search space, Switches}
}
×
- Giuseppe Cuccu, and Faustino Gomez. “When Novelty Is Not Enough.” In Applications of Evolutionary Computation, 234–43. Springer, Berlin, Heidelberg, 2011. Bibtex PDF
When Novelty Is Not Enough
Giuseppe Cuccu, and Faustino Gomez. “When Novelty Is Not Enough.” In Applications of Evolutionary Computation, 234–43. Springer, Berlin, Heidelberg, 2011.
@inproceedings{cuccu2011evostar,
title = {When Novelty Is Not Enough},
url = {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.383.826&rep=rep1&type=pdf},
doi = {10.1007/978-3-642-20525-5_24},
eventtitle = {European Conference on the Applications of Evolutionary Computation},
pages = {234--243},
booktitle = {Applications of Evolutionary Computation},
publisher = {Springer, Berlin, Heidelberg},
author = {Cuccu, Giuseppe and Gomez, Faustino},
year = {2011},
langid = {english}
}
×
- Leonardo Vanneschi, and Giuseppe Cuccu. “Variable Size Population for Dynamic Optimization with Genetic Programming.” In Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, 1895–96. GECCO ’09. ACM, 2009. Bibtex PDF
Variable Size Population for Dynamic Optimization with Genetic Programming
Leonardo Vanneschi, and Giuseppe Cuccu. “Variable Size Population for Dynamic Optimization with Genetic Programming.” In Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation, 1895–96. GECCO ’09. ACM, 2009.
@inproceedings{vanneschi2009gecco,
author = {Vanneschi, Leonardo and Cuccu, Giuseppe},
location = {New York, {NY}, {USA}},
title = {Variable Size Population for Dynamic Optimization with Genetic Programming},
isbn = {978-1-60558-325-9},
url = {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.205.7910&rep=rep1&type=pdf},
doi = {10.1145/1569901.1570222},
series = {{GECCO} '09},
pages = {1895--1896},
booktitle = {Proceedings of the 11th Annual Conference on Genetic and Evolutionary Computation},
publisher = {{ACM}},
year = {2009},
keywords = {dynamic optimization, genetic algorithms, genetic programming, genetic programming, Poster, variable size populations}
}
×
- Leonardo Vanneschi, and Giuseppe Cuccu. “A Study of Genetic Programming Variable Population Size for Dynamic Optimization Problems.” In International Conference on Evolutionary Computation (ICEC), 119–26, 2009. Bibtex PDF
A Study of Genetic Programming Variable Population Size for Dynamic Optimization Problems
Leonardo Vanneschi, and Giuseppe Cuccu. “A Study of Genetic Programming Variable Population Size for Dynamic Optimization Problems.” In International Conference on Evolutionary Computation (ICEC), 119–26, 2009.
@inproceedings{vanneschi2009icec,
author = {Vanneschi, Leonardo and Cuccu, Giuseppe},
location = {Madeira, Portugal},
title = {A Study of Genetic Programming Variable Population Size for Dynamic Optimization Problems},
url = {http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.205.7483&rep=rep1&type=pdf},
pages = {119--126},
booktitle = {International Conference on Evolutionary Computation (ICEC)},
year = {2009},
keywords = {genetic algorithms, genetic programming}
}
×