Hyper-parameter selection in deep neural networks using parallel particle swarm optimization

Pablo Ribalta Lorenzo; Jakub Nalepa; Luciano Sanchez Ramos; Jośe Ranilla Pastor

doi:10.1145/3067695.3084211

Hyper-parameter selection in deep neural networks using parallel particle swarm optimization

Pablo Ribalta Lorenzo
, Jakub Nalepa
, Luciano Sanchez Ramos
, Jośe Ranilla Pastor

Department of Algorithmics and Software (RAU/RAU5)

Future Processing
University of Oviedo

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

79 Citations (Scopus)

Abstract

The need of manual hyper-parameter selection can seriously hamper the model optimization of Deep Neural Networks (DNNs). Conventional automated approaches tackling this problem su.er from poor scalability or fail in certain scenarios. In this paper, we introduce a parallel method that applies Particle Swarm Optimization (PSO) for the hyper-parameter selection in DNNs. To estimate the best hyper-parameters, a population of particles is evolved, with their .tness calculated in parallel. .e experimental results demonstrate very desirable scalability properties for different DNNs. We show that the parallel PSO can further optimize existent models designed by experts in an affordable amount of time.

Original language	English
Title of host publication	GECCO 2017 - Proceedings of the Genetic and Evolutionary Computation Conference Companion
Publisher	Association for Computing Machinery, Inc
Pages	1864-1871
Number of pages	8
ISBN (Electronic)	9781450349390
DOIs	https://doi.org/10.1145/3067695.3084211
Publication status	Published - 15 Jul 2017
Event	2017 Genetic and Evolutionary Computation Conference Companion, GECCO 2017 - Berlin, Germany Duration: 15 Jul 2017 → 19 Jul 2017

Publication series

Name	GECCO 2017 - Proceedings of the Genetic and Evolutionary Computation Conference Companion

Conference

Conference	2017 Genetic and Evolutionary Computation Conference Companion, GECCO 2017
Country/Territory	Germany
City	Berlin
Period	15/07/17 → 19/07/17

Keywords

Deep Neural Networks
Hyper-Parameter Selection
Parallel Evolutionary Algorithm
Particle Swarm Optimization

ASJC Scopus subject areas

Software
Computational Theory and Mathematics
Computer Science Applications

Access to Document

10.1145/3067695.3084211

Cite this

Lorenzo, P. R., Nalepa, J., Ramos, L. S., & Pastor, J. R. (2017). Hyper-parameter selection in deep neural networks using parallel particle swarm optimization. In GECCO 2017 - Proceedings of the Genetic and Evolutionary Computation Conference Companion (pp. 1864-1871). (GECCO 2017 - Proceedings of the Genetic and Evolutionary Computation Conference Companion). Association for Computing Machinery, Inc. https://doi.org/10.1145/3067695.3084211

Lorenzo, Pablo Ribalta ; Nalepa, Jakub ; Ramos, Luciano Sanchez et al. / Hyper-parameter selection in deep neural networks using parallel particle swarm optimization. GECCO 2017 - Proceedings of the Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery, Inc, 2017. pp. 1864-1871 (GECCO 2017 - Proceedings of the Genetic and Evolutionary Computation Conference Companion).

@inproceedings{4bfdd6382e4b4006ad17244097cb9c68,

title = "Hyper-parameter selection in deep neural networks using parallel particle swarm optimization",

abstract = "The need of manual hyper-parameter selection can seriously hamper the model optimization of Deep Neural Networks (DNNs). Conventional automated approaches tackling this problem su.er from poor scalability or fail in certain scenarios. In this paper, we introduce a parallel method that applies Particle Swarm Optimization (PSO) for the hyper-parameter selection in DNNs. To estimate the best hyper-parameters, a population of particles is evolved, with their .tness calculated in parallel. .e experimental results demonstrate very desirable scalability properties for different DNNs. We show that the parallel PSO can further optimize existent models designed by experts in an affordable amount of time.",

keywords = "Deep Neural Networks, Hyper-Parameter Selection, Parallel Evolutionary Algorithm, Particle Swarm Optimization",

author = "Lorenzo, \{Pablo Ribalta\} and Jakub Nalepa and Ramos, \{Luciano Sanchez\} and Pastor, \{Jo{\'s}e Ranilla\}",

note = "Publisher Copyright: {\textcopyright} 2017 ACM.; 2017 Genetic and Evolutionary Computation Conference Companion, GECCO 2017 ; Conference date: 15-07-2017 Through 19-07-2017",

year = "2017",

month = jul,

day = "15",

doi = "10.1145/3067695.3084211",

language = "English",

series = "GECCO 2017 - Proceedings of the Genetic and Evolutionary Computation Conference Companion",

publisher = "Association for Computing Machinery, Inc",

pages = "1864--1871",

booktitle = "GECCO 2017 - Proceedings of the Genetic and Evolutionary Computation Conference Companion",

}

Lorenzo, PR, Nalepa, J, Ramos, LS & Pastor, JR 2017, Hyper-parameter selection in deep neural networks using parallel particle swarm optimization. in GECCO 2017 - Proceedings of the Genetic and Evolutionary Computation Conference Companion. GECCO 2017 - Proceedings of the Genetic and Evolutionary Computation Conference Companion, Association for Computing Machinery, Inc, pp. 1864-1871, 2017 Genetic and Evolutionary Computation Conference Companion, GECCO 2017, Berlin, Germany, 15/07/17. https://doi.org/10.1145/3067695.3084211

Hyper-parameter selection in deep neural networks using parallel particle swarm optimization. / Lorenzo, Pablo Ribalta; Nalepa, Jakub; Ramos, Luciano Sanchez et al.
GECCO 2017 - Proceedings of the Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery, Inc, 2017. p. 1864-1871 (GECCO 2017 - Proceedings of the Genetic and Evolutionary Computation Conference Companion).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Hyper-parameter selection in deep neural networks using parallel particle swarm optimization

AU - Lorenzo, Pablo Ribalta

AU - Nalepa, Jakub

AU - Ramos, Luciano Sanchez

AU - Pastor, Jośe Ranilla

PY - 2017/7/15

Y1 - 2017/7/15

N2 - The need of manual hyper-parameter selection can seriously hamper the model optimization of Deep Neural Networks (DNNs). Conventional automated approaches tackling this problem su.er from poor scalability or fail in certain scenarios. In this paper, we introduce a parallel method that applies Particle Swarm Optimization (PSO) for the hyper-parameter selection in DNNs. To estimate the best hyper-parameters, a population of particles is evolved, with their .tness calculated in parallel. .e experimental results demonstrate very desirable scalability properties for different DNNs. We show that the parallel PSO can further optimize existent models designed by experts in an affordable amount of time.

AB - The need of manual hyper-parameter selection can seriously hamper the model optimization of Deep Neural Networks (DNNs). Conventional automated approaches tackling this problem su.er from poor scalability or fail in certain scenarios. In this paper, we introduce a parallel method that applies Particle Swarm Optimization (PSO) for the hyper-parameter selection in DNNs. To estimate the best hyper-parameters, a population of particles is evolved, with their .tness calculated in parallel. .e experimental results demonstrate very desirable scalability properties for different DNNs. We show that the parallel PSO can further optimize existent models designed by experts in an affordable amount of time.

KW - Deep Neural Networks

KW - Hyper-Parameter Selection

KW - Parallel Evolutionary Algorithm

KW - Particle Swarm Optimization

UR - https://www.scopus.com/pages/publications/85026858152

U2 - 10.1145/3067695.3084211

DO - 10.1145/3067695.3084211

M3 - Conference contribution

AN - SCOPUS:85026858152

T3 - GECCO 2017 - Proceedings of the Genetic and Evolutionary Computation Conference Companion

SP - 1864

EP - 1871

BT - GECCO 2017 - Proceedings of the Genetic and Evolutionary Computation Conference Companion

PB - Association for Computing Machinery, Inc

T2 - 2017 Genetic and Evolutionary Computation Conference Companion, GECCO 2017

Y2 - 15 July 2017 through 19 July 2017

ER -

Lorenzo PR, Nalepa J, Ramos LS, Pastor JR. Hyper-parameter selection in deep neural networks using parallel particle swarm optimization. In GECCO 2017 - Proceedings of the Genetic and Evolutionary Computation Conference Companion. Association for Computing Machinery, Inc. 2017. p. 1864-1871. (GECCO 2017 - Proceedings of the Genetic and Evolutionary Computation Conference Companion). doi: 10.1145/3067695.3084211

Hyper-parameter selection in deep neural networks using parallel particle swarm optimization

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this