Jürgen Schmidhuber

Jürgen Schmidhuber, born in 1963 in Munich, serves as a central figure in the evolution of artificial intelligence and neural networks. His research career, spanning decades of academic and institutional appointments, focuses on deep learning architectures that underpin many contemporary computational tasks. His contributions bridge early theoretical developments in recursive structures and self-supervised learning with widespread modern applications.

Academic Foundation and Research Leadership

Schmidhuber completed his undergraduate and doctorate studies at the Technical University of Munich by 1991, with advisors Wilfried Brauer and Klaus Schulten. His career includes teaching roles at the Technical University of Munich and the Università della Svizzera italiana. Since 1995, he has led the Dalle Molle Institute for Artificial Intelligence Research in Switzerland. In 2021, he joined the King Abdullah University of Science and Technology in Saudi Arabia, where he directs the AI Initiative within the Computer, Electrical, and Mathematical Sciences and Engineering division.

Recurrent Architectures and Long Short-Term Memory

In the 1990s, Schmidhuber addressed the limitations of backpropagation in deep learning. Following the 1991 diploma thesis of his student Sepp Hochreiter, which analyzed the vanishing gradient problem, Schmidhuber facilitated the development of long short-term memory (LSTM). Introduced in a 1995 technical report and published in 1997, LSTM became a dominant technique for natural language processing. Subsequent refinements included the standard architecture in 2000 with Felix Gers and Fred Cummins, and connectionist temporal classification with Alex Graves in 2006.

Innovations in Neural Networks

Beyond LSTM, Schmidhuber explored diverse neural network concepts. In 1991, he introduced adversarial neural networks, termed artificial curiosity, where competing networks interact through a zero-sum game. This framework informs modern generative adversarial networks. In 1992, he published the fast weights programmer, a precursor to the unnormalized linear transformer. Furthermore, in 2011, his team at IDSIA demonstrated significant speed improvements for convolutional neural networks by utilizing graphics processing units, achieving superhuman performance in computer vision contests.

Commercial Ventures and Professional Recognition

In 2014, Schmidhuber founded NNAISENSE to explore commercial AI applications in sectors such as finance and heavy industry. While initially focused on artificial general intelligence, the company later shifted its emphasis toward asset management. His contributions to the field have received formal recognition, including the 2016 IEEE Neural Networks Pioneer Award. He is a member of the European Academy of Sciences and Arts and the European Laboratory for Learning and Intelligent Systems.

Fast facts

• Born: 1963, Munich, Germany
• Education: Technical University of Munich (PhD 1991)
• Key development: Long short-term memory (LSTM)
• Industry focus: AI applications via NNAISENSE
• Current position: Director, AI Initiative at KAUST
• Notable award: IEEE Neural Networks Pioneer Award (2016)

Questions readers ask

What is the significance of the fast weights programmer?

Published by Schmidhuber in 1992, the fast weights programmer is an alternative to recurrent neural networks that was later identified as equivalent to the modern unnormalized linear transformer.

What is the primary focus of the NNAISENSE company?

Founded in 2014, NNAISENSE initially targeted commercial AI applications in fields like finance and self-driving cars, later pivoting its focus toward asset management.

Achievements

• Affiliated with Dalle Molle Institute for Artificial Intelligence Research, Technical University of Munich and Università della Svizzera italiana
• Educated at Technical University of Munich
• Worked as computer scientist, artificial intelligence researcher and university teacher