Geoffrey Hinton | On working with Ilya, choosing problems, and the power of intuition

Listening to Jeffrey Hinton is such a joy. He seems to me one of the most authentic and transparent souls among the famous people. There are many talented beings worth of admiration. But If I were given the choice to meet and share time to talk with some of them, he would be absolute first in my list.

The most pleasant and articulate voice in the entire ai space right now. Absolutely no jargon, just clean and crisp explanations, only using terms for big concepts when necessary.

Nice to see the interviewer was also standing for the whole interview in honour of hinton's back pain. Always remember PSR (principle of sufficient reason): there is reason behind every event.

"these big neural nets can actually do much better than their training data." things like this mentioned in this talk challenge us to look over the concepts we previously missed. This is by far one of the best interview from Hinton.

Ability to explain complicated things in a simple way is sign of deep understanding. Best interview I know so far in this field. And offcause right questions supported it.

Geoffrey strikes me as a genuine ethical human, I hope he never hesitates to be open about observed dilemmas.

Purely by speaking, he transfers much more insight and information than I would see in most papers

one of the best interviews I have ever watched!

By YouSum Live 00:01:16 Early disappointments in brain understanding. 00:01:59 Influence of Donald Hebb and John Fornoyman. 00:02:33 Brain learning through neural net connections. 00:04:13 Collaborations with Terry Sinowski and Peter Brown. 00:05:08 Encounter with a young, intuitive student, Ilia. 00:06:00 Ilia's unique perspective on gradient optimization. 00:08:02 Scale and computation's impact on AI progress. 00:08:25 Breakthrough in character-level prediction models. 00:09:01 Neural net language models' training insights. 00:10:36 Integration of reasoning and intuition in models. 00:12:46 Potential for models to surpass human knowledge. 00:17:16 Multimodal learning enhancing spatial understanding. 00:18:18 Impact of multimodality on model reasoning abilities. 00:18:40 Evolutionary perspective on language and brain synergy. 00:18:41 Evolution of language and cognition. 00:18:57 Three views of language and cognition. 00:20:12 Transition from symbolic to vector-based cognition. 00:21:36 Impact of GPUs on neural net training. 00:23:13 Exploration of analog computation in hardware. 00:25:31 Importance of diverse time scales in learning. 00:27:37 Validation of neural networks' learning capabilities. 00:29:12 Inquiry into simulating human consciousness. 00:36:01 Brain's potential use of backpropagation for learning. 00:37:42 Brain's learning potential and beneficial failures. 00:38:02 AI advancements in healthcare for societal benefit. 00:39:00 Concerns about misuse of AI by malevolent actors. 00:39:23 International AI competition driving rapid progress. 00:40:03 AI assistants enhancing research efficiency and problem-solving. 00:41:51 Intuition in talent selection and diverse student profiles. 00:42:00 Developing intuition by filtering information effectively. 00:43:26 Focus on big models and multimodal data for AI progress. 00:44:08 Exploration of various learning algorithms for AI advancement. 00:45:11 Pride in developing the learning algorithm for Boltzmann machines. By YouSum Live

For the first time in my life, i see an interview and the sensation of "Fantastic questions" keeps popping up. Thank You!

The way that Hinton breaks complicated things down is another level. It proves again that if someone can't explain something to ANYONE, they don't understand it that much either

It is an absolute delight listening to his humble and patient explanation to the very basics from the very beginning of an Industry which has already crossed 5 Trillion 💵 💲 diverse across in hardware, semiconductor applications and had a revolutionary impact in Healthcare and Data Modelling domains. 🎉

Wow. All the questions were just great. Thanks for asking them.

Hinton is great. If you, with 4k subs, can get Hinton, please try to get Sutskever.

Here's a ChatGPT summary: - Geoffrey Hinton reflects on his intuitive approach to identifying talent, mentioning Ilya Sutskever's persistence and raw intuition. - Hinton describes his early experiences at Carnegie Mellon, including late-night programming sessions and the collaborative environment. - He discusses his transition from neuroscience to AI, influenced by books from Donald Hebb and John von Neumann. - Hinton emphasizes the importance of understanding how the brain learns and modifies connections in neural networks. - He recalls collaborations with Terry Sinowski and Peter Brown, highlighting their contributions to his understanding of neural networks and speech recognition. - Hinton shares the story of Ilya Sutskever's first meeting with him and Sutskever's intuitive approach to problem-solving. - He discusses the evolution of AI models, emphasizing the importance of scale and data in improving performance. - Hinton explains the concept of neural net language models and their ability to understand and predict the next symbol in a sequence. - He highlights the potential of large language models like GPT-4 to find common structures and make creative analogies. - Hinton discusses the potential for AI to go beyond human knowledge, citing examples like AlphaGo's creative moves. - He reflects on the importance of multimodal models in improving AI's understanding and reasoning capabilities. - Hinton shares his views on the relationship between language and cognition, favoring a model that combines symbolic and vector-based representations. - He recounts his early intuition about using GPUs for training neural networks and the subsequent impact on the field. - Hinton discusses the potential for analog computation to reduce power consumption in AI models. - He emphasizes the importance of fast weights and multiple timescales in neural networks, drawing parallels to the brain's temporary memory. - Hinton reflects on the impact of AI on his thinking and the validation of stochastic gradient descent as a learning method. - He discusses the potential for AI to simulate human consciousness and feelings, drawing on examples from robotics. - Hinton shares his approach to selecting research problems, focusing on challenging widely accepted ideas. - He highlights the importance of curiosity-driven research and the potential for AI to benefit society, particularly in healthcare. - Hinton expresses concerns about the misuse of AI by bad actors for harmful purposes. - He discusses the role of intuition in selecting talent and the importance of having a strong framework for understanding reality. - Hinton advocates for focusing on large models and multimodal data as a promising direction for AI research. - He reflects on the importance of learning algorithms and the potential for alternative methods to achieve human-level intelligence. - Hinton expresses pride in the learning algorithm for Boltzmann machines, despite its practical limitations. - Main message: Geoffrey Hinton emphasizes the importance of intuition, collaboration, and curiosity-driven research in advancing AI, while acknowledging the potential benefits and risks of AI technology.

Fantastic interview! The questions were awesome, the answers profound.

The work of Geoffrey Hinton on backpropagation is remarkable and has significantly accelerated the progress of AI, as we are experiencing today.

What a great interview. It actually captured some genuine conceptual insights - very rare for an interview on the subject these days!

“What do you think is the reason for some folks having better intuition? Do they just have better training data?” “I think it’s partly they don’t stand for nonsense”

What should I watch on Netflix is the question that half of humanity is struggling with these days. If only AI could help with that.