refs.md

Week 1: Knowledge Distillation

Main References

• Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. "Distilling the knowledge in a neural network."

• Mirzadeh, Seyed-Iman, et al. "Improved knowledge distillation via teacher assistant: Bridging the gap between student and teacher."

• Lopez-Paz, David, et al. "Unifying distillation and privileged information."

Further References

• Sanh, Victor, et al. "DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter."

• Hubara, Itay, et al. "Quantized neural networks: Training neural networks with low precision weights and activations."

• Polino, Antonio, Razvan Pascanu, and Dan Alistarh. "Model compression via distillation and quantization."

Week 2: Lottery ticket hypothesis

Main References

• Frankle, Jonathan, and Michael Carbin. "The lottery ticket hypothesis: Finding sparse, trainable neural networks."

• Frankle, Jonathan, et al. "The Lottery Ticket Hypothesis at Scale."

• Zhou, Hattie, et al. "Deconstructing lottery tickets: Zeros, signs, and the supermask."

Further References

• Gaier, Adam, and David Ha. "Weight Agnostic Neural Networks."

Week 3: Computational learning theory (PAC Learning)

Main References

• "Understanding Machine Learning: From Theory to Algorithms." Textbook by Shai Ben-David and Shai Shalev-Shwartz.

• "Foundations of Machine Learning (Adaptive Computation and Machine Learning series) Second edition" – 2018. Mehryar Mohri, Afshin Rostamizadeh, Ameet Talwalkar

• "An Introduction to Computational Learning Theory." Book by Michael Kearns and Umesh Virkumar Vazirani.

Week 4: Causality (Part 1)

Main References

• "Causality: Models, Reasoning and Inference". 2009. Second edition. by Judea Pearl

• "The Book of Why: The New Science of Cause and Effect." Book by Dana Mackenzie and Judea Pearl. 2018

• "Elements of Causal Inference: Foundations and Learning Algorithms." Book by Bernhard Schölkopf, Dominik Janzing, and Jonas Peters

Week 5: Causality (Part 2)

Main References

• "Causality: Models, Reasoning and Inference". 2009. Second edition. by Judea Pearl

• "The Book of Why: The New Science of Cause and Effect." Book by Dana Mackenzie and Judea Pearl. 2018

• "Elements of Causal Inference: Foundations and Learning Algorithms." Book by Bernhard Schölkopf, Dominik Janzing, and Jonas Peters

Week 6: Gaussian process

Main References

• "Gaussian Processes for Machine Learning." Book by Carl Edward Rasmussen and Christopher K. I. Williams

• "Gaussian Processes: From the Basics to the State-of-the-Art." Dr. Richard E. Turner.

• Görtler, et al., "A Visual Exploration of Gaussian Processes", Distill

• "Deep Gaussian Processes." Damianou, A. and Lawrence, N., 2013. Proceedings of the Sixteenth International Conference on Artificial Intelligence and Statistics

Week 7: Successor Representation in Reinforcement Learning

Main References

• Dayan, Peter. "Improving generalization for temporal difference learning: The successor representation."

• Gershman, Samuel J. "The successor representation: its computational logic and neural substrates."

• Kulkarni, Tejas D., et al. "Deep successor reinforcement learning."

• Barreto, André, et al. "Successor features for transfer in reinforcement learning."

• The present in terms of the future: Successor representations in Reinforcement learning. Blog post by Arthur Juliani.

Further References

• Botvinick, Matthew, and Ari Weinstein. "Model-based hierarchical reinforcement learning and human action control."

Week 8: Distributional Reinforcement Learning

Main References

• Bellemare, Marc G., Will Dabney, and Rémi Munos. 2017. “A Distributional Perspective on Reinforcement Learning.”

• Dabney, W., M. Rowland, M. G. Bellemare, and R. Munos. 2017. “Distributional Reinforcement Learning with Quantile Regression.”

• Mavrin, Borislav, et al. "Distributional Reinforcement Learning for Efficient Exploration."

• Yang, Derek, et al. "Fully Parameterized Quantile Function for Distributional Reinforcement Learning."

Further References

• Going beyond average for reinforcement learning. DeepMind blog post

• Distributional RL: Simple Machine Learning blog post

Week 9: Bayesian Interpolation

Main References

• MacKay, David J. C. (1992) Bayesian interpolation. Neural Computation

• Probabilistic machine learning and artificial intelligence

Further References

• Wilson, A.G. and Izmailov, P., 2020. Bayesian deep learning and a probabilistic perspective of generalization.