Bachelor's Thesis presentation. Danylo is advised by Severin Reiz.
The SCCS Colloquium is a forum giving students, guests, and members of the chair the opportunity to present their research insights, results, and challenges. Do you need ideas for your thesis topic? Do you want to meet your potential supervisor? Do you want to discuss your research with a diverse group of researchers, rehearse your conference talk, or simply cheer for your colleagues? Then this is the right place for you (and you are also welcome to bring your friends along).
Danylo Movchan: Implementing a learning-rate scheduler in a Newton-CG Optimizer for Deep Learning
SCCS Colloquium | •
Nowadays, Deep Neural Networks models are at the peak of their popularity and find applications in a variety fields, e.g. in translation engines, where Natural Language Processing is used. Training such networks requires enormous computing resources and can take up to 2 weeks, and most often uses rather naive first-order optimization algorithms. Given the fact that modern deep neural networks have several millions of parameters, second-order methods have long been considered unfeasible because of their quadratic complexity of network size.
Previous studies have shown that combining Newton's method with Conjugate Gradients method and Fast Exact Multiplication (short: Newton-CG) leads to speed-up and accuracy benefits in areas such as image classification and neural machine translation. In our work we tried to determine whether the use of different learning rate schedulers can help to develop these benefits, eliminating some of the drawbacks of standard Newton-CG.
The Newton-CG with learn-rate scheduler allows for bigger initial learning rates, while still being stable close to minimum, and thus, faster training.
Contribute a talk
To register and schedule a talk, you should fill the form Colloquium Registration at least two weeks before the earliest preferred date. Keep in mind that we only have limited slots, so please plan your presentation early. In special cases, contact firstname.lastname@example.org.
We invite students doing their Bachelor's or Master's thesis, as well as IDP, Guided Research, or similar projects at SCCS to give one 20min presentation to discuss their results and potential future work. The time for this is typically after submitting your final text. Check also with your study program regarding any requirements for a final presentation of your project work.
New: In regular times, we will now have slots for presenting early stage projects (talk time 2-10min). This is an optional opportunity for getting additional feedback early and there is no strict timeline.
Apart from students, we also welcome doctoral candidates and guests to present their projects.
During the colloquium, things usually go as follows:
- 10min before the colloquium starts, the speakers setup their equipment with the help of the moderator. The moderator currently is Irene López. Make sure to be using an easily identifiable name in the online session's waiting room.
- The colloquium starts with an introduction to the agenda and the moderator asks the speaker's advisor/host to put the talk into context.
- Your talk starts. The scheduled time for your talk is normally 20min with additional 5-10min for discussion.
- The moderator keeps track of the time and will signal 2min before the end of time (e.g. by turning on their video).
- During the discussion session, the audience can ask questions, which are meant for clarification or for putting the talk into context. The audience can also ask questions in the chat.
- Congratulations! Your talk is over and it's now time to celebrate! Have you already tried the parabolic slides that bring you from the third floor to the Magistrale?
Do you remember a talk that made you feel very happy for attending? Do you also remember a talk that confused you? What made these two experiences different?
Here are a few things to check if you want to improve your presentation:
- What is the main idea that you want people to remember after your presentation? Do you make it crystal-clear? How quickly are you arriving to it?
- Which aspects of your work can you cover in the given time frame, with a reasonable pace and good depth?
- What can you leave out (but maybe have as back-up slides) to not confuse or overwhelm the audience?
- How are you investing the crucial first two minutes of your presentation?
- How much content do you have on your slides? Is all of it important? Will the audience know which part of a slide to look at? Will somebody from the last row be able to read the content? Will somebody with limited experience in your field have time to understand what is going on?
- Are the figures clear? Are you explaining the axes or any other features clearly?
In any case, make sure to start preparing your talk early enough so that you can potentially discuss it, rehearse it, and improve it.
Here are a few good videos to find out more:
- Simon Peyton Jones: How to Give a Great Research Talk (see also How to Write a Great Research Paper)
- Susan McConnell: Designing effective scientific presentations
- Jens Weller: Presenting Code
Did you know that the TUM English Writing Center can also help you with writing good slides?
Work with us!
Do your thesis/student project in Informatics / Mathematics / Physics: Student Projects at the SCCS.