PhD Position Deep Meta-learning on Learning Curves to Improve Machine Learning
Delft University of Technology (TU Delft)
Challenge: Developing deep meta-learning algorithms to model and understand learning curve patterns in machine learning.
Impact: Faster, better, more cost-efficient training and tuning of learning.
We are looking for a critical and open-minded person to come work with us to deepen our understanding of learning curves. Learning curves in machine learning plot performance versus training set size. By extrapolating learning curves we can predict how many training samples are necessary for a particular performance. Learning curves can also be used to speed up learning algorithms, model selection, and hyperparameter tuning. However, unexpected and strange learning curves make this task difficult. Furthermore, there is much uncertainty about the general shape of learning curves: are they exponential, power law, or do they have other predictable shapes? A better understanding of learning curves can provide deeper insights into how learning algorithms work, and may inspire new machine learning theory and improved learning algorithms.
You will analyze learning curves, develop deeper knowledge about their shape, and exploit that knowledge to improve applications that rely on learning curves (hyperparameter tuning, model selection, predicting the amount of data needed). The plan in this project is to develop meta-learning algorithms for learning about learning curves. By analyzing a database composed of a large number of learning curves, we want to extract data-driven insights and exploit them. We see especially potential for deep learning methods (meta-learning and generative modeling), but there is room to explore and develop other kinds of models. Moreover, we are interested in translating findings into human-understandable insights (explainable AI) which can inspire new theoretical insights and new algorithms.
In this project you will develop new learning algorithms to learn from learning curves, develop new insights into learning curves, develop benchmarks, design, run, and analyze large-scale experiments. There is room in this project for freedom and creativity. On one hand, the project offers empirical challenges but there is also room for theoretical work. This can be balanced and explored according to your interests.
Predicting the amount of data necessary for learning is relevant for real-life settings where data collection is expensive, difficult, or time-consuming so that data collection costs can be minimized. This issue is important for small companies that want to apply machine learning in a cost-efficient manner. Model selection and hyperparameter tuning are compute-intensive tasks. New and more efficient algorithms based on learning curves will lead to more sustainable machine learning.
Our research environment offers a dynamic, stimulating, and diverse atmosphere, providing you with opportunities to collaborate with experts in the field. You will work within the Pattern Recognition and Bioinformatics group within the Computer Science department Intelligent Systems, which includes researchers working on machine learning, pattern recognition, computer vision, and socially perceptive computing. Our research group is very international and socially active. You will be advised by Tom Viering (http://tomviering.nl/).
We are looking for a candidate that meets the following criteria:
- A Master's degree or equivalent, or about to graduate with one, in a relevant field (Machine Learning, Statistics, Mathematics, Artificial Intelligence, Computer Science, Physics, Engineering, etc.).
- Solid background in linear algebra, probability theory, and statistics.
- Ability to program in Python, C++ or other programming language.
- Proficiency in spoken and written English.
- Can work together well in a team.
We encourage you to apply even if you feel you don't meet all the criteria above, as long as you are willing to acquire the complementary skills.
Doing a PhD at TU Delft requires English proficiency at a certain level to ensure that the candidate is able to communicate and interact well, participate in English-taught Doctoral Education courses, and write scientific articles and a final thesis. For more details please check the Graduate Schools Admission Requirements.
Conditions of employment
Doctoral candidates will be offered a 4-year period of employment in principle, but in the form of 2 employment contracts. An initial 1,5 year contract with an official go/no go progress assessment within 15 months. Followed by an additional contract for the remaining 2,5 years assuming everything goes well and performance requirements are met.
Salary and benefits are in accordance with the Collective Labour Agreement for Dutch Universities, increasing from € 2770 per month in the first year to € 3539 in the fourth year. As a PhD candidate you will be enrolled in the TU Delft Graduate School. The TU Delft Graduate School provides an inspiring research environment with an excellent team of supervisors, academic staff and a mentor. The Doctoral Education Programme is aimed at developing your transferable, discipline-related and research skills.
The TU Delft offers a customisable compensation package, discounts on health insurance and sport memberships, and a monthly work costs contribution. Flexible work schedules can be arranged.
For international applicants, TU Delft has the Coming to Delft Service. This service provides information for new international employees to help you prepare the relocation and to settle in the Netherlands. The Coming to Delft Service offers a Dual Career Programme for partners and they organise events to expand your (social) network.
TU Delft (Delft University of Technology)
Delft University of Technology is built on strong foundations. As creators of the world-famous Dutch waterworks and pioneers in biotech, TU Delft is a top international university combining science, engineering and design. It delivers world class results in education, research and innovation to address challenges in the areas of energy, climate, mobility, health and digital society. For generations, our engineers have proven to be entrepreneurial problem-solvers, both in business and in a social context.
At TU Delft we embrace diversity as one of our core values and we actively engage to be a university where you feel at home and can flourish. We value different perspectives and qualities. We believe this makes our work more innovative, the TU Delft community more vibrant and the world more just. Together, we imagine, invent and create solutions using technology to have a positive impact on a global scale. That is why we invite you to apply. Your application will receive fair consideration.
Challenge. Change. Impact!
Faculty Electrical Engineering, Mathematics and Computer Science
The Faculty of Electrical Engineering, Mathematics and Computer Science (EEMCS) brings together three scientific disciplines. Combined, they reinforce each other and are the driving force behind the technology we all use in our daily lives. Technology such as the electricity grid, which our faculty is helping to make completely sustainable and future-proof. At the same time, we are developing the chips and sensors of the future, whilst also setting the foundations for the software technologies to run on this new generation of equipment – which of course includes AI. Meanwhile we are pushing the limits of applied mathematics, for example mapping out disease processes using single cell data, and using mathematics to simulate gigantic ash plumes after a volcanic eruption. In other words: there is plenty of room at the faculty for ground-breaking research. We educate innovative engineers and have excellent labs and facilities that underline our strong international position. In total, more than 1000 employees and 4,000 students work and study in this innovative environment.
Click here to go to the website of the Faculty of Electrical Engineering, Mathematics and Computer Science.
For more information about this vacancy, or if you would like to have a chat before applying, please contact Tom Viering (firstname.lastname@example.org)
Are you interested in this vacancy? Please apply via the application button no later than 20 October 2023 and do not forget to include all the following documents:
- Motivation letter (max 2 pages) addressed to Tom Viering. The motivation letter should summarize: (i) why you would like to do a PhD, (ii) why you are interested in the project/topic, (iii) why your profile is suitable for the job, and (iv) what you hope to gain from the position.
- CV (max 2 pages).
- Academic transcripts (both MSc and BSc degrees).
- A pre-employment screening can be part of the selection procedure.
- You can apply online. We will not process applications sent by email and/or post.
- Please do not contact us for unsolicited services.