Internship | Infrared Foundation Model

Introduction
Foundation models have changed computer vision. Due to their broad pre-training, they are ideal starting points for further finetuning for a range of downstream tasks, leading to new possibilities even with limited amounts of training data. For RGB image tasks, foundation models have become a default choice. However, for infrared images this is not so straightforward. Recently, a foundation model for infrared images was proposed (Liu et al, ECCV 2024), InfMAE, using an information-aware masking strategy, because infrared images exhibit less diversity and less information content (e.g. less texture).

Surprisingly, finetuning this infrared model for object detection and semantic segmentation, did not result in significantly better results, compared to finetuning a large RGB model. Both for the finetuned RGB models as well as the infrared model, a good performance could be achieved on the infrared tasks. Sometimes a RGB model performed even better than the infrared model. That begs the question: how to get most out of RGB foundation models (e.g. DINO v2) for infrared tasks?

Liu, F., Gao, C., Zhang, Y., Guo, J., Wang, J., & Meng, D. (2024, September). InfMAE: A foundation model in the infrared modality. In European Conference on Computer Vision (pp. 420-437). Cham: Springer Nature Switzerland.

Hu, Z., Yang, B., & Ye, M. (2025). Empowering Visible-Infrared Person Re-Identification with Large Foundation Models. Advances in Neural Information Processing Systems, 37, 117363-117387.

Fan, R., Zhao, W., Lin, M., Wang, Q., Liu, Y. J., & Wang, W. (2024, May). Generalizable Thermal-based Depth Estimation via Pre-trained Visual Foundation Model. In 2024 IEEE International Conference on Robotics and Automation (ICRA) (pp. 14614-14621). IEEE.

Yuan, M., Cui, B., Zhao, T., Wang, J., Fu, S., & Wei, X. (2024). UniRGB-IR: A Unified Framework for RGB-Infrared Semantic Tasks via Adapter Tuning. arXiv preprint arXiv:2404.17360.

What will you be doing?
Explore the opportunities of foundation models for infrared images. Research how to transfer the knowledge in RGB to infrared and how to finetune it adequately for infrared. Investigate the added value for object detection by integrating the recipes into a modern model such as Grounding DINO.

You will perform this assignment within TNO’s Intelligent Imaging department. The Intelligent Imaging department is a passionate, creative, and dedicated team of professionals (60 people) specializing in developing groundbreaking applications in the field of computer vision. Our team members have diverse backgrounds, ranging from the medical field to artificial intelligence. Intelligent Imaging is a young and growing department that has built up a lot of expertise over the past years in AI and deep learning.

We are looking for a student that wants to join our cutting-edge research team. This position is perfect for students who are passionate about AI and advanced machine learning techniques. Additional requirements include that you are in the final stages of your master's degree in artificial intelligence, computer science, physics, mathematics, electrical engineering, or a similar degree. You have some experience in computer vision, artificial intelligence, deep learning and Python-programming.

Please include the following practical information in your cover letter:

You want an internship opportunity on the precursor of your career; an internship gives you an opportunity to take a good look at your prospective future employer. TNO goes a step further. It’s not just looking that interests us; you and your knowledge are essential to our innovation. That’s why we attach a great deal of value to your personal and professional development. You will, of course, be properly supervised during your work placement and be given the scope for you to get the best out of yourself. Furthermore, we provide:

At TNO, we innovate for a healthier, safer and more sustainable life. And for a strong economy. Since 1932, we have been making knowledge and technology available for the common good. We find each other in wonder and ingenuity. We are driven to push boundaries. There is all the space and support for your talent and ambition. You work with people who will challenge you: who inspire you and want to learn from you. Our state-of-the-art facilities are there to realize your vision. What you do at TNO matters: impact makes the difference. Because with every innovation you contribute to tomorrow’s world. Read more about TNO as an employer.

At TNO we encourage an inclusive work environment, where you can be yourself. Whatever your story and whatever unique qualities you bring to the table. It is by combining our unique strengths and perspectives that we are able to develop innovations that make a real difference in society. Want to know more? Read what steps we are taking in the area of diversity and inclusion.

After the first CV selection, the application process will be conducted by the concerning department. TNO will provide a suitable internship agreement. If you have any questions about this vacancy, you can contact the contact person mentioned below.

For this internship vacancy it is required that the AIVD issues a security clearance (VGB) after conducting a security screening. Take into account that this process may take about 8 weeks. If you have been abroad for more than 6 consecutive months, or if you do not have the Dutch nationality, it may take longer. Read more about security screening on the AIVD website.

Important to be aware of before applying: