Insect-Foundation: A Foundation Model and
Large-scale 1M Dataset for Visual Insect Understanding

https://uark-cviu.github.io/
Hoang-Quan Nguyen
Thanh-Dat Truong
Xuan-Bac Nguyen
Ashley Dowling
Xin Li
Khoa Luu
CVIU Lab
EECS Department
University of Arkansas
College of Nanotechnology, Science, and Engineering
University at Albany
ENPL Department
Division of Agriculture
University of Arkansas




Abstract

In precision agriculture, the detection and recognition of insects play an essential role in the ability of crops to grow healthy and produce a high-quality yield. The current machine vision model requires a large volume of data to achieve high performance. However, there are approximately 5.5 million different insect species in the world. None of the existing insect datasets can cover even a fraction of them due to varying geographic locations and acquisition costs. In this paper, we introduce a novel “Insect1M” dataset, a game-changing resource poised to revolutionize insect-related foundation model training. Covering a vast spectrum of insect species, our dataset, including 1 million images with dense identification labels of taxonomy hierarchy and insect descriptions, offers a panoramic view of entomology, enabling foundation models to comprehend visual and semantic information about insects like never before. Then, to efficiently establish an Insect Foundation Model, we develop a micro-feature self-supervised learning method with a Patch-wise Relevant Attention mechanism capable of discerning the subtle differences among insect images. In addition, we introduce Description Consistency loss to improve micro-feature modeling via insect descriptions. Through our experiments, we illustrate the effectiveness of our proposed approach in insect modeling and achieve State-of-the-Art performance on standard benchmarks of insect-related tasks. Our Insect Foundation Model and Dataset promise to empower the next generation of insect-related vision models, bringing them closer to the ultimate goal of precision agriculture.

Download

OneDrive


- Numeber of Images: 1,017,036 - Number of Species: 34,212

Taxonomy Hierarchy

- Number of Class: 15 - Number of Order: 91 - Number of Suborder: 54
- Number of Superfamily: 209 - Number of Family: 1189 - Number of Subfamily: 1059
- Number of Tribe: 1315 - Number of Subtribe: 213 - Number of Genus: 11127



Insect Foundation Model

Code and Model


Architecture Parameters Download
ViT-S/16 21M Comming Soon
ViT-B/16 85M Comming Soon

Examples



Teams

Current Members

Thanh-Dat Truong Hoang-Quan Nguyen Jesse Ford Ashley Dowling Xin Li Khoa Luu
Ph.D. Candidate Ph.D. Student Undergrad Professor Professor Professor
University of Arkansas University of Arkansas University of Arkansas University of Arkansas University at Albany University of Arkansas
tt032@uark.edu hn016@uark.edu jdf028@uark.edu adowling@uark.edu xli48@albany.edu khoaluu@uark.edu

Alumni

Pierce Helton Ahmed Moustafa
Undergrad Undergrad
University of Arkansas University of Arkansas

Related News

Insects Get 'Caught on Camera' to Help Farmers in Latest Short Takes
Short Takes: Caught on Camera: Insects Edition
Training AI to Detect Crop Pests (Oct 3, 2022)
Researchers Receive NSF Funding to Build a Smarter Insect Trap

Publications

Patents

[1] Thanh-Dat Truong, Ashley PG Dowling, Randy J. Sasaka, and Khoa Luu. Sensor-based Smart Insect Monitoring System in the Wild. US Patent, PCT/US2023/021330.

[2] Thanh-Dat Truong, Ashley PG Dowling, and Khoa Luu. Smart Insect Control Device via Artificial Intelligence in Realtime Environment. US Patent, PCT/US2023/021307.

Papers

[1] Hoang-Quan Nguyen, Thanh-Dat Truong, Xuan Bac Nguyen, Ashley Dowling, Xin Li, Khoa Luu. Insect-Foundation: A Foundation Model and Large-scale 1M Dataset for Visual Insect Understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.

[2] Thanh-Dat Truong, Chi Nhan Duong, Pierce Helton, Ashley Dowling, Xin Li, and Khoa Luu. CoMaL: Conditional Maximum Likelihood Approach to Self-supervised Domain Adaptation in Long-tail Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence (Under Review).

[3] Thanh-Dat Truong, Ngan Le, Bhiksha Raj, Jackson Cothren, and Khoa Luu (2023). FREDOM: Fairness Domain Adaptation Approach to Semantic Scene Understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.

[4] Pierce Helton, Ashley Dowling, and Khoa Luu. Artificial Intelligence System for Automatic Imaging, Quantification, and Identification of Arthropods in Leaf Litter and Pitfall Samples. Inquiry Journal, 2022.

[5] Thanh-Dat Truong, Naga Venkata Sai Raviteja Chappa, Xuan Bac Nguyen, Ngan Le, Ashley Dowling, and Khoa Luu. OTAdapt: Optimal Transport-based Approach For Unsupervised Domain Adaptation. In Proceedings of International Conference on Pattern Recognition (ICPR), 2022.

[6] Thanh-Dat Truong, Chi Nhan Duong, Ngan Le, Son Lam Phung, Chase Rainwater, and Khoa Luu. BiMaL: Bijective Maximum Likelihood Approach to Domain Adaptation in Semantic Scene Segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021.

[7] Jalata, Ibsa, Naga Venkata Sai Raviteja Chappa, Thanh-Dat Truong, Pierce Helton, Chase Rainwater, and Khoa Luu. "EQAdap: Equipollent Domain Adaptation Approach to Image Deblurring." IEEE Access 10 (2022): 93203-93211.