About the project
This project began as an assignment for my university, in collaboration with the Netherlands Plant Eco-phenotyping Center (NPEC), prepared by one of my lecturers, Dr. Alican Noyan. NPEC aimed to automate the measurement of plants and accurately identify the locations of plant roots (root top and root tip) using only high-quality images of the plants. The performance of the algorithms was promising enough that I continued working with NPEC even after the educational phase of the project concluded.
As a result, this project is divided into two phases: the educational phase and a subsequent phase where, together with four classmates, we are working to improve the results and co-author a research paper with NPEC.
Phase 1
Dataset
The dataset for this project consists of 129 petri dish images, each with corresponding root and shoot masks. The images are 4200 x 3000 pixels in size.
Approach
The first step was to automatically crop the black edges around the petri dish to retain only the part of the image containing the most useful information. Due to the large size of the images, they were divided into patches (256 x 256 pixels each).
Next, a basic U-Net model was trained to perform root segmentation, and a separate U-Net model was trained to segment the shoots. Below is a video showing the progress of the root U-Net model during training:
Post-processing algorithms were applied to reduce noise in the root segmentation output, after which the output was skeletonized. To identify the root tip and top, we took the highest and lowest points of the root, respectively. The Dijkstra algorithm was used to calculate the distance between the lowest and highest points, giving the length of the primary root. Any other root system endpoint was considered to be the end of a lateral root.
Result single plant GIF
Robotic Plant Innoculation
In addition to root segmentation, we also had to inoculate the roots at the root tips using a virtual environment. A simulated version of the OT-2 robot was created by one of my lecturers for this task. We experimented with two approaches to control the robot: a reinforcement learning algorithm and a PID controller, with the PID controller providing the best results.
Complete Pipeline Results
For the final delivery of the university project, I developed a complete pipeline where an image of plant roots was loaded into the simulated OT-2 robot. The image was passed through the segmentation pipeline, which returned the location of each root tip. The PID-controlled robot then moved to the root tips and inoculated the roots, as showcased in the video below:
Phase 2
After the educational phase, a few of my classmates and I decided to continue improving the project and eventually co-write a research paper with NPEC and our mentor, Dr. Alican Noyan.
During this second phase, I was responsible for developing an algorithm to identify which roots belong to which plants. To solve this problem, I converted the root structures into graphical networks. The algorithm traces all the roots extending from the primary root, and when roots overlap or collide, it calculates which path is more likely to belong to the root being followed.
Results
The Complete Pipeline with Real-Life Inoculation
When given a time series of plant images, the algorithm can also output the root lengths over time:
