Application of hyperspectral imaging technology in detection of loquat bruise grade

Loquat is a plant of the Rosaceae family and the genus Eriobotrya. It is an evergreen small tree known as the "crown of fruits". It is an economical fruit that can be used for both medicine and food. It has a sweet and sour taste and can be eaten raw, preserved, and used for brewing. Loquat matures in late spring and early summer, which is the off-season for fruits. It is a fruit that is not sold during the off-season. From maturity to final sale, loquats need to go through a series of processes such as picking, storage, packaging, and transportation. Because loquats have a thin skin, fine texture, soft and juicy texture, they are very easy to be bruised during this process, and the surface turns black, which affects terminal sales and causes great economic losses. Therefore, it is very important to judge whether loquats are bruised and the degree of bruises. Selecting bruised loquats in advance can save storage and transportation costs: loquats with minor bruises can be made into loquat juice, loquat paste, etc.; loquats with slightly more severe bruises can be removed and canned loquats can be preserved; loquats with severe bruises can be directly disposed of to save storage costs.

At present, during the picking and post-harvest processing of loquats, whether loquats are damaged is often identified by the operator's naked eye, which is affected by personal habits, light intensity and subjective psychological factors, with low efficiency and poor accuracy. Therefore, a method is needed to achieve high-precision, rapid and non-destructive detection of the degree of loquat bruises.

1. Experimental part

1.1 Samples

The experimental sample loquats were purchased from an orchard, totaling 135. In order to reduce the influence of other irrelevant factors on the experiment, the size of loquats was about 60 mm on the long axis and 40 mm on the short axis. Before the test, the samples were selected to remove surface damage and deformed samples to ensure that the samples had no defects in appearance and no mechanical damage. Finally, the surface of the loquats was cleaned and numbered.

Traditional manual classification was based on the standard of fresh loquats in GB/T 13867-1992. The operator roughly divided the bruised loquats based on personal experience. Due to the subjectivity of the operator and environmental influences such as light intensity, the classification of loquats is prone to misjudgment. Loquats themselves have individual differences. Affected by the hardness, size, maturity, etc. of the loquats themselves, under the same force, the area of the loquat damage area and the depth of the bruises will also be different. This does not conform to the principle of a single variable. Therefore, this study adjusted the force by controlling the collision height, and kept the same force to collide with the samples in the same group to obtain bruised loquats. The magnitude of the force is simulated by the falling force in the real environment, and the magnitude of the force is deduced by the conventional falling height of the loquat of standard mass. The appropriate metal ball is further selected according to the collision area, and the collision height is finally calculated.

The surface bruised sample in the experiment is obtained by a free fall collision device (as shown in Figure 1). A metal ball with a diameter of 30 mm and a mass of 100 g is free-falling at 0.4, 0.5 and 0.6 m from the loquat surface to hit the equatorial area of the loquat, so as to simulate loquats with different degrees of damage in reality. When the switch is closed, the electromagnetic induction device projects infrared light vertically downward to locate the bruised area; when the switch is opened, the end of the electromagnetic induction device is filled with a magnetic field, and electromagnetic induction occurs. At this time, the collision metal ball is placed here to be fixed; finally, the switch is closed, the magnetic field disappears, and the metal ball is free-falling to collide with the loquat. After the operation is completed, the sample is stored in an environment at room temperature of 24 ℃ to keep the sample temperature consistent with the room temperature. Loquats with different bruise levels at the same storage time are shown in Figure 2. The experiment began after the bruised samples were left to stand for 3 hours, and the hyperspectral imaging system was used to obtain hyperspectral images of loquats with mild, moderate, and severe bruises for subsequent model building. Since the loquat samples were measured one by one, the measurement time of all samples was slightly different.

This study applied a 400-1000nm hyperspectral camera, and the FS13 product of Hangzhou Caipu Technology Co., Ltd. can be used for related research. The spectral range is 400-1000nm, the wavelength resolution is better than 2.5nm, and up to 1200 spectral channels. The acquisition speed can reach 128FPS in the full spectrum, and the highest after band selection is 3300Hz (supporting multi-region band selection).

The average reflectance spectrum of loquat is shown in Figure 4. The spectral waveform change trends of loquats of several grades are basically the same. The peaks and troughs are located at the same wavelength, but the reflectance values are different. At the same wavelength, as the degree of bruise increases, the reflectance decreases. The reason for this is that after the collision of loquats, the original cell wall and cell membrane are damaged, and the water inside the cell is lost to the surface of the loquat. As the degree of bruise increases, the released water increases. In the image, the surface of the loquat turns black, and the longer the loquat is bruised, the darker the color of the bruised part; in the spectrum, due to the release of water from the inside of the loquat, the surface water content increases and the reflectance decreases. Since the loss of water in loquat cells is a slow-changing process, with the loss of water, the surface color and spectrum of the loquat will also change slowly. Therefore, the spectrum detection results after the collision of loquats at different times will be slightly different, but the overall trend is unchanged. In this experiment, the spectrum of loquats was collected 3 hours after they were bruised. This is because the transportation time from the collection of loquats from farmers' orchards to storage is about 3 hours. High-precision sorting of bruised loquats before storage can reduce the losses caused by rot and infection of normal loquats by bruised loquats.

The resolution of the loquat image collected by the hyperspectral imaging system is 960×366 pixels. The image contains too much conveyor belt background. Because it is not a pure black background, it has gray values under the action of halogen lamp. The variables used in the experiment are the mean of the RGB channel and the mean of the HSI model as variables combined with spectral information for modeling and analysis. Therefore, the threshold segmentation method is used to separate the loquat sample image from the picture as the foreground. According to the obtained boundary value, the image mask method is used to retain the foreground gray value for subsequent calculation, and the color feature and spectral feature are mixed for modeling and analysis. The overall process is shown in Figure 6. The extracted spectral features, image RGB features, and image HSI features are used to establish four loquat bruise degree models: spectral feature model, spectral feature combined with image RGB feature model, spectral feature combined with image HSI feature model, and spectral feature combined with mixed image feature model.

2. Experimental analysis

2.1 Analysis of loquat bruise degree based on RF algorithm

RF integrates multiple weak classifiers and selects the category with the most votes as the classification result from the classification results output by multiple classifiers. It has high accuracy and generalization ability. The loquat bruise degree model established based on the RF algorithm is shown in Table 1. The prediction accuracy of the RF model based on spectral features, spectral features combined with RGB image features, spectral features combined with HSI image features, and spectral features combined with mixed image features are 86.67%, 91.11%, 91.11%, and 91.11%, respectively. In the RF model, the model based on spectral features has the lowest accuracy. The accuracy of the models established after adding color features has been improved. However, the accuracy of the model set in the case of spectral features combined with RGB color features, HSI color features, and mixed image features is the same. The number of misjudgments within the observation group shows that the improvement in accuracy is mainly achieved by reducing the number of misjudgments in the severe bruise group.

2.2 Analysis of loquat bruise degree based on PLS-DA algorithm

PLS-DA is a statistical method that reduces the dimensionality of high-dimensional data and establishes a regression model and analyzes the results. The loquat bruise degree model established based on the PLS-DA algorithm is shown in Table 2. The PLS-DA model based on spectral features, spectral features combined with RGB image features, spectral features combined with HSI image features, and spectral features combined with mixed image features has a modeling set accuracy of 88.89%, 91.11%, 91.11%, and 90%, respectively, and a prediction set accuracy of 86.67%, 86.67%, 88.89%, and 86.67%, respectively. In the PLS-DA model, from the results of the training set, the model based on spectral features has the lowest accuracy, and the accuracy of the models established after adding color features has been improved. The prediction set accuracy of the spectral feature, spectral feature combined with RGB color feature, and mixed image feature models is the same. This is due to the small number of samples in the prediction set group. Their RMSECs are 0.323, 0.304, and 0.321, respectively. Compared with the single spectral feature model, the model combined with color features has a smaller RMSEC and better stability.

3. Conclusion

The hyperspectral images of loquat samples with different degrees of bruises were collected using a hyperspectral imaging system. The image information of loquat samples was extracted based on threshold segmentation and image masking methods. The average grayscale values of the R, G, and B channels and the average grayscale values of the H, S, and I channels were extracted from the bruised loquat images as the color features of the loquat bruise degree model, and the average spectrum of 100 pixels in the interest area was used as the spectral feature. The spectral features and color features were combined with chemometric methods to establish spectral features, spectral features combined with RGB color features, spectral features combined with HSI color features, and spectral features combined with mixed color features for qualitative judgment. The results showed that among the loquat bruise degree models established using RF, PLS-DA, ELM, LIN-LS-SVM, and RBF-LS-SVM algorithms, the spectral feature combined with the mixed color feature model had the best classification effect, with accuracy rates of 91.11%, 86.67%, 95.56%, and 100%, respectively. The RBF-LS-SVM model had the highest accuracy, reaching 100%. The hyperspectral imaging system was used to achieve qualitative analysis of loquats with different bruise degrees. This study provides a theoretical basis for the subsequent use of hyperspectral imaging technology combined with color features to qualitatively identify fruits.