MARnet: multi-scale adaptive residual neural network for chest X-ray images recognition of lung diseases

1.   Introduction

Pesticide spraying is one of the most important farming processes, as it plays a crucial role in increasing agricultural productivity. The conventional spraying methods in strawberry greenhouses, typically, employ hazardous chemicals that are consistently applied by an operator with a manual sprayer. Despite the use of pesticide protection equipment (i.e., protective suit, gas mask, etc.), farmers are still exposed to toxic and dangerous chemicals that can cause health problems ^[1,2,3]. In addition, un-suitable spraying can infect both the plants and the soil, potentially causing harm to the final consumer. Because of this drawback, agricultural robotics has become a major interest of researchers in order to reduce the intervention of the farmers in risky situations, together with the protection of the environment and final consumers.

Nowadays, robots play a significant role in crop inspection and treatment as well as in weeds, pests, and diseases detection. Agricultural robots have been improved through the development of mobile platforms equipped with a variety of sensors and algorithms for performing agricultural tasks. Agricultural robotics mobile technology has been widely used for a variety of purposes, such as the harvesting of cherry tomatoes based on stereo vision and a fruit collector ^[4]. An autonomous robot has been developed for intra-row weeding in vineyards by means of a rotary weeder using sonar and feeler to detect the trunk of the plants ^[5]. Another example can be found in ^[6], where a wheeled robot has been designed for precise wheat seeding.

Robust and efficient autonomous navigation is a critical component for mobile robots that operate autonomously in agricultural environments, and it is becoming increasingly important in current robotics research. The majority of cultures are planted in straight rows with almost equal spacing between the rows, which favors the use of robotics to achieve agricultural tasks, e.g., pesticide spraying. This way, several agricultural navigation methods have been developed using plant rows as landmarks for navigation algorithms ^[7]. For that, vision sensors have been widely employed in plant rows identification due to their low cost and ability to provide a large amount of data that can be used to guide robots. Methods for detecting crop rows based on the Hough transform are broadly used in robot visual navigation systems ^[8,9,10]. Astrand and Baerveldt ^[8,9] developed a crop row recognition method that is also based on the Hough transform. Chen et al ^[10] successfully realized the automatic detection of the transplanting robot's navigation target by extracting the target row based on the improved Hough transform. Montalvo et al ^[11] proposed a least-squares method for detecting crop rows in a grassed cornfield.

On the other hand, the use of laser rangefinder (LIDAR) sensors for navigation has resulted in a great number of research contributions, making them one of the most commonly used sensor systems in robotic platforms. They have the advantages of providing direct distance measurements, being less sensitive to environmental variables (e.g. work in the dark) and having a greater range than other sensors. Also, the recent cost reductions have increased the interest in this technology. Because of these, they are commonly used for local navigation between cultural ranges, which entails determining the robot's relative location across cultural ranges and guiding directions to avoid collisions ^[12,13,14]. Barawid et al. ^[15] used a 2D LIDAR to develop a real-time guidance system for driving an autonomous vehicle in an orchard, using again the Hough transform to extract plant rows in order to guide the vehicle. In ^[12], Hiremath and colleagues propose an autonomous robot navigation model in a maize field using 2D LIDAR and a particle filtering algorithm. It estimates the robot's state in relation to its surroundings, such as the robot's cap and lateral deviation.

The main contribution of this work is the development of an autonomous navigation method for an agricultural mobile robot called "AgriEco Robot" operating in a strawberry greenhouse. We present a 2D LIDAR-based controller navigation approach that steers the robot using the estimated heading and lateral offset of the robot relative to crop rows. This allows the "AgriEco Robot" to autonomously guide itself between the crop rows while automatically spraying the pesticide. Then, after successfully navigating between the rows and detecting their ends, the robot autonomously takes care of reaching the next crop rows and continues the navigation between them while resuming the spraying process. The control software is based on Robot Operating System (ROS) ^[16], which runs on a Jetson Tx2 high-performance processor.

2.   Systems and methods

2.1.   Mobile robot description

The "AgriEco Robot" (Figure 1) has been devised to work specifically within an agricultural greenhouse and navigate between rows of strawberries for automatic pesticide spraying. The robot's dimensions (65 cm long x 55 cm wide) were chosen in accordance with strawberry cultivation standards.

The mobile robot platform is composed of a chassis made of steel, equipped with a four-wheel drive; a spraying system installed in the rear part of the robot, consisting of two motorized arms; and a tank of pesticide with an emergent pump. It also carries three on-board perception sensors: a stereo camera, a 2D LIDAR and an omnidirectional camera. In the rest of this section, we describe the robot's main components.

2.1.1.   Robotic platform

The robotic platform is composed of a robotic chassis and four-wheel-drive with an in-wheel brush-less direct current engine (BLDC) linked with its controllers (see Figure 1). The chassis is made of lightweight steel and has been designed and analyzed to support loads up to 100 kg and to tow up to 80 kg. The four wheels operate separately and rotate along a single axis ^[17,18]. Each in-wheel motor has an independent controller which provides, through an Arduino Mega 2560 microcontroller, total and smooth control over the following functions: forward and backward motion, brake, and speed variation.

The BLDC motor controller constitutes the driver of the motor. It acts as an intermediary between the motor and the battery, receiving a control signal from a microcontroller and converting it into a higher current signal that can drive the motor. The controller converts the DC voltage from the battery to AC voltage using power electronic switches. The controller, in turn, is characterized by a maximum current of 15 A and a power of 250 W and provides inputs for Hall effect sensors. PWM (Pulse Width Modulation) signals are generated and filtered by low-pass RC to control the motor speed variation. The "AgriEco Robot" is completely powered by a 36 V/30 Ah Lithium-ion battery.

2.1.2.   On-board sensors

The sensory system of the robot comprises a Hokuyo URG-04LX-UG01 2D LIDAR, a ZED stereo camera, and an omnidirectional camera system, but it is not used in this work. The 2D LIDAR (Figure 1) has been mounted in the front of the robot, at a height of 12 cm above the ground. According to the manufacturer, this device has a scanning area of 240° and provides an angle resolution of 0.36° with a scan frequency of 10 Hz. The detection range is approximately 20‒5,600 mm with an accuracy of ±30 mm at a distance between 60 and 1000 mm, and ±3% of the measurement up to 4095 mm. It is connected to the onboard Jetson TX2 via a USB 2.0 interface and is powered by an additional 5 V power source.

The ZED stereo camera (Figure 1), developed by Stereolabs ^[19], integrates two cameras with a maximum resolution of 4416 x 1242 px and a frame rate of 15 fps. It has been specially designed for autonomous navigation and 3D analysis applications, and it provides robust visual odometry estimations. The ZED SDK supports ROS integration through the zed-ros-wrapper package. It uses depth perception to accurately estimate the camera's 6 DoF pose (x, y, z, roll, pitch, yaw) with frequency up to 100 Hz and thus the pose of the system it is mounted on. The positional tracking accuracy for ZED cameras is +- (0.01, 0.1, 0.1) in meters for the x, y and z-axes. The camera has been positioned on the robot at a height of 44 cm from the ground, with a pitch angle of 15° so that it points to the ground.

The omnidirectional vision sensor (Figure 1) is composed of a CCD camera and a spherical mirror in a face-to-face configuration. The catadioptric system has been positioned perpendicular to the robot. It covers a 360-degree field of view of the robot environment which makes it suited for autonomous navigation, obstacle detection, and localized pesticide spraying.

2.1.3.   Processing unit

The "Agri-Eco Robot" systems uses a combination of CPU and GPU cores in a real time application, which implies the need for considerable computing power to fulfill the embedded sensors' requirements, like the Zed camera, which recommend NVIDIA TX2 ^[20] as the optimal embedded processing card. The Nvidia Jetson TX2 is a unit that is equipped with a Quad-core 2.0 Ghz 64-bit ARMv8 A57, a dual-core 2.0 Ghz ARMv8 Denver, a 256 CUDA core 1.3 MHz Nvidia Pascal and 8 GB memory. This embedded card runs the developed systems in 0.25 s.

It runs on an Ubuntu Linux 18.04 operating system with the open-source Melodic Robot Operating System (ROS). The general scheme of the complete system is shown in Figure 2.

Our system is based on the versatile and widely employed ROS framework ^[21], which is able to execute multiple programs in parallel in the form of nodes that communicate through services and messages broadcast via topics. ROS includes a set of libraries, a wide range of sensor drivers and a set of tools which make easy the implementation of the proposed application. The "AgriEco Robot" system implements in this work three ROS nodes:

● The ZED node (zed-ros-wrapper) broadcast its data in a set of topics providing access to the stereo images, the depth map, the 3D point cloud and an estimation of the camera's 6-DOF tracking.

● The Hokuyo node produces messages containing the values of the laser scans.

● The microcontroller node receives commands from other process and controls the robot motion and spraying systems.

2.2.   Spraying system

The spraying system (Figure 1 and Figure 3) has been developed with the aim of optimizing the automatic pesticide spraying by the "AgriEco Robot". This system has been attached to the rear part of the robot and is composed of a 10 L bank (1) of polythene with a submerged pump (2) and two motorized arms (5) built by 3D printing using poly-lactic acid (PLA) material. The pipe is made of pure silicone and is tied to the pump inside the bank. The subsystem is completed by a T-type connection (3) to distribute the pesticide to both arms. A set of adjustable flow nozzles (6) are fixed at the end of the pipes.

2.3.   Autonomous navigation method

The "AgriEco Robot", must be able to navigate through strawberry crop rows within an agricultural greenhouse to perform tasks such as automatic of spraying pesticide. For this purpose, we have developed a navigation method, based on the 2D LIDAR sensor, allowing the robot to autonomously navigate through the crop rows inside a strawberry greenhouse. As it will be explained later, the proposed autonomous navigation method is divided into two major parts: between and outside the crop rows.

2.3.1.   Autonomous navigation between crop rows

We propose a 2D LIDAR-based navigation method that uses 2D range scans to autonomously drive between the strawberry plants by benefiting from the row-based arrangement of the crops. The robot moves and controls its trajectory between crop rows based on its heading and lateral offset estimated until the detection of the end of the row. For such a task, the LIDAR scans its surroundings in a plane parallel to the ground and uses the measurements to determine the robot localization relative to the rows. This procedure can detect if the robot is navigating correctly (i.e. the robot is positioned in the center of the space between crop rows and moving forward), or it deviates to the left or right with respect to the crop rows, as seen in Figure 5.

The deviation is related to two parameters: the robot's lateral shift from the center of the row and the robot's deviation angle from the center of the row. In each situation, the motion controller generates a decision for the mobile robot, to correct its trajectory or to move straight ahead in case the robot is well positioned between the rows.

Figure 6 shows the scheme of the navigation system where R and L represent the distances to the rows detected by the laser scanner at angles of 0° and 180°, respectively. The width of the robot is represented by W, and C is half the length of the robot. In turn, Cr and Cl are the perpendicular distances between the laser scanner (positioned at the center of the robot) and the right and left rows, respectively. In case the robot deviates to the left, $Cl$ is extracted from the laser scanner data, and $Cr$ is calculated as follows:

With:

Similarly, if the robot deviates to the right, $Cr$ is extracted from the laser scanner data, and $Cl$ is calculated.

The robot's deviation angles $\theta$ to the right and left between crop rows is represented by θ. Finally, $f$ denotes the displacement of the robot center with respect to the middle point between rows (see Figure 6b).

Apart from this, there are other crucial factors that must be estimated during the navigation of the mobile robot, such as the accurate position of the robot limits with respect to the crop rows, to ensure the operation safety. As shown in Figure 6a, these factors have been represented by the following parameters: $k$ denotes the distance corner of the robot to the right row, while $m$ is its analog to the left crop row. The correct estimation of these parameters allows the robot to safely navigate without hitting the crops, and they are computed through these equations depending on the type of deviation the robot is suffering:

Deviation to the left:

Deviation to the right:

Once we detect the position of the robot between the crop rows, the navigation motion controller provides the needed speed and steering values. The strategies for motions were described in detail in ^[17]. To keep the proper robot trajectory between the rows, we follow the scheme shown in Figure 7, which depicts the motion corrections the robot applies when it deviates from the middle of the crops (i.e. the reference position). Thus, the input of the motion control is composed of the lateral and the heading offsets of the robot. Then, the control system outputs three different control actions: i) an angular correction to the left by acting the two right motors, ii) an angular correction to the right by acting the left motors, iii) and a linear velocity by acting the four motors with the same speed.

Finally, in order to increase the safety of the robot in case an emergency occurs, and the system loses control of the actuators, the robot stops moving if it did not get any new command every 0.5 s. The farmers use many daily tools in farms, and they can forget an object in the robot's trajectory. For this reason, during the navigation process within the greenhouse, the "AgriEco Robot" will be coupled with an obstacle detector based on the 2D Lidar/visual sensors embedded on the robot ^[18].

2.3.2.   Autonomous navigation outside crop rows

Once the robot has safely navigated between the strawberry crop rows, when it reaches the end of the rows, the robot must deal with its transition to the next row, which is performed as follows:

The first step is to detect the end of the current row by means of the LIDAR in the front of the robot, which is easily carried out by inspecting the laser readings. Then, we use the odometry system provided by the ZED camera to get an estimation of the total travelled distance from the beginning of the rows and compare it with their actual length in order to ensure that the robot has reached the end of the rows. After that, to enter the next one, the robot performs a sequence of a circular-arc movement followed by a backwards action and a new final circular-arc movement, as shown in Figure 8.

Once the robot is placed at the beginning of the new crop row, the odometry system is reset to start computing the distance travelled for the new row, and the robot continues navigating and spraying pesticides until all the rows are completed.

Finally, the pesticide spraying system operates automatically and simultaneously with the autonomous navigation, with the robot following a stop-and-go strategy. This way, the robot moves a specific distance according to real distance between successive plants (known beforehand), subsequently stopping and spraying the plants located at both sides of the robot. Then, it restarts its navigation and repeats this process until the end of the rows. The robot 2D LIDAR-navigation can encounter certain difficulties, such as LIDAR sensor failure. For this purpose, in order to improve the robustness and the efficiency of our process, we have incorporated a Zed-based verification system that provides the robot's orientation angle as an auxiliary system during the robot navigation.

3.   Experimental results and discussion

In this section we present the experiments performed to evaluate the robustness and performance of the "AgriEco Robot" when operating within a real greenhouse, shown in Figure 4, which has been built at the Faculty of Science in Rabat, Morocco, (GPS coordinates: 34.008287475248935, -6.838260257670796) and has dimensions of 5 m x 9 m x 2.5 m (width, length and height, respectively). The greenhouse consists of four rows with an inter-row space of 70 cm and an aisle length of 500 cm. Each row has a height of 25 cm and includes 11 plants. The experiment consisted of automatically navigating between the two rows in the right of the image while spraying pesticides, detecting the end of the rows, changing to the next aisle and continuing navigating and spraying. A calibration was made beforehand for the 2D LIDAR by using two perpendicular walls to ensure that the sensor was properly positioned in the middle of the robot. In order to assess the performance of the robot navigation, we computed the lateral error (i.e. the deviation of the following trajectory from the center of the aisle) as well as the heading error of the robot (see Figure 9 and Figure 10).

As can be seen in the figure, the lateral error remains bounded by a few centimeters during the whole experiment, presenting a root mean square (RMS) value of 2.99 cm, while, in turn, the heading RMS error is 3.27°. In this experiment, the robot navigated at a speed of 0.44 m/s (which can be considered a safe speed for indoor operation) through a trajectory of approximately 5 m. Finally, it is important to note that in the actual field, the rows were not symmetrically aligned, but the calculation of the distance between rows in real time allowed the robot to reactively adjust its navigation to the actual row alignments.

4.   Conclusions

This work presented an autonomous navigation method for an agricultural robot based on a 2D LIDAR sensor and a motion control. Our system is able to automatically estimate the robot's lateral deviation with respect to the center of the crops aisles as well as its heading. Control actions are then sent to the robot motors in order to correct them. The entire system has been implemented in ROS, which has facilitated the integration and communication of the software and the hardware of our robot. The autonomous navigation method has been evaluated in a real-world scenario with a robot spraying pesticides within strawberry greenhouses, proving its usefulness and performance for agricultural application. As future work, we will focus on improving the spraying procedure by developing an automatic and targeted pesticide spraying system. For this, we intend to use an image processing approach based on the omnidirectional vision sensor embedded on the robot to automatically recognize crops and servo the sprayer system's motor arms.

Acknowledgments

The authors of this paper are thankful to the Minister of Higher Education, Research and Innovation (MHERI) and the National Center for Scientific and Technical Research of Morocco (CNRST) for financing this project.

Conflict of interest

The authors declare that there is no conflict of interest in this paper.