Validity and reproducibility of food photographic estimation for evaluating meals in evacuation shelters

1.   Introduction

Recently, the number of natural disasters such as earthquakes and torrential rains has been increasing in Japan [1]. In comparison with other developed countries, Japan has higher risk for natural disasters [2]. In affected local governments, dietary assessments in evacuation shelters provide a basis to secure food and maintain the health of the evacuees [3]. In this context, following the heavy rain disaster that happened in Kumamoto in July 2020, the Kumamoto Prefectural Government asked local and dispatched dietitians to collect weighed food record (WFR) with food photographs as supplementary information. However, WFR was time consuming for the dietitians even though it provides the most accurate dietary information [4]. A simpler and time- and manpower-saving method would be desirable for dietary assessment in emergent cases.

The validity and reproducibility of food photographic estimation have been evaluated in laboratory, cafeteria and free-living settings targeting all range of ages from infants to the elderly, using digital cameras at first, back in the 1990s, then more into advanced equipment, such as personal digital assistants and smartphones [5].

Contrary to free-living settings where a wide variety of food items are available [6], in evacuation shelters, the same meals are uniformly provided, and the menu is static and repeated every day [7]. These features in shelter meals might facilitate dietitians' portion size estimation due to the limited variety of food photographs.

This study evaluated the validity and reproducibility of food photographic estimation of shelter meals via WFR, collected at evacuation shelters in Kumamoto Prefecture in 2020. Whenever validity and reproducibility are ensured, anyone even without expertise can take food photographs and assist onsite dietitians in having more time to provide individual nutritional guidance for evacuees who need special care.

2.   Materials and methods

2.1.   Weighed food record with food photographs

After two weeks of heavy rain in July 2020, WFR was conducted to assess meals in evacuation shelters as a part of public services provided by the Kumamoto Prefectural Government in Japan. Therefore, statistical sampling techniques were not employed. The affected municipal governments selected three municipalities with numerous evacuees. At 12 shelters in the three municipalities, dietitians recorded all meals served daily for 2 or 3 consecutive days and collected data from 96 meals in total.

Food consumption data was not collected because of the following two reasons. Firstly, it was difficult to assess individual consumption in chaotic emergency shelters. It was considered unethical to collect dietary intake data from a significant number of evacuees who needed a rest to recover from substantial physical and psychological damages caused by the disaster. Secondly, in food service facilities for mass feeding such as schools and hospitals, nutrition management and evaluation by health authorities was based on what meals they served rather than how much individual consumers ate.

Dietitians at each shelter filled WFR following the form directions; they weighed each cooked food dividing it into its ingredients using digital cooking scales. For example, a rice ball was weighed after dividing it into rice, fillings and laver. Noting that, condiments were recorded for only 9 out of 96 meals. Dietitians placed meals on a sheet of A4 paper (ISO 216:2007) and took photographs from a straight-above view. Since standard measurement tools such as a ruler were not necessarily available in shelters, A4 paper was used as a standard scale because of its definite shape and availability.

2.2.   Nutrition calculation

We calculated the energy, proteins, vitamins B₁, B₂, and C, and salt contents in the 96 meals. We selected energy, protein, and the three water soluble vitamins since Japan's Ministry of Health, Labour and Welfare set “Nutritional Reference Values to be used as Near-term Targets for Planning and Assessment of Meal Provision in Evacuation Shelters” for them because they are important in times of disaster [8]. The three micronutrients were selected since they have a short storage period in the body and their deficiencies were common in shelters. On the other hand, some minerals like calcium and iron and vitamin B₁₂ have low priority within three months after disaster, and therefore, their values were not set for this period. We used salt instead of sodium for practical purpose since the Nutritional Reference Values, Dietary Reference Intakes for Japanese, the Standard Food Composition Tables [9], nutrition facts, and national health promotion plans in Japan consistently use salt.

Nutrition calculation was conducted by Excel add-in software (Excel Eiyo Plus; KENPAKUSHA Co., Ltd.), which used the newest 8^th edition of the Standard Food Composition Tables in Japan [9]. Regarding the meals for which the weight of condiments was not recorded, we used the food composition tables for cooked and processed foods or the supplemental recipe data in Excel Eiyo Plus, to determine the condiments weights. If the typical standard recipe of the meal could not be found in either method, we excluded the meal from the analysis. Whenever energy and nutrient contents were shown on the food label or manufacturer's website, these values were used for the analyses instead of the calculated ones.

2.3.   Selection of food photograph

Seven meals were not photographed in error and food photos were available for 89 out of 96 meals. These 89 photos were screened according to four exclusion criteria: 1) photos with incomplete WFR (n = 21), 2) duplication of the same food (n = 13), 3) foods not listed in the food composition tables or Excel Eiyo Plus (n = 19), 4) beverage only (n = 1). Therefore, a total of 54 photos were excluded, and the remaining 35 photos were used for the analyses. Out of the 35 meals, 18 were food aid, 13 boxed meals and 4 were hot meal service. Figure 1 shows an example of boxed meal. The 35 meals consisted of 196 food items in total.

2.4.   Portion size estimation by food photograph

We sent by email a form to 35 senior students at a women's university in Tokyo, asking for their participation in this study. They had nearly completed their educational program for registered dietitian when this study was conducted.

Overall, 24 students answered the form (response rate was 74%) and 21 of them agreed to participate in the portion size estimation by food photograph. They were not provided any training of estimation for this study but they were given a written instruction on how to estimate food weights. The written instruction included the following four tasks: 1) see photographs of meals that were placed on a sheet of A4 paper, 2) describe the names of foods in the photographs since foods were not labelled, 3) list up ingredients included in the food by freely referring to any media, and 4) estimate weight of each ingredient. They were given a reward of 3060 JPY after completing all estimation and a postestimation questionnaire.

Ingredients and weights of foods included in each meal photograph were estimated by 3 students, and 105 estimation data of 35 meal photographs were collected. In this present nutritional assessment, meal photographs were taken at the evacuation shelters and sent to research institutes that remotely backed up the affected prefecture as volunteers. Similarly, our participants received meal photographs via email. After they completed the portion size estimation of five meals randomly assigned, they sent us back by email their estimations in Excel sheet. In addition, they were allowed to freely access to any information from books and online during the estimation analysis. Based on their portion size estimation record, we calculated energy and nutrients of each meal using Excel Eiyo Plus.

2.5.   Postestimation questionnaire

We sent a postestimation questionnaire via a Google form to all participants by email and they sent back their answers to us. We asked for the frequency of cooking through a multiple-choice question (every day, 3–5 days/week, 1–2 days/week, never), in addition to an open-ended question regarding their impression toward estimation (easiness, difficulty and ideas for improvement in estimation).

2.6.   Statistical analyses

We used total meal weight as a direct indicator to show the difference between WFR and food photographic estimation. For to what extent food photographic estimation method could assess food intake and meal composition ingredients, we used energy and nutrient intakes and each food weight that consisted of the meal. To assess the validity of food photographic estimations, we applied three statistical analyses. In the first analysis, the total grams of meals and their contents of energy and nutrients already calculated by WFR and food photograph estimation were compared by Wilcoxon signed-rank test since they were not normally distributed according to Shapiro–Wilk test. In the second analysis, the total grams of meals and their calculated contents were divided into quartiles. Joint classification was used to examine agreement between the two methods by calculating the number of meals in the same, adjacent, and extreme quartiles. In the third analysis, we compared the difference in portion size of each food item between the two means of provision: served by weight or by piece.

To assess the reproducibility of food photographic estimation, coefficients of between-estimator variation of total grams of meal were calculated. They were also grouped by meal categories using Kruskal–Wallis test.

To examine the association between participants' cooking experience and accuracy of estimation, the total grams of meals derived from the two methods (food photographic estimation and WFR) were compared between the ones who cooked and those who did not cook by Mann–Whitney U test.

All statistical analyses were performed using IBM SPSS Statistics 27, with the significance level at 0.05.

2.7.   Ethics approval of research

To use the WFR and photographs for this study, we obtained written permission for secondary use from the Kumamoto Prefectural Government (permission notice/health promotion department no. 885) on one condition; the names of shelters and their located municipalities were not to be revealed. The study protocol was approved by the ethics committee of Ochanomizu University (2021–53), and all participants gave informed consent.

3.   Results

3.1.   Validity

The differences between WFR and food photographic estimation were not statistically significant for total grams of the meals and their contents in energy and nutrients except for the salt that was overestimated by food photographs (Table 1).

Table 2 shows the degree of agreement in total grams of meals between WFR and food photographic estimation. Out of the 35 meals, 22 were classified in the same quartiles, and no meal was detected in extreme quartiles. Table 3 shows the results of the same analysis for energy and nutrients. The only meal that was classified in the extreme quartiles for energy, vitamins B₁ and B₂ and salt was the same meal shown in Figure 1. This meal box was underestimated by food photograph since it was classified into the 4^th and 1^st quartiles by WFR and food photographic estimation, respectively. Figure 1 shows a small and coarse image size; therefore, all the three participants mistook the ingredients of the simmered food shown in its upper left.

Table 4 shows the difference in portion size of each food item between WFR and food photographic estimation. For staple foods in food aid, ratios of pictures with legible nutrition facts were 4/12 for rice ball, 0/5 for sandwich, 3/4 for jam bun and 3/4 for melon bun. The differences tended to be smaller for the foods that have more pictures of nutrition facts.

Main or side dishes served by weight tended to be more overestimated than those served by piece, but the difference between the two serving methods was not statistically significant (Mann–Whitney U test, p = 0.452). Regarding the Japanese pickles, umeboshi was overestimated about two times compared to other pickles.

3.2.   Reproducibility

Figure 2 shows the coefficient of between-estimator variation for the total grams of the same meal by meal categories. Their medians [interquartile range] were as follows: 0.16 [0.07–0.25] for food aid, 0.13 [0.06–0.16] for boxed meals and 0.15 [0.07–0.22] for hot meal service. The variation was larger for food aid and hot meal service, whereas the difference between the three meal categories was not statistically significant.

3.3.   Time required for estimation

The number of food items included in the 35 meals ranged from 1 (food aid) to 16 (boxed meal), and therefore, the time required for estimation also varied widely among participants; some took only 3 minutes to complete, and others did 1.5 h.

3.4.   Participants' cooking experience and estimation

The responses to the postestimation questionnaire are summarized in Table 5. According to their self-reporting frequency of cooking, we divided the participants into two groups: the ones who cooked (1–7 days/week; n = 16) and those who did not (never; n = 5). When total grams of meals derived from the two methods (food photographic estimation and WFR) were compared between the two groups, the median values of the ratio [interquartile range] were as follows: 1.06 [0.98–1.26] for those who cooked and 0.93 [0.69–1.03] for those who did not cook. Participants who never cooked for themselves significantly underestimated (Mann–Whitney U test, p = 0.016). Among the former, those who cooked every day (n = 3) estimated more accurately (1.04 [0.99–1.19]).

3.5.   Mistakes and oversights in estimation

Twenty-one food items were totally different from what the participants selected to be the actual ones. Especially six of them were deep fried foods with similar appearances (it was hard to tell what was inside the batter). Three of them were cases in which participants mistook red-fleshed fish with white-meat fish because the picture showed the skin-side of the fish.

Twenty-eight food items were in the picture but remained not estimated; nine were soy sauce in small container and four were cabbage used for raising bottoms. The picture of both container and cabbage was blurry because they were underneath other foods.

4.   Discussion

4.1.   Validity

This study first evaluated the food photographic method in evacuation shelters. The results suggested that the food photographic method could provide good estimates of the total grams (g), energy (kcal) and five nutrients compared with WFR. A comparison between the two methods demonstrated a fair agreement for total grams (g), energy (kcal), proteins (g), vitamins B₁, B₂ and C (mg) (Table 1). Only a significant difference between the two methods regarding salt (g) might be due to condiments and umeboshi (salt pickled plum). The salt content in one umeboshi is 7.6%–18.2% [9], and it was overestimated about two times by food photographic method. The percentage of under-estimations for both energy and salt and the percentage of under-estimations for both vitamin B₁ and B₂ were consistent with each other. Considering that energy and salt were taken from the meal as a whole and vitamins were taken especially from side dishes, it was likely that estimation of side dishes was particularly difficult among the food items.

The estimation of the portion sizes of the food items served by weight appeared to be more difficult than those served by piece (Table 4). Unlike foods such as Chinese dumpling and cherry tomato, whose portion size is usually shown as the number of pieces, the amount of food served by weight varied depending on how it was arranged in the box. For better estimation, one participant suggested a standard portion size model (Table 5). A previous study also reported that increasing the number of reference photographs from 3 to 8 increased estimation accuracy [10]. Furthermore, the development of a food photobook of standard portion size could aid estimation.

The main challenge was the blurriness of some photographs (Figure 1). Participants underestimated the portion sizes and could not report some food items inside the meal box especially fried foods (Table 4), which may cause underestimation of energy (kcal) and three of the nutrients; vitamins B₁ and B₂ (mg), and salt (g) (Table 3). Most previous studies specified the distance and angle to food in details while taking photographs [11],[12]. In fact, one of these studies reported the importance of complying the angle of photographs for better estimation of the amounts of some nutrients [13]. The angle was not fixed while taking photographs in the present study, which may have affected the lower accuracy of estimating portion sizes. When using photographs taken from a specified angle such as 45° rather than directly above, the height and the bottom area of boxes can be shown as an indicator to estimate the portion size, which may lead to more accurate estimates. In addition, when elaborating more information about food items, in particular its ingredients, participants would be less likely to mistake or miss food items in the photograph.

4.2.   Reproducibility

Between-estimator variation was larger for food aid and hot meal service than for boxed meal (Figure 2). According to a previous study that calculated coefficients of variation for vegetable weight estimates, the values varied widely among vegetables [14]. This may be due to the estimation of soups that were not served in boxed meals. When estimating instant soups in food aid, some participants reported their dry weight before cooking, and others reported their weight cooked with hot water. For the photographs of the soups from hot meal service, it was difficult to know their content because the ingredients were submerged.

4.3.   Participants' cooking experience and estimation

Participants who never self-cooked tended to underestimate compared to those who did, possibly because they have learned some standard portion sizes through cooking regularly. This result was consistent with previous studies [15],[16]. In food photographic estimation, accuracy might be improved if those who cook regularly are recruited through preliminary interviews.

4.4.   Limitations and strengths

The following five points are the limitations of this study.

First, while various image processing techniques for food quality evaluation are already known [17], this study is a human-performed portion size estimation using food photograph. These methods are designed for food in normal eating conditions and may differ for food served in evacuation centers, where people store meals at room temperature and use limited heat sources. Because human-performed portion size estimation is still being conducted at disaster sites, we examined the possibility of using photographs to reduce the burden of filling out the meal survey form. This study using food photographs can be applied to meals served in shelters; however, it will be necessary to further investigate image processing techniques in the future.

Second, statistical sampling techniques were not employed since WFR was conducted as a part of public services. Therefore, the 12 shelters could not represent the whole shelters in the affected areas. The 35 meals used in this study were not necessarily the typical ones served in evacuation shelters because the dietary data were collected in one prefecture during a heavy rain disaster. Compared with recent disasters in Japan, hot meal service was limited due to hygienic concerns during the COVID-19 pandemic [18].

Third, among the 35 meals used as standards, complete records of weights of all ingredients were available for only nine meals. Many standard WFR values that were estimated based on standard recipes from the Standard Tables of Food Composition or Excel Eiyo Plus did not exactly agree with the values given on the nutrition facts labels.

Fourth, the angle and quality of photographs taken at shelters were not uniform, which may have reduced the accuracy of estimating portion sizes.

Fifth, all the participants were senior students in a dietitian training course at the same women's university. Their homogeneity in year in university, educational background, and sex could eliminate potential factors that could affect accuracy in estimation.

5.   Conclusions

Although photographs have been used as an auxiliary tool, rather than a standalone one, combined with WFR and dietary recalls, food photographic estimations have the potential to be used as a rapid method for data collection and nutritional assessment in evacuation shelters. This study also showed some measures to improve its accuracy.