Training data collection and development of the artificial intelligence model
We have improved pre -described artificial intelligence model14 Through training with more than 30,000 shiny parts of LCT fibers, which were taken from 60 video surgeries from endoscopic surgeries conducted in multiple Japanese hospitals from 2018 to 2022. These surgeries included 26 infectious surgeries, and 15 rectum surgical operations And 16 reforms of the hernia. We used the Incubit Inc. web application. (Tokyo, Japan) for the illustrative comments. After reviewing the training data used in our previous study14LCT fiber explained more accurately and uniform. Seven surgeons on the digestive system and explained the trained under the supervision of the surgeons carefully explained the LCT areas in the photos to create the training data. We set the artificial intelligence model based on U -Net and Deeplab V3. The artificial intelligence model was then developed to automatically divide the LCT fibers.
External evaluation committee and test data set
To ensure fairness in the evaluation process, an independent external committee has been created. The committee was composed of the surgicals of the external digestive system and the contract research organization that communicated with external surgeons and supervised the study. The external committee chose video and pictures scenes for evaluation based on the criteria in this study.
To create a test data set, 10 stomach surgeries, 10 portns are chosen, and 5 ingredients randomly from a set of data from surgeries captured by two laparoscopic imaging systems®Olympus Inc. , Tokyo, Japan and Da Vinci XI surgical system®Intuity Surgan Inc. , Sunnyvale, CA) during surgeries conducted in multiple hospitals in Japan in 2022. These test data were not included in the training data. From each of the 25 video clip, two frames depicted clearly LCT were extracted, resulting in a total of 50 fixed photos. We included the evaluation images that clearly showed the presence of LCT in the middle of the photos. On the contrary, we excluded the images in which the LCT was not clearly visible due to bleeding, smoking or artifacts, which have deteriorated LCT due to inflammation or previous treatment. In addition, we excluded the evaluation scenes in which the surgery did not apply smoothly.
Ways to explain the surgeon
Explanation of the conditions for the surgeon is a clearly defined image of the correct area of LCT through manual illustrations, which reflects the recognition of the surgeon.
External surgeons distinguished the outline of the LCT area on raw images without confirming AI’s prediction images. The research team was drawn up to the LCT fiber area according to the signs of surgeons. After the plate, the external surgeon examines the explained area. If the external surgeon rules to review the explanatory comments to be necessary, the research team re -painted it. This process was repeated so that there are no other reviews. After an external surgeon decided to explanatory comments, another surgeon examined him in the same way, and repeated the same process. Explain the conditions for the surgeon was completed through manual explanation and with the approval of external surgeons.
Evaluation methods to perform the artificial intelligence model
Then the performance of the artificial intelligence model was evaluated by comparing the raw image, the image of AI, and the surgeon explained using both the following four methods (Figure 1).
Four evaluation methods of the artificial intelligence model.
The quantitative arithmetic evaluation of fixed images
Explanation of the conditions for the surgeon was determined by a consensus of three surgeons in the external evaluation committee. The similarity between AI’s prediction image was evaluated and the surgeon’s conditions are explained by the dice coefficient, which is a wide range used in retail medical images analysis19. The value of the cut is set 0.5. The recall was used as an LCT detection indicator. Formulas to calculate the blossom and recall coefficient are as follows, with higher values indicating better results.
Dice = \ (\: \ frac {tp} {tp+frac {1} {2} (FP+Fn)} \)Call = \ (\: \ frac {tp} {tp} \)
Here, a real positive TP, a false positive FP, and FN is a wrong negative.
The visual evaluation is a comparison between raw prediction images and AI
To help assess the agreement accurately between recognizing surgeons and predicting artificial intelligence, 10 visual external surgeons have compared primary images with AI prediction images to determine the performance of the prosecution. This approach was taken due to the complexity of identifying and interpreting dice transactions as a measure of agreement in clinical preparation. The surgeons who did not participate in the illustrations process conducted this visual evaluation. They evaluated the compatibility rate of artificial intelligence predictions and the nature of excessive detection of artificial intelligence in the same 50 fixed images used in the arithmetic evaluation.
Surgeons compared pairs of corresponding images that consist of the image of two raw image and the AI prediction image and answered the following questions on a scale of 5 points: for the degree of artificial intelligence compatibility? “As for excessive discovery of artificial intelligence,” artificial intelligence recognizes structures like LCT that you define as not LCT, that is, better describes the nature of this wrong positivity? “((A wrong but little positive effect; false positive but there is no effect on Surgical or procedure; wrongly positive effect on surgical judgment and procedure;
The visual evaluation compares the images of the surgeon and the prediction of Amnesty International
To compare the prediction of artificial intelligence with surgeons recognition, an additional evaluation was made with both AI prediction image and the surgeon explained. Two groups of manual illustrations were created, one by the three external surgeons, the corresponding to the surgeon’s explanation, and one by our research team independently. The ten external surgeons on the evaluation committee chose the photos that were the closest and most farthest of their own confession while blinded to any of the three photos, the AI prediction image was. Half 50 pictures were evaluated, after excluding clear, clear positives that can be easily identified as Amnesty International’s expectations.
Video scenes visual evaluation
In the preparation of the censorship laboratory, ten external surgeons were presented with two parallel screens, one of which shows the raw video and the other shows the video of artificial intelligence. They were asked to evaluate the effect of artificial intelligence analysis in an actual time on perception and at stress levels.
Test videos consist of 10 cases that were randomly chosen out of a total of 25, each of 30 seconds. The surgeons carefully reviewed the RAW and AI videos on the screens and answered the following questions on a 5 -point scale: for artificial intelligence ability to support recognition, “Are you looking to show artificial intelligence analysis that makes it easy to recognize the LCT areas?”; As for the levels of stress resulting from the display of artificial prediction, “Is it possible that you will cause artificial intelligence analysis any stress due to the wrong positives, wrong negatives or overlapping on the screen?”
The total flow of roads appears in Figure 1.