WO2019045101A1 - Image processing device and program - Google Patents

Abstract

Accepts cartoon image data, and uses information obtained by machine learning to identify each of the top, face, body, and text based on the cartoon image data, and identifies information for identifying the top and the face. It is an image processing apparatus in which the information to be identified, the information to identify the body part, and the information to identify the character part are respectively generated, and the information is provided to predetermined information processing.

Description

Image processing apparatus and program

The present invention relates to an image processing apparatus and program.

In recent years, the technology of processing the image data of a cartoon, extracting the speech part and translating it for another language, and changing the arrangement for each frame to make it suitable for the screen of a smartphone etc. can be considered. ing.

In order to perform such processing, it has been conventionally considered to specify processing of a drawn person part and a serif part using the result of color and character recognition processing and the like (Non-Patent Document 1).

Christophe Rigaud, et. Al., Speech ballon and speaker association for comics and manga understanding., Proceedings of the 13th International Conference on Document Analysis and Recognition, pp. 351-355, IEEE, 2015

On the other hand, in recent years, a technique for detecting an object from an image by machine learning has been developed and widely studied. However, in the conventional general object detection processing, detection is performed on the premise that a specific part in an image includes one object, so it is considered in the case where multiple detection targets overlap each other. Not.

However, in cartoon image data, it is common for a person's body or face to be drawn inside a frame (or across multiple frames), and a serif character is placed overlapping these parts. is there. Therefore, if the object detection processing by machine learning is applied as it is, parts such as the top, the body of the character, the face, and the speech can not be detected with sufficient accuracy.

The present invention has been made in view of the above-mentioned circumstances, and in the processing of cartoon image data, the machine learning processing is used to compare the recognition accuracy of the top, face, body and character parts with the conventional one. An object of the present invention is to provide an image processing apparatus and program that can be improved.

The present invention for solving the problems of the above-mentioned conventional example is an image processing apparatus, which comprises an accepting means for receiving cartoon image data, and detecting a frame portion of a cartoon drawn in the image data from the image data. Such as frame detection means in the machine-learned state, face detection means in the state of machine-learned to detect the face portion drawn in the image data from the image data, and the image data from the image data Body detecting means in a state of machine learning to detect a body part drawn in, and character detecting means in a state of machine learning to detect a character part included in the image data from the image data And information for identifying a frame portion detected by the frame detection means based on the accepted cartoon image data, and a face portion detected by the face detection means. Detection information generation means for generating information for identifying the body part detected by the body detection means, and information for identifying the character part detected by the character detection means; The information specifying the top part, the information specifying the face part, the information specifying the body part, and the information specifying the character part are to be subjected to predetermined information processing.

According to the present invention, it is possible to improve the recognition accuracy when recognizing a top, a face, a body, and a character from cartoon image data using machine learning processing as compared with the conventional one.

It is a block diagram showing the example of composition of the image processing device concerning an embodiment of the invention. It is a functional block diagram showing the example of the image processing device concerning an embodiment of the invention. It is an explanatory view showing an outline example of comics image data which the image processing device concerning an embodiment of the present invention makes an object of processing. It is an internal functional block diagram showing the outline example of the detection processing part of the image processing device concerning an embodiment of the invention. It is an internal functional block diagram showing the other example of the detection process part of the image processing apparatus which concerns on embodiment of this invention. FIG. 7 is an explanatory diagram showing a schematic example of processing according to another example of the detection processing unit of the image processing apparatus according to the embodiment of the present invention. It is a functional block diagram showing the example of composition of the detection processing part of the image processing device concerning an embodiment of the invention.

Embodiments of the present invention will be described with reference to the drawings. The image processing apparatus 1 according to the embodiment of the present invention includes the control unit 11, the storage unit 12, the operation unit 13, the display unit 14, and the input / output unit 15, as illustrated in FIG. It is configured.

The control unit 11 is a program control device such as a CPU, and executes a program stored in the storage unit 12 to receive cartoon image data, and information for specifying a top part based on the received cartoon image data and , Information for identifying the face part, information for identifying the body part, and information for identifying the text part. The control unit 11 according to the present embodiment is a frame detector in a state of being machine-learned to detect a comic piece drawn in the image data from the image data in the process of identifying each of these parts. And a face detector in a state of being machine-learned to detect a face part drawn in the image data from the image data, and a body part drawn in the image data from the image data A body detector in a machine-learned state and a character detector in a machine-learned state to detect character portions included in the image data from the image data are used.

In addition, the control unit 11 executes predetermined information processing using the generated information for identifying the frame part, the information for identifying the face part, the information for identifying the body part, and the information for identifying the character part Do. This information processing includes, for example, processing for outputting an image representing each part, optical character recognition processing for a character string within a range specified by information specifying a character part, and division for dividing image data for each frame part There is processing etc. The operation of these control units 11 will be described in detail later.

The storage unit 12 is a memory device or the like, and holds a program executed by the control unit 11. This program may be stored in a computer readable non-transitory recording medium and provided, and may be stored in the storage unit 12. The storage unit 12 also operates as a work memory of the control unit 11.

The operation unit 13 is a mouse, a keyboard, or the like, receives an instruction operation of the user, and outputs the operation to the control unit 11. The display unit 14 is, for example, a display or the like, and displays and outputs information based on an instruction input from the control unit 11.

The input / output unit 15 is, for example, a network interface or the like, receives data (image data and the like) from the outside, and outputs the data to the control unit 11. The input / output unit 15 also sends data to an external device or the like according to an instruction input from the control unit 11.

Next, the operation of the control unit 11 will be described. The control unit 11 according to the present embodiment executes the program stored in the storage unit 12 to functionally receive the reception unit 21, the detection processing unit 22, and the detection information as illustrated in FIG. 2. The configuration includes a generation unit 23 and an information processing unit 24. The detection processing unit 22 also includes a frame detection unit 31, a face detection unit 32, a body detection unit 33, and a character detection unit 34.

The receiving unit 21 receives cartoon image data and outputs the data to the detection processing unit 22. Here, the cartoon image data is generally image data drawn by overlapping the face portion (F), the body portion (B) and the character portion (C) with each other (FIG. 3), and at least one frame (M) is included. Further, in order to use the neural network in the detection processing unit 22, the receiving unit 21 enlarges or reduces the comic image data and resizes it to a size suitable for the input of the neural network.

The frame detection unit 31 of the detection processing unit 22 has a frame detector in a state of being machine-learned to detect a comic piece drawn in the image data from the image data. Specifically, the frame detector included in the frame detection unit 31 is R-CNN (Regions with CNN features) (Girshick, Ross, et al. "Rich feature hierarchies for accurate object detection and semantic segmentation." Proceedings of the IEEE conference on computer vision and pattern recognition. 2014), Fast R-CNN (Girshick, Ross. "Fast r- cnn." Proceedings of the IEEE International Conference on Computer Vision. 2015), Faster R-CNN (Ren, Shaoqing, et al. "Faster R-CNN: Real-time object detection with region proposal networks." Advances in neural information processing systems. 2015), YOLO (You Only Look Once) (Redmon, Joseph, et al. "You only look once: Unified, real-time object detection. "arXiv preprint arXiv: 1506.02640 (2015) or SSD (single shot multi-box detector; Sing It can be realized by adopting a neural network configured by various methods such as le Shot MultiBox Detector) (Liu, Wei, et al. "SSD: Single Shot MultiBox Detector." arXiv preprint arXiv: 1512.02325 (2015)).

As schematically shown in FIG. 4, a detector 40 employing a neural network such as an SSD is configured to include a base network unit 41 and a classifier 42. Here, the base network unit 41 outputs the range of the image including the candidate to be detected and the feature amount of the image within the range. In addition, the classifier 42 is based on the output feature amount whether or not the detection target (in the case of the frame detection unit 31, a frame line dividing frames of comic image data) is included in the range of the output image. To judge.

The detector 40 employing such an SSD or the like artificially designates the range to be detected (in the case of the frame detection unit 31, the range of the shape circumscribed to the frame line dividing the frame of the comic image data) artificially Machine learning using the sample of. Here, since a specific method of machine learning and a method of using the detector 40 are widely known, the detailed description thereof is omitted here.

The face detection unit 32 has a face detector in a state of being machine-learned to detect a face portion of a character drawn in the image data from the image data. This face detector can also be realized by employing a neural network configured by various methods such as an SSD, as with the frame detector included in the frame detection unit 31. The face detector performs machine learning using a sample of image data in which a range of a predetermined shape circumscribing the face of a character included in cartoon image data, which is a range of a detection target, is artificially specified.

The body detection unit 33 has a body detector in a state of being machine-learned to detect the body part of the character drawn in the image data from the image data. This body detector can also be realized by employing a neural network configured by various methods such as an SSD, as with the frame detector included in the frame detection unit 31. The body detector performs machine learning using a sample of image data artificially designating a range of a predetermined shape circumscribing the body of a character included in cartoon image data, which is a range to be detected.

The character detection unit 34 has a character detector in a state of being machine-learned to detect a character portion drawn in the image data from the image data. This character detector can also be realized by employing a neural network configured by various methods such as an SSD, as with the frame detector included in the frame detection unit 31. The character detector performs machine learning using a sample of image data in which a range of a predetermined shape circumscribed to a character portion included in cartoon image data, which is a range to be detected, is artificially specified.

The detection information generation unit 23 includes, for the cartoon image data received by the reception unit 21, information for identifying a top portion detected by the frame detection unit 31, information for identifying a face portion detected by the face detection unit 32, and a body The information which specifies the body part which the detection part 33 detected, and the information which specifies the character part which the character detection part 34 detected are generated.

The information processing unit 24 executes predetermined information processing using the generated information for identifying the top part, the information for identifying the face part, the information for identifying the body part, and the information for identifying the text part. . This information processing includes, for example, processing for performing optical character recognition (OCR) on an image of a specified character portion and outputting the result. In addition, the information processing unit 24 may translate and output a character string obtained as a result of optical character recognition into another language by machine translation processing.

An example of the present embodiment has the above configuration and operates as follows. In the following description, the frame detection unit 31, the face detection unit 32, the body detection unit 33, and the character detection unit 34 by the control unit 11 adopt an SSD, and each of the image data in advance and the image data It is assumed that machine learning has been performed so as to detect a comic piece, a face, a body, and a character part of the drawn cartoon.

The image processing apparatus 1 sets a frame detector in parallel to image data of the processing target, with the image data of a cartoon (not included in the machine learning sample) input from the user as the processing target. A face detector, a body detector, and a character detector respectively detect a top part, a face part of a character, a body part, and a character part, and obtain information for specifying the range of the detected image.

Then, the image processing apparatus 1 performs predetermined information processing, for example, identification using information identifying the top part, information identifying the face part, information identifying the body part, and information identifying the character part. Optical character recognition (OCR) is performed on the image of the character part, and a character string obtained as a result of the optical character recognition is translated into another language by machine translation processing and output.

[Example of sharing base network]
In the description so far, the frame detection unit 31, the face detection unit 32, the body detection unit 33, and the character detection unit 34 each include a base network and a detector that are independent of each other. It is not limited to this example. For example, the frame detection unit 31, the face detection unit 32, the body detection unit 33, and the character detection unit 34 may share one base network.

That is, in this example, the frame detection unit 31, the face detection unit 32, the body detection unit 33, and the character detection unit 34 are, as illustrated in FIG. And the feature amount of the image within the range, which is machine-learned, and based on the image data to be processed, the range of the image to be the candidate of the detection target and the feature amount of the image within the range Classifiers 42a, 42b, 42c, 42d provided independently corresponding to the base network unit 41 'for output, the frame detection unit 31, the face detection unit 32, the body detection unit 33, and the character detection unit 34, respectively. And

Also in this example, the base network unit 41 'and the classifiers 42a, b, c, and d may be neural networks based on SSDs, but the SSDs are modified and used in the following points. That is, in the general output stage of the SSD, a plurality of candidate areas (anchor boxes) for detecting an object are determined in advance (a set of a plurality of anchor boxes is called an anchor set), and among the plurality of candidate areas From this, identify the area in which the object of interest is included.

In this example of the present embodiment, there is one network (base network unit 41 ') before the output stage, but at the output stage, anchor sets (for example, 8732 anchor boxes are provided in each anchor set) Corresponding to each of the frame detection unit 31, the face detection unit 32, the body detection unit 33, and the character detection unit 34, four copies are used as classifiers 42a, b, c, and d. That is, in the SSD in this example, with respect to each anchor box, position specification error with respect to the area of the object detected in the anchor box (four-dimensional information including information of upper left coordinates and information of width and height) Outputs a total of five-dimensional information with the degree of certainty that can be included (here normalized by the sigmoid function), but the first of the anchor boxes in the same four anchor sets replicated by the anchor box The anchor box in the first anchor set A1 is in a state in which the top portion of the image data is machine-learned. Further, for the anchor box in the second anchor set A2, the face portion of the image data is machine-learned, and for the anchor box in the third anchor set A3, the body portion of the image data is machine-learned The anchor box in the fourth anchor set A4 is in a state in which the character portion of the image data is machine-learned (FIG. 6).

Specifically, for the information on the anchor box in the first anchor set output by the output stage, a rectangle circumscribing the frame of the frame is estimated when the learning sample is input. , Backpropagating the error from the output stage to update the parameters of the classifier 42a and the base network section 41 '.

Similarly, regarding the information of the anchor box in the second anchor set output by the output stage, an output is performed so that a rectangle circumscribing the face portion of the character is estimated when the learning sample is input. The error is back-propagated from the stage to update the parameters of the classifier 42b and the base network section 41 '. In addition, regarding the information on the anchor box in the third anchor set output by the output stage, an error is generated from the output stage so that a rectangle circumscribed to the character's body is estimated when a sample for learning is input. Are back-propagated to update the parameters of the classifier 42c and the base network unit 41 '. Furthermore, regarding the information of the anchor box in the fourth anchor set output by the output stage, the error is reversed from the output stage so that the rectangle circumscribing the character is estimated when the sample for learning is input. It propagates and updates the parameters of the classifier 42d and the base network unit 41 '.

ここで、ｇは１以上、Ｇ（ｍ）以下の整数であり、Ｇ（ｍ）は、上記ｍ番目のサンプルに含まれる正解の個数であり、ｔ（ｍ，ｇ）、及びＢ（ｍ，ｇ）は上記ｍ番目のサンプルのｇ番目の正解のクラス（コマ、顔、身体、文字のいずれであるかを表す情報）と、外接矩形とを表す。 It is to be noted that, for the a-th anchor box (a = 1, 2,... 8732) in the i-th (i = 1, 2, 3, 4) anchor set, the m-th sample (mini batch learning is performed as a mini Assuming that the batch size is M, the assignment s (m, i, a) of m = 1, 2,..., M) and the overlap J (m, i, a) thereof are defined as follows.

Here, g is an integer of 1 or more and G (m) or less, and G (m) is the number of correct answers included in the m-th sample, and t (m, g) and B (m, g) g) represents a g-th correct class (information indicating which is a top, a face, a body, or a character) of the m-th sample and a circumscribed rectangle.

ここで、ｚはニューラルネットワークの出力を表し、Ａ（ｍ，ｐｏｓ）は、ｍ番目のサンプルについてオブジェクトが割り当てられたアンカーボックスの添字集合であり、具体的には、

などとしておく。 Then, the loss function (Loss function) L (z) is set as the sum of the position identification error Lloc (m, z) and the certainty factor Lconf (m, z) as follows.

Here, z represents the output of the neural network, and A (m, pos) is the index set of the anchor box to which the object is assigned for the m-th sample, specifically,

And so on.

なお、Ａ（ｍ，ｎｅｇ）は、ハードネガティブ（hard negative）の集合であって、オブジェクトに割り当てられていないアンカーボックスのうち、ｌ（ｍ，ｉ，ａ，ｚ）が大きい順に上位ｋ｜Ａ（ｍ，ｐｏｓ）｜個を選択して得られる（ハードネガティブマイニングと呼ばれる方法であるので、ここでの詳細な説明を省略する）。また、huber()は、ヒューバー関数（huber関数）である。この関数についても広く知られているのでここでの詳細な説明を省略する。 Lloc (m, z) and Lconf (m, z) are defined as follows.

Note that A (m, neg) is a set of hard negative and among the anchor boxes not assigned to the object, the top k | A in the order of large l (m, i, a, z) It is obtained by selecting (m, pos) | pieces (it is a method called hard negative mining, so the detailed description is omitted here). Also, huber () is a Huber function (huber function). Since this function is also widely known, the detailed description is omitted here.

Also in the image processing apparatus 1 of the present embodiment in which the frame detection unit 31, the face detection unit 32, the body detection unit 33, and the character detection unit 34 are configured as described above, the frame detection unit 31, the face detection unit 32, The classifiers 42a, b, c, d corresponding to the body detection unit 33 and the character detection unit 34 respectively specify information for identifying a top, information for identifying a face, and information for identifying a body. Since the information for identifying the character part is estimated, predetermined information processing is executed using these.

[Example of dividing image data]
In addition, as the operation of the receiving unit 21, the control unit 11 according to the present embodiment does not enlarge or reduce the entire input cartoon image data, and resizes it to a size suitable for the input of the neural network, but does not The extracted cartoon image data is divided into a plurality of divided portions based on predetermined conditions, and divided portions obtained by the division (partial cartoon image data, hereinafter referred to as It may be resized to a size suitable for the input of and output to the detection processing unit 22.

Here, the predetermined condition is, for example, dividing the original cartoon image data (width w, height h) into 2 × 2 pieces (each having a width w / 2, a height h / 2, an overlap It may be a condition to divide into four areas which do not match. Further, this condition may be, for example, a condition of division at a portion where white (background color) continues based on the content of the cartoon image data. Furthermore, the predetermined condition may be a condition of division into pieces.

In this example of the present embodiment, the frame detection unit 31, the face detection unit 32, the body detection unit 33, and the character detection unit 34 of the detection processing unit 22 do not divide a frame part from each of the partial image data And detect the face part, body part and character part. In this example, the machine learning process may also be performed using partial image data obtained by division.

The detection information generation unit 23 detects, for each partial image data, information specifying the frame portion detected by the frame detection unit 31, information detected by the face detection unit 32, information specifying the face portion, and the body detection unit 33 The information for identifying the body part and the information for identifying the character part detected by the character detection unit 34 are generated, and these are put together and the top part, face part, body part, and the like in the original cartoon image data Generates information that identifies each of the letter parts.

The information processing unit 24 uses the information specifying the top part generated by the detection information generation unit 23, the information specifying the face part, the information specifying the body part, and the information specifying the character part. Execute the information processing of.

Further, as described above, the predetermined condition for division into partial image data may be a condition of division for each frame. In this case, the control unit 11 may detect a top portion by the operation of the detection processing unit 22 as the frame detection unit 31, and generate partial image data by dividing the detected top portion.

That is, in this example of the present embodiment, as illustrated in FIG. 7, at least one curve (such as a circular arc) circumscribed to a frame portion (a frame line dividing a frame) output by the frame detection unit 31. The information which specifies “which may be included in the unit” is output to the face detection unit 32, the body detection unit 33, and the character detection unit 34. Then, each of the face detection unit 32, the body detection unit 33, and the character detection unit 34 sets partial image data with each frame portion as partial image data for each frame portion specified by the information output from the frame detection unit 31. The face part, the body part and the character part are detected from each of the Also in this example, the machine learning process related to the face detection unit 32, the body detection unit 33, and the character detection unit 34 may be performed using partial image data obtained by division.

Also in this example, the detection information generation unit 23 detects, for each frame portion detected by the frame detection unit 31, the information for identifying the face portion detected by the face detection unit 32, and the body portion detected by the body detection unit 33. Information identifying the character and information identifying the character part detected by the character detection unit 34 are generated, and these are put together to form each of the top, face, body, and character in the original cartoon image data. Generate information to identify

The information processing unit 24 uses the information specifying the top part generated by the detection information generation unit 23, the information specifying the face part, the information specifying the body part, and the information specifying the character part. Execute the information processing of.

[Composition of detection results]
When dividing image data to be processed, the control unit 11 resizes the image data before division to a size suitable for input to the neural network and performs the operation as the detection processing unit 22. Good. That is, as the operations of the frame detection unit 31, the face detection unit 32, the body detection unit 33, and the character detection unit 34, the control unit 11 calculates the frame part, the face part, the body part and the character part from each of the image data before division. To detect

Then, the control unit 11 stores information for identifying the frame part, the face part, the body part and the character part detected here, and further divides the image data to be processed into a plurality of divisions based on a predetermined condition. It is divided into parts, and the partial image data obtained by the division is resized to a size suitable for the input of the neural network, and the operation as the detection processing unit 22 is performed.

According to this example, information for identifying the top part, face part, body part and character part detected for the image data before division and the face part, body part and character part for each partial image data obtained after division Information to be identified will be obtained.

Then, the control unit 11 detects the top part, the face part, the body part and the character part detected from the image data before division, and the face part, the body part and the part detected from each of the partial image data after division. If a face part, a body part and a character part are detected from at least one of before or after division using information specifying a character part, the detection information generation unit 23 detects it from the at least one It generates and outputs information for identifying the face part, the body part and the character part (the detection results of each part are integrated and output, respectively).

In this example, the so-called OR of the face, body and character detected from the image data before division and the face, body and character detected from the image data after division is As a face portion, a body portion and a character portion detected from the image data to be processed, information for specifying a face portion, a body portion and a character portion detected from the image data to be processed is output.

In addition, since the top part can be generally detected with higher accuracy than the face part, the body part, or the character part, only one of the image data before division (or the image data after division may be used) Although it is considered sufficient if it is detected from the control unit 11, the control unit 11 also outputs information for specifying the frame portion when it is detected from at least one of the image data before division and the image data after division also for the frame portion. You may do it.

In addition, when outputting information of each part (each of a top, a face, a body, and a character) detected from one of the above, the duplication is excluded for the information of the overlapping parts. Output.

In addition, here, it is decided to output information for specifying a frame part, a face part, a body part and a character part detected from any of the image data before division and the image data after division. That is, for example, when the image data to be processed is image data of one page of a cartoon, “OR (logical sum)” of one detected in the entire page and one detected in each divided portion divided into portions I decided to take However, this example of the present embodiment is not limited to this, and the coma part, the face part, the body part and the character part commonly detected from both the image data before division and the image data after division Information to be specified may be output (that is, for example, when the image data to be processed is image data of one page of a cartoon, it is detected for each divided portion divided into portions and detected on the entire page) You may take "AND" with things).

Further, although the division mode is one type here, plural types of partial image data obtained by division in plural types of division modes may be generated. For example, partial image data obtained by dividing in a plurality of types of modes, such as partial image data obtained by division for each frame, partial image data obtained by division into 2 × 2, etc. (image before division It is also possible to output information specifying a frame part, a face part, a body part and a character part detected from any one or more of (or each of the data may be added). Also in this case, when duplication occurs, the duplication is excluded and output.

[Effect of the embodiment]
As described above, according to the present embodiment, independent detectors respectively correspond to the parts of the frame part, the face part of the character, the body part, and the character part, which are mutually overlapping or included in the cartoon image data. Alternatively, since the classification is performed by each of the classifiers, the accuracy of the detection can be improved as compared with the conventional detection using machine learning.

Reference Signs List 1 image processing apparatus, 11 control unit, 12 storage unit, 13 operation unit, 14 display unit, 15 input / output unit, 21 reception unit, 22 detection processing unit, 23 detection information generation unit, 24 information processing unit, 31 frame detection unit , 32 face detection units, 33 body detection units, 34 character detection units, 40 detectors, 41, 41 'base network units, 42 classifiers.

Claims (7)

Receiving means for receiving cartoon image data;
Frame detection means in a state of being machine-learned to detect a comic piece drawn in the image data from the image data;
Face detection means in a state of being machine-learned to detect a face portion drawn in the image data from the image data;
Body detection means in a state machine-learned to detect a body part drawn in the image data from the image data;
Character detection means in a state of being machine-learned to detect character parts included in the image data from the image data;
Information specifying the top part detected by the frame detecting means, information specifying the face part detected by the face detecting means, and the body detected by the body detecting means, based on the received cartoon image data Detection information generation means for generating information for specifying a portion and information for specifying a character portion detected by the character detection means;
Including
The image processing apparatus, wherein the information for specifying the generated frame part, the information for specifying a face part, the information for specifying a body part, and the information for specifying a character part are provided for predetermined information processing. The image processing apparatus according to claim 1,
The frame detection means, the face detection means, the body detection means, and the character detection means share in common the range of images to be candidates for detection and the feature amount of the images within the range. A base network unit that outputs a range of images to be candidates for detection and a feature amount of an image within the range based on image data that has been learned and is to be processed;
A classifier provided corresponding to each of the frame detection means, the face detection means, the body detection means, and the character detection means, which is included in the range of the corresponding image based on the feature amount of the image. An image processing apparatus including a classifier for classifying whether an image to be captured is a top part, a face part, a body part, or a character part. The image processing apparatus according to claim 1 or 2,
The detection information generation means includes the face detection means and the body detection means from each of divided portions obtained by dividing the received cartoon image data into a plurality of divided portions based on predetermined conditions. And an image processing apparatus for generating information for specifying a face portion, information for specifying a body portion, and information for specifying a character portion by means of a character detection unit. The image processing apparatus according to claim 3,
The detection information generation means uses the information for identifying the frame portion detected by the frame detection means, and uses each of the identified frame portions as the divided portion, from among each of the divided portions, the face detection means Image processing for generating information for specifying a face portion, information for specifying a body portion, and information for specifying a character portion by using the body detection means and the character detection means, for each top part which is a divided portion apparatus. The image processing apparatus according to claim 3 or 4, wherein
The detection information generation means is information for specifying the face portion detected by the face detection means, the body detection means and the character detection means from the image data before dividing the received cartoon image data, and the body part Information to identify the character part and information to specify the character part,
An image processing apparatus that integrates and outputs the generated information, information for identifying a face portion detected from each of the divided portions, information for identifying a body portion, and information for identifying a character portion. The image processing apparatus according to any one of claims 1 to 5, wherein
An image processing apparatus, wherein the frame detection means, the face detection means, the body detection means, and the character detection means are configured using a single shot multi-box detector (SSD). Computer,
Receiving means for receiving cartoon image data;
Frame detection means in a state of being machine-learned to detect a comic piece drawn in the image data from the image data;
Face detection means in a state of being machine-learned to detect a face portion drawn in the image data from the image data;
Body detection means in a state machine-learned to detect a body part drawn in the image data from the image data;
Character detection means in a state of being machine-learned to detect character parts included in the image data from the image data;
Information specifying the top part detected by the frame detecting means, information specifying the face part detected by the face detecting means, and the body detected by the body detecting means, based on the received cartoon image data Detection information generation means for generating information for specifying a portion and information for specifying a character portion detected by the character detection means;
To act as
The program by which the information which specifies the produced | generated top part, the information which specifies a face part, the information which specifies a body part, and the information which specifies a character part are provided to a predetermined information processing.

Images