TWI870063B - The method, the computing device, the computer-readable storage medium, and the computer program product for generating an image - Google Patents

Abstract

The present application provides a method for generating an image, which is suitable for generating an artificial intelligence image by means of a computing device. The method includes receiving at least one inputting image, generating a keyword character group based on the at least one inputting image, performing at least one character string modifying operation on the keyword character group based on a set of modifying instructions corresponding to a modifying request after receiving the modifying request sent from any one of the at least one modifying button and then generating a modifying character group, and generating the artificial intelligence image based on the modifying character group. Thereby, the method can be used for generating the artificial intelligence image and solves the problem that a general user must enter a corresponding instruction set in order to generate an image desired by the user. In addition, a computing device, a computer-readable storage medium, and a computer program product for carrying out the method are also provided.

Description

Method for generating images, computer device, computer readable recording medium and computer program product

The present application relates to a method, a computer device, a computer-readable recording medium and a computer program product, and in particular to a method, a computer device, a computer-readable recording medium and a computer program product for generating an image.

Thanks to the development of artificial intelligence, many drawing software or image generators are able to generate image files that users expect after introducing artificial intelligence technology, and help many media workers (for example, YouTubers or forum hosts) or ordinary people to deepen the audience's impression of the content and/or situation description of their published articles through artificial intelligence images.

However, existing drawing software or image generators require users to input corresponding command sets (Prompts) to generate the image files expected by the users. In other words, although the user can eventually generate the image files expected by the user, the user must first input a text description and the command sets corresponding to each specific photo editing operation.

According to the description of the prior art, the purpose of this application is to solve the shortcomings of the prior art. Specifically, one purpose of this application is to solve the problem that the user must input the corresponding command group to generate the image desired by the user, especially to solve the problem that the user who is not familiar with the command group must input the corresponding command group to generate the image desired by the user.

The present application provides a method for generating an image, which is suitable for generating an artificial intelligence image through a computer device. The method includes receiving at least one input image, generating a keyword character group based on the at least one input image, after receiving a photo editing request sent from any one of at least one photo editing buttons, performing at least one string editing operation on the keyword character group based on an editing instruction set corresponding to the photo editing request, and generating an edited character group, and generating an artificial intelligence image based on the edited character group.

In some embodiments, the step of generating a keyword character set based on at least one input image includes inputting the at least one input image into an image description model, and outputting the keyword character set through the image description model. The image description model automatically generates a keyword character set corresponding to the at least one input image based on the at least one input image.

In some embodiments, the step of generating an artificial intelligence image based on the edited character set includes inputting the edited character set into an image generation model, and outputting the artificial intelligence image through the image generation model. The image generation model automatically generates an artificial intelligence image corresponding to the edited character set based on the edited character set.

In some embodiments, the method for generating an image further includes automatically generating at least one photo editing button based on at least one of the content of the keyword character group and the user's historical editing record.

In some embodiments, the method for generating an image further includes determining whether to stack at least one string editing operation based on a stacking attribute value corresponding to each of the at least one editing button.

In some embodiments, a stack attribute value is set as stackable or non-stackable.

In some embodiments, the stack attribute value is set based on at least one of a user's preference setting and a default value.

In some embodiments, the method for generating an image further includes determining a degree of editing of a keyword character group by at least one string editing operation based on an editing weight value corresponding to each of at least one editing button.

In some embodiments, the edit weight value is set based on at least one of the content of the keyword character group, the user's historical editing record, and a default value.

In some embodiments, the method for generating an image further includes receiving object information corresponding to a physical size of the object, and determining a size of the artificial intelligence image based on the object information.

In some embodiments, the step of generating a keyword character set based on at least one input image includes receiving at least one decorative image, performing at least one composition operation on the at least one input image and at least one decorative image to generate a layout image, inputting the layout image into an image description model, and outputting a keyword character set through the image description model. The image description model automatically generates a keyword character set corresponding to the layout image based on the layout image.

In some embodiments, the method for generating an image further includes transmitting the artificial intelligence image to an image output device, and physically outputting the artificial intelligence image through the image output device.

The present application also provides a method for generating an image, which is suitable for generating a style conversion image through a computer device. The method includes receiving at least one input image, and after receiving a style conversion request sent from any one of at least one style conversion buttons, inputting at least one input image into a style conversion model based on the style conversion request, and outputting the style conversion image through the style conversion model. The style conversion model performs at least one style conversion operation on at least one input image based on the style learning result of at least one learning image, and generates a style conversion image.

In some embodiments, at least one learning image is an artificial intelligence image generated by any of the methods for generating images as previously described.

In some embodiments, the method for generating an image further includes transmitting the style-converted image to an image output device, and physically outputting the style-converted image through the image output device.

The present application also provides a computer device for generating an image, which is suitable for being connected to a user terminal signal to receive at least one input image from the user terminal and to generate an output image based on the at least one input image. The computer device includes an image receiving module, a processing module and a storage module. The image receiving module is configured to be connected to a user terminal signal and is configured to be suitable for receiving at least one input image. The processing module is configured to be connected to the image receiving module signal, and is configured to execute any of the methods for generating images as described above to generate an artificial intelligence image after executing the program code stored in the storage module, or to execute any of the methods for generating images as described above to generate a style conversion image, and use the artificial intelligence image or the style conversion image as an output image. The storage module is configured to be connected to the processing module signal, and is configured to store the program code.

In some embodiments, the computer device for generating images further includes an image output module, which is configured to be signal-connected to the processing module and is suitable for transmitting the output image to the image output device.

The present application also provides a computer-readable recording medium for generating images. When a computer device loads the program code in the memory and executes the program code, any of the methods for generating images as described above can be completed.

The present application also provides a computer program product for generating images. When a computer device loads the computer program product and executes the computer program product, it can complete any of the methods for generating images as described above.

The technical means provided by this application can produce beneficial effects that cannot be achieved by previous technologies. Specifically, one beneficial effect of this application is that the user can generate the image file expected by the user without the need to input the command set by himself, thereby reducing the user's operation threshold and/or operation difficulty, so that users who are not familiar with the command set can also generate the image file expected by the user through the technical means provided by this application.

This application will be described in detail through the following embodiments and the attached drawings to help those with ordinary knowledge in the technical field to which this application belongs to understand the purpose, features and efficacy of this application. It should be noted that the various steps described herein can be executed sequentially, in reverse order, or by appropriately changing or skipping the order during the control process. It should be noted that the "first step can be executed after the second step" described herein can mean that the first step is directly executed after the second step is executed, or it can mean that after the second step is executed, other steps (for example, the third step) are executed first and then the first step is executed.

In addition, in the description of the present application, it should be noted that terms such as "first", "second" and "third" are used to distinguish between elements, rather than to limit the elements themselves or to indicate a specific order of the elements. It should be noted that in the following description, the same elements or steps may be represented by the same numbers.

Please refer to FIG. 1 , which is a schematic diagram illustrating a connection relationship between a computer device 200 in an embodiment of the present application, which is signal-connected to at least one of a user device 110 and a server 120, and which is signal-connected to at least one of an image output device 310 and a server 320.

Since the user device 110 and/or the server 120 can be configured to be signal-connected to the computer device 200, it can be specifically configured to configure a signal line between the user device 110 and/or the server 120 and the computer device 200, so that the user device 110 and/or the server 120 are suitable for providing an input image (specifically at least one input image) to the computer device 200 via the signal line. In some embodiments, the input image can be provided by, for example, a co-branded manufacturer, an artist, or a consumer, but is not limited thereto. In some embodiments, the input image can be, for example, a portrait, a fruit, an animal, an anime character, or a self-created character, but is not limited thereto. In a specific example, the user can provide one or more photo files as the input image to the computer device 200 through the user device 110.

In some embodiments, the computer device 200 may receive input images from the user device 110 and/or the server 120 via a physical signal line connection, wherein the physical signal line may be, for example, a network signal line that complies with the Internet protocol, but is not limited thereto. In other embodiments, the computer device 200 may also receive input images from the user device 110 and/or the server 120 without using a physical signal line connection. More specifically, the computer device 200 may also receive input images from the user device 110 and/or the server 120 via a virtual signal line connection, wherein the virtual signal line may be, for example, Wi-Fi, 4G/5G, Bluetooth, near field communication, etc. that complies with the wireless communication protocol, but is not limited thereto.

In some embodiments, the user device 110 may be a computer device capable of storing files, which may specifically be a smart phone, a tablet computer, a personal computer, etc., but is not limited thereto. In some embodiments, the server 120 may be a computer device capable of storing files, which may specifically be a physical host or a virtual cloud server, but is not limited thereto.

The computer device 200 may be configured to receive an input image from the user device 110 and/or the server 120, and to generate an artificial intelligence image based on the received input image, which may specifically be generated by executing the various steps of the method for generating an image provided by the present application to generate an artificial intelligence image and/or a style conversion image. In some embodiments, the input image may also be stored in the computer device 200 in advance, so that the computer device 200 can also generate an artificial intelligence image and/or a style conversion image based on the input image stored in the computer device 200.

The computer device 200 can be basically configured to include an image receiving module 210, a processing module 220 (i.e., at least one of the processing module 220A and the processing module 220B), and a storage module 230. In some embodiments, the computer device 200 can further include an image output module 240. In other embodiments, the computer device 200 can further include an image database 250. In other embodiments, the computer device 200 can further include an editing database 260. In other words, the image output module 240, the image database 250, and the editing database 260 can be selectively configured in the computer device 200 according to the needs of the user.

The image receiving module 210 may be configured to communicate with the user device 110 and/or the server 120, and may be configured to be suitable for receiving input images (specifically, at least one input image) from the user device 110 and/or the server 120. In some embodiments, the computer device 200 may store the input images received from the user device 110 and/or the server 120 through the image receiving module 210 in the image database 250.

The processing module 220 can be configured to be signal-connected to the image receiving module 210, and can be configured to implement any of the methods for generating images provided in the present application. More specifically, the processing module 220 can be configured to generate an artificial intelligence image and/or a style conversion image based on the input image after executing each step of the method for generating images provided in the present application. In some embodiments, the processing module 220 can be a central processing unit, which can specifically be any type of processor known to a person of ordinary skill in the art to which the present application belongs, but is not limited thereto.

In some embodiments, the processing module 220 may refer to a processing module 220A capable of generating an artificial intelligence image based on an input image. In other embodiments, the processing module 220 may refer to a processing module 220B capable of generating a style conversion image based on an input image. In other embodiments, the functions and configurations of the processing module 220A and the processing module 220B may be integrated into the same processing module, so that the processing module can generate not only an artificial intelligence image based on an input image, but also a style conversion image based on an input image.

The storage module 230 can be configured to be signal-connected to the processing module 220 (including the processing module 220A and/or the processing module 220B), and can be configured to store a program code, so that the processing module 220 can execute each step of the method for generating an image provided by the present application after loading and executing the program code. In some embodiments, the storage module 230 can be basically configured to include a volatile memory (specifically, at least one volatile memory) and a non-volatile memory (specifically, at least one non-volatile memory). In some embodiments, the volatile memory may be any type of memory known to a person skilled in the art, such as a dynamic random access memory or a static random access memory, but not limited thereto. In some embodiments, the non-volatile memory may be any type of memory known to a person skilled in the art, such as a read-only memory, a flash memory, or a non-volatile random access memory, but not limited thereto.

The image output module 240 may be configured to be signal-connected to the processing module 220 (including the processing module 220A and/or the processing module 220B), and may be configured to output the image generated by the processing module 220. In some embodiments, the image output module 240 may output the artificial intelligence image generated by the processing module 220A and/or the style conversion image generated by the processing module 220B to the image output device 310 and/or the server 320.

In some embodiments, the artificial intelligence image generated by the processing module 220A and/or the style conversion image generated by the processing module 220B can be stored in the image database 250. After the processing module 220A and/or the processing module 220B receive a request for image output, the processing module 220A and/or the processing module 220B can output the artificial intelligence image and/or the style conversion image stored in the image database 250 to the image output device 310 and/or the server 320, respectively.

The image database 250 may be configured to store images, which may be, for example, input images, artificial intelligence images, and/or style-converted images, but are not limited thereto. In some embodiments, the file format of the images stored in the image database 250 may be, but are not limited to, Joint Photographic Experts Group (JPEG), Portable Network Image (PNG), Bitmap Image Format (BMP), Graphics Interchange Format (GIF), or Tagged Image File Format (TIFF), etc.

The editing database 260 may be configured to store the user's historical editing records, which may be, for example, historical records of string editing operations and/or historical records of style conversion operations, particularly historical records of string editing operations and/or historical records of style conversion operations corresponding to input images, but not limited thereto. In some embodiments, the user's historical editing records may also include the time points of executing string editing operations and/or the time points of executing style conversion operations.

Since the computer device 200 can be configured to be signal-connected to the image output device 310 and/or the server 320, it can be specifically configured by configuring a signal line between the computer device 200 and the image output device 310 and/or the server 320, so that the computer device 200 is suitable for outputting artificial intelligence images to the image output device 310 and/or the server 320 via the signal line.

In some embodiments, the computer device 200 may output the artificial intelligence image to the image output device 310 and/or the server 320 by connecting with a physical signal line, wherein the physical signal line may be, for example, a network signal line that complies with the Internet protocol, but is not limited thereto. In other embodiments, the computer device 200 may output the artificial intelligence image to the image output device 310 and/or the server 320 without connecting with a physical signal line. More specifically, the computer device 200 may output the artificial intelligence image to the image output device 310 and/or the server 320 by connecting with a virtual signal line, wherein the virtual signal line may be, for example, Wi-Fi, 4G/5G, Bluetooth, near field communication, etc. that complies with the wireless communication protocol, but is not limited thereto.

Thus, the computer device 200 shown in FIG1 can not only generate an artificial intelligence image and/or a style conversion image based on the received input image, but also can generate the artificial intelligence image and/or style conversion image without the user having to input an additional command set. In other words, the computer device for generating images provided by the present application can reduce the user's operating threshold and/or operating difficulty, thereby enabling users who are not familiar with the command set to generate the image file desired by the user through the computer device.

Please refer to FIG2, which is a block diagram of a processing module 220A of an embodiment of the present application. The configuration of the processing module 220A may be substantially the same as the processing module 220 shown in FIG1, except that the processing module 220A may be specifically configured to generate an artificial intelligence image based on the input image, and the function and configuration of generating a style conversion image based on the input image may be omitted.

The processing module 220A can be basically configured to include an image receiving unit 221A, a string generating unit 222A, a picture editing request receiving unit 223A, a string editing unit 224A, an image generating unit 225A and an image output unit 226A. In some embodiments, the processing module 220A can further include an image description model 270. In some embodiments, the processing module 220A can further include an image generating model 280. In other words, the image description model 270 and/or the image generating model 280 can be selectively configured in the processing module 220A according to the needs of the user. In some embodiments, the function and configuration of the image description model 270 can be integrated into the string generating unit 222A. In some embodiments, the function and configuration of the image generating model 280 can be integrated into the image generating unit 225A.

The image receiving unit 221A may be configured to be suitable for receiving input images (specifically, at least one input image) from the user device 110 and/or the server 120 through the image receiving module 210. In some embodiments, the image receiving unit 221A may also be configured to receive input images from the image database 250, that is, the input images may be stored in the image database 250 in advance through the image receiving module 210, so that the image receiving unit 221A can also receive input images from the image database 250.

The string generation unit 222A can be configured to generate a keyword character set based on an input image, more specifically, the string generation unit 222A can be configured to generate a keyword character set based on an input image received by the image receiving unit 221A. In some embodiments, the string generation unit 222A can input the input image to the image description model 270, and automatically generate and output a keyword character set through the image description model 270. In some embodiments, the image description model 270 can be, for example, a CLIP model or DeepBooru, but is not limited thereto.

In more detail, the Clip model is a deep learning model that is trained in advance with multiple data (each data includes a training image and its corresponding training string), so that the trained Clip model can match the input image to the corresponding string after recognizing the received input image; and the DeepBooru is also a deep learning model that is trained in advance with multiple data (each data includes a training image and its corresponding training string), so that the trained DeepBooru can also match the input image to the corresponding string (i.e., keyword character group) after recognizing the received input image.

The Clip model and DeepBooru can both generate corresponding strings based on images. The difference between the two is that DeepBooru can directly generate keyword character sets corresponding to the input image, while the Clip model generates a complete string corresponding to the input image (and then generates keyword character sets based on the complete string generated by the Clip model). In some embodiments, the Clip model can be used in conjunction with a large language model such as ChatGPT to generate keyword character sets based on the complete string generated by the Clip model, but is not limited thereto.

The photo editing request receiving unit 223A can be configured to receive a photo editing request sent by a photo editing button (such as the photo editing button 630 shown in Figure 6), that is, when the user clicks on any one of at least one photo editing button, the clicked photo editing button will send a corresponding photo editing request, so that the photo editing request receiving unit 223A can receive the photo editing request corresponding to the clicked photo editing button.

The string editing unit 224A may be configured to perform a string editing operation on a keyword character set. Specifically, the string editing unit 224A may be configured to perform a string editing operation on the keyword character set generated by the string generating unit 222A. More specifically, the string editing unit 224A may be configured to perform a string editing operation on the keyword character set generated by the string generating unit 222A and generate an edited character set after the image editing request receiving unit 223A receives an image editing request sent by any one of the at least one image editing buttons. The string editing operation may include adding, modifying and/or deleting, but is not limited thereto. In some embodiments, the string editing operations can be determined in advance according to the editing buttons 630 shown in FIG. 6 , respectively. Therefore, the string editing unit 224A can generate the editing character set according to the string editing operations corresponding to the editing buttons 630 shown in FIG. 6 .

The image generation unit 225A can be configured to generate an artificial intelligence image based on an edited character set. More specifically, the image generation unit 225A can be configured to generate an artificial intelligence image based on an edited character set generated by the string editing unit 224A. In some embodiments, the image generation unit 225A can input the edited character set into the image generation model 280, and automatically generate and output an artificial intelligence image through the image generation model 280. In some embodiments, the image generation model 280 can be any model known to a person of ordinary skill in the art to which the present application belongs that can generate an image based on a string, such as Stable Diffusion, but not limited thereto.

The Stable Diffusion is a deep learning model that is trained in advance with multiple data (each data includes a training string and its corresponding training image), so that after the trained Stable Diffusion recognizes the received edited character group, it can generate an artificial intelligence image corresponding to the edited character group based on the edited character group.

The image output unit 226A may be configured to output the artificial intelligence image generated by the image generation unit 225A. In some embodiments, the image output unit 226A may store the artificial intelligence image in a specific file format in the image database 250. In some embodiments, the image output unit 226A may output the artificial intelligence image in a specific file format to the image output device 310 and/or the server 320 through the image output module 240.

Thus, the processing module 220A shown in FIG2 can not only generate an artificial intelligence image based on the received input image, but also generate the artificial intelligence image without the user having to input an additional command set. In other words, the computer device for generating images provided by the present application can reduce the user's operating threshold and/or operating difficulty, thereby enabling users who are not familiar with the command set to generate the image file desired by the user through the computer device.

Please refer to Figure 3, which is a flow chart of a method for generating an image of the first embodiment of the present application. More specifically, the method shown in Figure 3 can generate an artificial intelligence image, and the method includes steps S310, S320, S330 and S340. In some embodiments, the method shown in Figure 3 can use Visual Studio Code as a development environment, and can be implemented with the framework of pytorch and huggingface diffusers using python as the programming language. In some embodiments, the method shown in Figure 3 can use open source libraries such as torch, diffusers and numpy.

In step S310, an input image (specifically, at least one input image) is received. Step S310 can be executed through the image receiving unit 221A of the processing module 220A as shown in FIG. 2 . In some embodiments, the input image can be an original image file (i.e., an image file that has not undergone any image processing operation), such as a photo file obtained by a camera module (not shown), but not limited to this. In other embodiments, the input image can also be an image file that has undergone an image processing operation (specifically, at least one image processing operation), such as an image file obtained by an image processing model such as Photoshop, but not limited to this. In some embodiments, the image processing operation can include a background removal operation, an image resizing operation, an image mirror flip operation, or an image rotation operation, but not limited to this.

In step S320, a keyword character set is generated based on the input image. Step S320 can be executed by the string generation unit 222A of the processing module 220A as shown in FIG2 . In some embodiments, step S320 can be executed after step S310. In some embodiments, after executing step S310, an image processing operation can be performed on the input image first, and then step S320 is executed (i.e., a keyword character set is generated based on the input image after the image processing operation).

In some embodiments, the input image can be analyzed by a pre-established image recognition model (not shown) to identify the main elements and/or other elements (i.e., non-main elements) in the input image, and then generate keyword character sets based on the elements in the input image. More specifically, ControlNet-openpose, for example, can be used to analyze the hand posture and limb movements of the input image, and ControlNet-Canny, for example, can be used to analyze the object edge detection and boundary delineation of the input image, and then generate keyword character sets based on the analysis results.

In step S330, after receiving a photo editing request sent from any one of the photo editing buttons (specifically, at least one photo editing button), a string editing operation (specifically, at least one string editing operation) is performed on the keyword character group based on the editing instruction set corresponding to the photo editing request, and an edited character group is generated. Step S330 can be executed by the photo editing request receiving unit 223A and the string editing unit 224A of the processing module 220A as shown in FIG2 . In some embodiments, step S330 can be executed after step S320.

In some embodiments, the photo editing buttons may be pre-established, so that the user can operate the input image through any of the photo editing buttons (i.e., perform string editing operations on the keyword character group corresponding to the input image). In some embodiments, the number of the photo editing buttons may be plural, so that the user can click a suitable photo editing button under a better user experience. In some embodiments, the label content corresponding to each of the photo editing buttons may be edited, so that the user can more intuitively know the expected results of the string editing operations performed on the keyword character group by each photo editing button according to the label content.

In a specific example, the photo editing buttons may include a first photo editing button (the label content is displayed in American style), a second photo editing button (the label content is displayed in Japanese style) and a third photo editing button (the label content is displayed in Ghibli style). When the user clicks the third photo editing button, the third photo editing button can issue a photo editing request, so that after the photo editing request receiving unit 223A receives the photo editing request sent by the third photo editing button, the string editing unit 224A can perform a string editing operation on the keyword character group based on the editing instruction set corresponding to the photo editing request (i.e., add a text description of "ghibli" to the content of the keyword character group).

More specifically, the user can click the third photo editing button (the label content shows Ghibli style) to switch the pre-trained model (base model) and style fine-tuning layer (LORA layer) that are most suitable for this style, and after testing, select the most suitable model and parameter combination to combine to generate an instruction set corresponding to the third photo editing button, and use the instruction set to perform string editing operations on keyword character groups.

Thus, through step S330, the user can perform string editing operations without having to input an additional command group. In other words, the step can lower the user's operation threshold and/or operation difficulty, so that users who are not familiar with the command group can also perform string editing operations on the keyword character group corresponding to the input image by directly clicking the image editing button.

In some embodiments, step S330 may be executed more than once, that is, the processing module 220A may execute step S330 once or more times. In some embodiments, the number of times the processing module 220A executes step S330 may depend on the number of times the user clicks the photo editing buttons. Specifically, after the user clicks any one of the photo editing buttons to cause the processing module 220A to execute step S330 and generate an edit character group, the user may click any one of the photo editing buttons to cause the processing module 220A to execute step S330 again and generate a new edit character group (i.e., the previously generated edit character group is regarded as the current keyword character group, and a string editing operation is performed on the current keyword character group). In some embodiments, after receiving the restore request, the processing module 220A may restore the current edited character group to the keyword character group or the old edited character group (the old edited character group may be, for example, the edited character group before the most recent modification, i.e., the edited character group before the most recent string editing operation). In some embodiments, when the processing module 220A continuously receives the same image editing request, the processing module 220A only performs the corresponding string editing operation on the keyword character group or the current edited character group after the first receipt of the image editing request, i.e., the processing module 220A no longer performs the corresponding string editing operation on the current edited character group after receiving the second, third, ..., Nth image editing request.

In step S340, an artificial intelligence image is generated based on the edited character set. Step S340 can be executed by the image generation unit 225A of the processing module 220A as shown in Figure 2. In some embodiments, step S340 can be executed after step S330.

Thus, the method shown in FIG3 can not only generate an artificial intelligence image based on the received input image, but also generate the artificial intelligence image without the user having to input an additional command set. In other words, the computer device for generating images provided by the present application can reduce the user's operating threshold and/or operating difficulty, thereby enabling users who are not familiar with the command set to generate the image file desired by the user through the computer device.

Please refer to FIG. 4A , which is a detailed flow chart of generating a keyword character set (i.e., step S320) based on at least one input image according to an example of the present application. That is, step S320 shown in FIG. 3 may include steps S410A, S420A, and S430A, and step S320 may be completed by executing steps S410A, S420A, and S430A, wherein steps S410A, S420A, and S430A may be executed by the string generation unit 222A and the image description model 270 of the processing module 220A as shown in FIG. 2 .

In step S410A, the input image is input to the image description model 270. In some embodiments, step S410A may be performed after step S310. In some embodiments, the image description model 270 may be, for example, a CLIP model or DeepBooru, but is not limited thereto.

In step S420A, a keyword character set is generated through the image description model 270. In some embodiments, step S420A can be executed after step S410A. Since the image description model 270 has been trained in advance through a plurality of data (each data includes a training image and its corresponding training string), the trained image description model 270 can automatically and more accurately generate a keyword character set corresponding to the input image based on the input image after recognizing the input image.

In step S430A, the keyword character set is outputted through the image description model 270. In some embodiments, step S430A may be executed after step S420A. That is, by executing step S430A, the processing module 220A may further process the keyword character set generated and outputted by the image description model 270 (i.e., step S330).

In some embodiments, step S420A and step S430A may be integrated into one step, that is, generating and outputting keyword character sets through the image description model 270.

Thus, by executing the steps shown in FIG. 4A , a keyword character set corresponding to the input image can be automatically and more accurately generated based on the input image.

Please refer to FIG. 4B , which is a detailed flowchart of an example of generating an artificial intelligence image based on editing a character set (i.e., step S340) of the present application. That is, step S340 shown in FIG. 3 may include steps S410B, S420B, and S430B, and step S340 may be completed by executing steps S410B, S420B, and S430B, wherein steps S410B, S420B, and S430B may be executed by the image generation unit 225A and the image generation model 280 of the processing module 220A as shown in FIG. 2 .

In step S410B, the edited character set is input to the image generation model 280. In some embodiments, step S410B may be executed after step S330. In some embodiments, the image generation model 280 may be any model known to a person of ordinary skill in the art to which the present application belongs that can generate an image based on a string, such as Stable Diffusion, but not limited thereto.

In step S420B, an artificial intelligence image is generated through the image generation model 280. In some embodiments, step S420B can be executed after step S410B. Since the image generation model 280 has been trained in advance through a plurality of data (each data includes a training string and its corresponding training image), the trained image generation model 280 can generate an artificial intelligence image corresponding to the edited character group based on the edited character group after recognizing the received edited character group.

In step S430B, the artificial intelligence image is outputted through the image generation model 280. In some embodiments, step S430B may be executed after step S420B. That is, by executing step S430B, the computer device 200 may further process the artificial intelligence image generated and outputted by the image generation model 280 (for example, displaying the artificial intelligence image on the user's screen or outputting the artificial intelligence image to an image output device, but not limited thereto).

In some embodiments, step S420B and step S430B may be integrated into one step, that is, generating and outputting an artificial intelligence image through the image generation model 280 .

Thus, by executing the steps shown in FIG. 4B , an artificial intelligence image corresponding to the edited character set can be automatically and more accurately generated based on the edited character set.

Please refer to FIG5, which is a schematic diagram illustrating the results generated after executing various steps of an example of the present application. More specifically, FIG5 illustrates that the processing module 220A can generate an artificial intelligence image 540 based on the input image 510 by executing steps S310, S320, S330 and S340 as shown in FIG3, wherein the detailed description of each step is described below.

First, the processing module 220A can receive the input image 510 from the user device 110 and/or the server 120 via the image receiving module 210 by executing step S310. Secondly, the processing module 220A can generate the keyword character set 520 based on the received input image 510 by executing step S320. Furthermore, after receiving a photo editing request sent from any one of the photo editing buttons 630 shown in FIG. 6 , the processing module 220A can perform a string editing operation on the keyword character set 520 based on the editing instruction set corresponding to the photo editing request by executing step S330, and generate the edited character set 530. Finally, the processing module 220A can generate an artificial intelligence image 540 based on the edited character set 530 by executing step S340.

Please refer to FIG. 6 , which is a schematic diagram of a display screen 600 on a user device for illustrating an embodiment of the present application. The display screen 600 on the user device may basically include a first display area 610 and a photo editing button 630 (specifically, at least one photo editing button 630). In some embodiments, the display screen 600 on the user device may further include a second display area 620. It should be noted that the configuration relationship (e.g., relative size and/or relative position) of each element shown in FIG. 6 can be adjusted according to the needs of the user, that is, the configuration relationship shown in FIG. 6 is only an example for schematic illustration.

The photo editing buttons 630 allow the user to operate the input image 510 by clicking any one of the photo editing buttons 630 (i.e., perform a string editing operation on the keyword character set 520 corresponding to the input image 510). In some embodiments, the photo editing buttons 630 may be pre-established. In other embodiments, the photo editing buttons 630 may be automatically established based on at least one of the content of the keyword character set 520 and the user's historical editing record (as shown in step S710 of FIG. 7 ). In some embodiments, the number of photo editing buttons 630 may be plural, so as to provide the user with a better user experience by clicking a suitable photo editing button 630. In some embodiments, by editing the label content corresponding to at least one photo editing button 630, the user can more intuitively understand the expected results of the string editing operation performed by each photo editing button 630 on the keyword character group 520 based on the label content.

In some embodiments, the first display block 610 may be configured to display an artificial intelligence image 540. More specifically, after the user clicks any one of the photo editing buttons 630, the user may be aware of the generated artificial intelligence image 540 through the first display block 610. Thus, through the display content of the first display block 610, the user may evaluate whether the current artificial intelligence image 540 meets the user's expectations, and decide whether to regenerate a new artificial intelligence image 540 through the photo editing button 630.

In some embodiments, the second display block 620 may be configured to display at least one of the keyword character set 520 and the edit character set 530, that is, the second display block 620 may first display the keyword character set 520, and after the string editing operation is performed, the second display block 620 may display the edit character set 530. In some embodiments, the second display block 620 may be omitted, that is, the display screen 600 on the user device may only display the first display block 610 and the photo editing button 630, but not the second display block 620, thereby providing the user with a better user experience.

Please refer to FIG. 7, which is a flow chart of a method for generating an image according to the second embodiment of the present application. More specifically, the method shown in FIG. 7 can generate an artificial intelligence image, and the method includes steps S310, S320, S330, S340, and S710. Steps S310, S320, S330, and S340 are substantially the same as those shown in FIG. 3, that is, the method shown in FIG. 7 may include steps S310, S320, S330, and S340 substantially the same as those shown in FIG. 3, and further includes step S710.

In step S710, a photo editing button (specifically, at least one photo editing button) is automatically generated based on at least one of the content of the keyword character group and the user's historical editing record. Step S710 can be executed by a button generation unit (not shown) of the processing module 220A. In some embodiments, step S710 can be executed after step S320. In some embodiments, a photo editing button can be automatically generated based on the content of the keyword character group through a text analysis model. In some embodiments, the text analysis model can be, for example, BERTScore, but is not limited thereto.

The BERTScore is a text analysis model that can automatically evaluate the semantic similarity between two texts. Therefore, the BERTScore can be used to calculate the scores of keyword character groups and other character groups, and the BERTScore can be used to provide an editing character group that is closest to the current keyword character group, thereby automatically generating a photo editing button that can generate the editing character group based on the keyword character group.

In a specific example, when the content of the keyword character group is "cat, rating:safe, animal, no_humans, simple_background, realistic, animal_focus, looking_at_viewer, pubic_hair, black_background", since the BERTScore can calculate that the score of Ghibli style is 0.7, the score of Picasso style is 0.2, and the score of oil painting style is 0.1, the BERTScore can automatically generate a fourth photo editing button (the label content is displayed as Ghibli style), a fifth photo editing button (the label content is displayed as Picasso style), and a sixth photo editing button (the label content is displayed as oil painting style).

In some embodiments, the photo editing buttons can be automatically generated based on the user's historical editing records stored in the editing database 260. For example, the user's frequently used string editing operations can be calculated based on the user's historical editing records, and then the photo editing buttons can be generated based on the user's frequently used string editing operations.

Thus, through step S710 as shown in FIG. 7 , suitable photo editing buttons can be automatically generated according to the user's usage situation, so that the user can more effectively click the suitable photo editing button to perform string editing operations on the keyword character group corresponding to the input image and generate an artificial intelligence image.

In some embodiments, step S710 may further automatically generate a retouching button according to the content of the edited character set. That is, after executing step S330, step S710 may be executed again, so that through step S710, a retouching button is automatically generated according to at least one of the content of the edited character set and the user's historical editing record.

Please refer to Figure 8, which is a schematic diagram of another example of the present application after executing various steps. More specifically, Figure 8 illustrates that the processing module 220A can generate an artificial intelligence image 840 based on the input image 510 by executing steps S310, S320, S330 and S340 as shown in Figure 3, as described in detail below.

Since the user can click on each image editing button to make the processing module 220A execute the corresponding string editing operation, in some embodiments, each image editing button can have its own stacking attribute value, so that the processing module 220A can determine whether to stack the string editing operation based on the stacking attribute value corresponding to each image editing button when executing step S330. In other words, the stacking attribute value can be configured to determine whether each string editing operation can be stacked with each other. Since the processing module 220A can determine whether each string editing operation can be stacked with each other based on the stacking attribute value, the processing module 220A can stack the stackable string editing operations, so that the user can generate more diverse artificial intelligence images. At the same time, the non-stackable string editing operations can prevent the user from clicking the editing button with the opposite effect and generating an artificial intelligence image beyond the user's expectation.

Please refer to Figures 5 and 8 at the same time. After receiving a photo editing request sent from any one of the photo editing buttons 630 shown in Figure 6, the processing module 220A can perform a string editing operation on the keyword character group 520 based on the editing instruction set corresponding to the photo editing request (i.e., adding a text description of "beagle dog" to the content of the keyword character group 520) by executing step S330, and generate an edited character group 530 as shown in Figure 5. If the user clicks on any of the retouching buttons 630 whose stacked attribute value is set as stackable, the processing module 220A can perform a string editing operation on the editing character set 530 shown in FIG5 (i.e., add the text description "ghibli" to the content of the editing character set 530 shown in FIG5 ) based on the editing instruction set corresponding to the retouching request after receiving the retouching request sent by the retouching button 630 whose stacked attribute value is set as stackable by executing step S330 again, and generate the editing character set 830 shown in FIG8 . That is, the editing character set 830 shown in FIG8 is generated in response to the user clicking on two retouching buttons whose stacked attribute values are set as stackable. Finally, the processing module 220A can generate an artificial intelligence image 840 based on the edited character set 830 by executing step S340, thereby generating more diverse artificial intelligence images.

In some embodiments, each stacking attribute value can be set as stackable or non-stackable, thereby making it easier to determine whether each string editing operation can be stacked with each other. In other embodiments, the stacking attribute values that can be stacked with each other can be set to the same group code, thereby determining whether each string editing operation can be stacked with each other through the classification of the group code (for example, the same group code indicates that each string editing operation is stackable with each other, while different group codes indicate that each string editing operation is non-stackable with each other), thereby making it more accurate to determine whether each string editing operation can be stacked with each other.

In some embodiments, the stacking attribute value may be set according to at least one of a user's preference setting and a default value. In a specific example, the stacking attribute value may be set according to the default value first, for example, the seventh photo editing button (label content displayed as Ghibli style) and the eighth photo editing button (label content displayed as American style) are set as stackable, while the ninth photo editing button (label content displayed as Japanese style) and the tenth photo editing button (label content displayed as realistic style) are set as non-stackable, and then the stacking attribute value of each photo editing button may be adjusted according to the user's preference setting, for example, the ninth photo editing button (label content displayed as Japanese style) is adjusted to be stackable.

Please refer to Figure 9, which is a schematic diagram of another example of the present application after executing various steps. More specifically, Figure 9 illustrates that the processing module 220A can generate an artificial intelligence image 940 based on the input image 510 by executing steps S310, S320, S330 and S340 as shown in Figure 3, as described in detail below.

In some embodiments, each of the image editing buttons may have its own editing weight value, so that the processing module 220A may determine the editing degree of the keyword character group by the string editing operation based on the editing weight value corresponding to each of the image editing buttons when executing step S330. In some embodiments, the editing weight value may be set to a value between 0 and 1, and a middle value (e.g., 0.5) may be used as a reference point. For example, when the editing weight value is set to 0.5, it means that the editing degree of the keyword character group by the string editing operation is the basic setting; when the editing weight value is set to greater than 0.5, it means that the editing degree of the keyword character group by the string editing operation is the enhanced setting (that is, the editing degree is higher than the basic setting); and when the editing weight value is set to less than 0.5, it means that the editing degree of the keyword character group by the string editing operation is the weakened setting (that is, the editing degree is lower than the basic setting).

In a specific example, the editing weight value of the eleventh retouching button (label content displayed as Ghibli style) can be set to a value greater than 0.5 (e.g., 0.7), and the editing weight value of the twelfth retouching button (label content displayed as beagle) can be set to a value of 0.5. When the user clicks the eleventh retouching button, the processing module 220A can perform a string editing operation on the keyword character group at a higher editing level than the basic setting when executing step S330, so that the editing level of the eleventh retouching button is more significant.

Please refer to FIG8 and FIG9 at the same time. The edit character set 830 shown in FIG8 is that the keyword character set 520 can be generated in response to the user clicking two stacking attribute values set as stackable editing buttons (for example, the eleventh editing button and the twelfth editing button mentioned above); and the edit character set 930 shown in FIG9 is that the keyword character set 520 can be further generated in response to the editing weight value of the eleventh editing button and the editing weight value of the twelfth editing button. More specifically, since the editing weight value of the eleventh editing button (the label content is displayed as Ghibli style) can be set to a value greater than 0.5 (for example, 0.7), the content of the edit character set 930 shown in FIG9 is "ghibli++". Finally, the processing module 220A can generate an artificial intelligence image 940 based on the edited character set 930 by executing step S340, thereby generating more diverse artificial intelligence images.

In some embodiments, the editing weight value may be set according to at least one of the content of the keyword character group, the user's historical editing record, and a default value. In a specific example, the editing weight value may be set according to the default value, for example, the editing weight value of the eleventh editing button (label content displayed as Ghibli style) may be set to a value greater than 0.5 (e.g., 0.7), and the editing weight value of the twelfth editing button (label content displayed as beagle) may be set to a value of 0.5. In a specific example, the editing weight value may be set according to the user's historical editing record, for example, the editing weight value corresponding to the editing button that the user uses more frequently is set to a value greater than 0.5. In a specific example, the edit weight value can be automatically set based on the content of the keyword character group. For example, each edit weight value can be set by BERTScore as described previously.

Please refer to FIG. 10 , which is a flow chart of a method for generating an image according to the third embodiment of the present application. More specifically, the method shown in FIG. 10 can generate an artificial intelligence image, and the method includes steps S310, S320, S330, S340, S1010, and S1020. Steps S310, S320, S330, and S340 are substantially the same as those shown in FIG. 3 , that is, the method shown in FIG. 10 may include steps S310, S320, S330, and S340 substantially the same as those shown in FIG. 3 , and further include steps S1010 and S1020.

In step S1010, object information corresponding to the physical size of the object is received. Step S1010 can be executed by the object information receiving unit (not shown) of the processing module 220A. The object information may include object dimensions such as length, width, and height. More specifically, when the length and width of the physical size of the object are a first size and a second size, respectively, the object information receiving unit can receive object information of the first size and the second size. In a specific example, when the physical size of an object (e.g., a mobile phone case) is 151×75 mm, the object information receiving unit can receive object information of 151×75 mm.

In step S1020, the size of the artificial intelligence image is determined based on the object information. Step S1020 can be executed by a size determination unit (not shown) of the processing module 220A. In some embodiments, step S1020 can be executed after step S1010. The size of the artificial intelligence image may include dimensions such as length and width. More specifically, when the object information received by executing step S1010 is a first size and a second size, respectively, the processing module 220A can determine the size of the artificial intelligence image to be the first size and the second size by executing step S1020. In some embodiments, the size of the artificial intelligence image can be in pixels. In a specific example, when the received object information is 151×75 mm, the processing module 220A can determine the size of the artificial intelligence image to be 2000 pixels×2000 pixels by executing step S1020. That is, the image outline frame of the artificial intelligence image can be set to 2000 pixels×2000 pixels.

In some embodiments, the size ratio of the first display area 610 as shown in Figure 6 may depend on the physical size of the object. More specifically, after executing step S1010 and step S1020, the processing module 220A may automatically adjust the size ratio of the first display area 610.

Thus, through steps S1010 and S1020 as shown in FIG. 10 , the size of the AI image can be further set according to the physical size of the object, so that the user can directly generate an AI image that is substantially the same size as the physical size of the object, thereby intuitively evaluating whether the generated AI image meets the user's expectations (including the size, position, and proportion of each element in the AI image).

Please refer to FIG11, which is a detailed flowchart of another example of the present application for generating a keyword character set based on at least one input image (i.e., step S320). That is, step S320 shown in FIG3 may include steps S1110, S1120, S1130, S1140, and S1150, and step S320 may be completed by executing steps S1110, S1120, S1130, S1140, and S1150, wherein steps S1110, S1120, S1130, S1140, and S1150 may be executed by the processing module 220A shown in FIG2.

In step S1110, a decorative image (specifically, at least one decorative image) is received. Step S1110 may be executed by the image receiving unit 221A of the processing module 220A as shown in FIG2 . In some embodiments, step S1110 may be executed after step S310. In some embodiments, step S1110 and step S310 may be executed simultaneously. In some embodiments, step S1110 may be similar to step S310 shown in FIG3 , and the difference between step S1110 and step S310 shown in FIG3 is that what is received by executing step S1110 is a decorative image. In some embodiments, the decorative image may refer to an image that includes a single object and has been background-removed, such as a basketball, a microphone, a doll, a necklace, a hat, or sunglasses.

In step S1120, the input image and the decorative image are subjected to a composition operation (specifically, at least one composition operation) and an arrangement image is generated. Step S1120 may be executed by an image arrangement unit (not shown) of the processing module 220A. In some embodiments, step S1120 may be executed after step S1110. The composition operation includes adjusting the size and/or position of the input image and/or the decorative image, but is not limited thereto. In some embodiments, the user can manually operate the processing module 220A to perform composition operations on the input image and the decorative image. In a specific example, the user can operate a smart phone and use a finger to select the input image and the decorative image to perform composition operations such as moving, flipping, and scaling. In another specific example, the user can also operate a personal computer and use a mouse to perform composition operations on the input image and the decorative image, thereby allowing the user to generate an arrangement image that better meets the user's expectations. In other embodiments, the processing module 220A can automatically perform composition operations on the input image and the decorative image, thereby generating an arrangement image more quickly and providing the user with a possible arrangement image. More specifically, the processing module 220A can first analyze the string content corresponding to the input image and the string content corresponding to the decorative image, then adjust the content of the string content to automatically generate new string content, and generate an image corresponding to the new string content (i.e., a layout image) based on the new string content.

In step S1130, the layout image is input to the image description model 270. Step S1130 can be executed through the string generation unit 222A and the image description model 270 of the processing module 220A as shown in FIG2. In some embodiments, step S1130 can be executed after step S1120. In some embodiments, step S1130 can be similar to step S410A shown in FIG4A, and the difference between step S1130 and step S410A shown in FIG4A is that the generated layout image is input by executing step S1130.

In step S1140, a keyword character set is generated through the image description model 270. Step S1140 can be executed through the string generation unit 222A and the image description model 270 of the processing module 220A as shown in FIG2. In some embodiments, step S1140 can be executed after step S1130. In some embodiments, step S1140 can be substantially the same as step S420A shown in FIG4A.

In step S1150, the keyword character set is outputted through the image description model 270. Step S1150 may be executed through the string generation unit 222A and the image description model 270 of the processing module 220A as shown in FIG2 . In some embodiments, step S1150 may be executed after step S1140. In some embodiments, step S1150 may be substantially the same as step S430A shown in FIG4A .

In some embodiments, step S1140 and step S1150 may be integrated into one step, that is, generating and outputting keyword character groups through the image description model 270.

Thus, by executing the steps shown in FIG. 11 , the user can generate more diverse artificial intelligence images without having to input additional command sets.

Please refer to FIG. 12, which is a flow chart of a method for generating an image according to the fourth embodiment of the present application. More specifically, the method shown in FIG. 12 can generate an artificial intelligence image, and the method includes steps S310, S320, S330, S340 and S1210. Steps S310, S320, S330 and S340 are substantially the same as the steps shown in FIG. 3, that is, the method shown in FIG. 12 may include steps S310, S320, S330 and S340 substantially the same as those in FIG. 3, and further includes step S1210.

In step S1210, the artificial intelligence image is transmitted to the image output device, and the artificial intelligence image is physically outputted through the image output device. Step S1210 can be executed through the image output unit 226A of the processing module 220A as shown in FIG. 2 . In some embodiments, step S1210 can be executed after step S340. More specifically, when the processing module 220A executes step S1210, the artificial intelligence image can be output to the image output device 310 through the image output module 240, so that the image output device 310 can print the artificial intelligence image on a physical object.

Thus, through step S1210 as shown in FIG. 12 , the artificial intelligence image generated by the processing module 220A can be further output and printed on a physical object, so that the artificial intelligence image can be more widely used in various fields of physical printing needs such as garment printing, automobile paint printing, mobile phone case printing or stickers.

Please refer to FIG. 13, which is a block diagram of a processing module 220B of another embodiment of the present application. The configuration of the processing module 220B may be substantially the same as the processing module 220 shown in FIG. 1, except that the processing module 220B may be specifically configured to generate a style conversion image based on the input image, and the function and configuration of generating an artificial intelligence image based on the input image may be omitted.

The processing module 220B can be basically configured to include an image receiving unit 221B, a style conversion request receiving unit 223B, a style conversion unit 224B and an image output unit 226B. In some embodiments, the processing module 220B can further include a style conversion model 290, that is, the style conversion model 290 can be selectively configured in the processing module 220B according to the needs of the user. In some embodiments, the function and configuration of the style conversion model 290 can be integrated into the style conversion unit 224B.

The image receiving unit 221B may be configured to be suitable for receiving an input image (specifically, at least one input image). In some embodiments, the function and configuration of the image receiving unit 221B may be substantially the same as the image receiving unit 221A shown in FIG. 2 .

The style conversion request receiving unit 223B can be configured to receive a style conversion request sent by a style conversion button (such as the style conversion button 1630 shown in Figure 16), that is, when the user clicks any one of at least one style conversion button, the clicked style conversion button will send a corresponding style conversion request, so that the style conversion request receiving unit 223B can receive the style conversion request corresponding to the clicked style conversion button.

The style conversion unit 224B may be configured to input an input image into a style conversion model 290, and to generate and output a style conversion image through the style conversion model 290. In some embodiments, the style conversion model 290 may be, for example, AdaAttN (Adaptive Attention Normalization), but is not limited thereto. The AdaAttN may learn shallow and deep features from a learning image, and calculate weighted statistics and normalization of each point so that the analysis results can show the same local feature statistics. In addition, the AdaAttN derives a new local feature loss to enhance local visual quality. Thereby, the AdaAttN is able to perform a series of calculations and analyses on the learning image, and based on the features learned from the learning image, convert the input image into a style-converted image having the features.

In some embodiments, the style conversion model 290 can perform a style conversion operation on the input image based on the style learning result of the learning image (specifically, at least one learning image) and generate a style conversion image. More specifically, by inputting the learning image into the style conversion model 290 in advance, the style conversion model 290 can imitate the characteristics of the learning image, thereby the style conversion model 290 can convert the input image into a style conversion image having the characteristics of the learning image based on the characteristics of the learning image.

In some embodiments, the learning image may be an artificial intelligence image generated by any of the methods for generating images described above. Thus, the style conversion model 290 may imitate the characteristics of the artificial intelligence image, so that the style conversion model 290 can convert the input image into a style conversion image having the characteristics of the artificial intelligence image based on the characteristics of the artificial intelligence image.

The image output unit 226B may be configured to output the style conversion image generated by the style conversion unit 224B. In some embodiments, the image output unit 226B may store the style conversion image in a specific file format in the image database 250. In some embodiments, the image output unit 226B may output the style conversion image in a specific file format to the image output device 310 and/or the server 320 through the image output module 240.

Thus, the processing module 220B shown in FIG13 can not only generate a style conversion image based on the received input image, but also generate the style conversion image without the user having to input an additional command set. In other words, the computer device for generating images provided by the present application can reduce the user's operating threshold and/or operating difficulty, thereby enabling users who are not familiar with the command set to generate the image file desired by the user through the computer device.

Please refer to FIG. 14, which is a flow chart of a method for generating an image according to the fifth embodiment of the present application. More specifically, the method shown in FIG. 14 can generate a style conversion image, and the method includes steps S1410, S1420, S1430, and S1440. In some embodiments, the method shown in FIG. 14 can use Visual Studio Code as a development environment and use the framework of pytorch with python as the programming language to implement the method. In some embodiments, the method shown in FIG. 14 can use open source libraries such as torch, itertools, and numpy.

In step S1410, an input image (specifically, at least one input image) is received. Step S1410 may be performed by the image receiving unit 221B of the processing module 220B as shown in FIG13. In some embodiments, step S1410 may be substantially the same as step S310 shown in FIG3.

In step S1420, after receiving a style conversion request sent from any one of the style conversion buttons (specifically, at least one style conversion button), the input image is input to the style conversion model 290 based on the style conversion request. Step S1420 can be executed through the style conversion request receiving unit 223B and the style conversion model 290 of the processing module 220B as shown in Figure 13. In some embodiments, step S1420 can be executed after step S1410.

In some embodiments, the style conversion buttons may be pre-established, so that the user can perform style conversion operation on the input image through any one of the style conversion buttons. In a specific example, the style conversion buttons may include a first style conversion button (the label content is displayed as ink painting style), a second style conversion button (the label content is displayed as abstract style) and a third style conversion button (the label content is displayed as sketch style). When the user clicks the third style conversion button, the third style conversion button may issue a style conversion request, so that after the style conversion request receiving unit 223B receives the style conversion request sent by the third style conversion button, the processing module 220B may input the input image into the style conversion model 290. In some embodiments, the style transfer model 290 may be, for example, AdaAttN (Adaptive Attention Normalization), but is not limited thereto.

In step S1430, a style-converted image is generated through the style-converted model 290. Step S1430 may be performed through the style-converted model 290 of the processing module 220B as shown in FIG. 13. In some embodiments, step S1430 may be performed after step S1420. The style-converted model 290 may convert the input image into a style-converted image having the features learned from the learning image based on the features learned from the learning image, thereby generating the style-converted image.

In step S1440, the style conversion image is outputted through the style conversion model 290. Step S1440 can be executed through the style conversion model 290 of the processing module 220B as shown in FIG13. In some embodiments, step S1440 can be executed after step S1430. That is, by executing step S1440, the processing module 220B can further perform subsequent processing (for example, step S1710 shown in FIG17) on the style conversion image generated and outputted by the style conversion model 290.

Thus, the method shown in FIG. 14 can not only generate a style conversion image based on the received input image, but also generate the style conversion image without the user having to input an additional command set. In other words, the computer device for generating images provided by the present application can reduce the user's operating threshold and/or operating difficulty, thereby enabling users who are not familiar with the command set to generate the image file desired by the user through the computer device.

Please refer to Figure 15, the source of Figure 15 is: Liu, S., Lin, T., He, D., Li, F., Wang, M., Li, X., ... & Ding, E. (2021). Adaattn: Revisit attention mechanism in arbitrary neural style transfer. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 6649-6658). Figure 15 is a schematic diagram illustrating the result of converting at least one input image into different style conversion images of the present application.

The input image may refer to any one of the input images 1501X to 1508X shown in FIG. 15 , but is not limited thereto. The learning image may refer to any one of the learning images 1501Y to 1506Y shown in FIG. 15 , but is not limited thereto. In a specific example, when the received input image is the input image 1508X and the user clicks the fourth style conversion button (the label content is displayed as a sketch style), the style conversion model 290 may convert the input image 1508X into a style conversion image 15086Z having the features based on the features learned from the learning image 1506Y.

Similarly, when the received input image is another input image, the style conversion model 290 can also convert the input image into a style conversion image having the features based on the features learned from the learning image corresponding to the style conversion button according to the style conversion button clicked by the user. For example, the style conversion model 290 can convert the input image 1508X into style conversion images 15081Z to style conversion images 15085Z having the features based on the features learned from the learning images 1501Y to 1505Y, respectively. Similarly, other input images 1501X to 1507X can also be converted into their corresponding style-converted images.

Please refer to FIG. 16, which is a schematic diagram of a display screen 1600 on a user device for illustrating another embodiment of the present application. The display screen 1600 on the user device may basically include a first display area 1610 and a style conversion button 1630 (specifically, at least one style conversion button 1630). It should be noted that the configuration relationship (e.g., relative size and/or relative position) of each element shown in FIG. 16 can be adjusted according to the needs of the user, that is, the configuration relationship shown in FIG. 16 is only an example of schematic illustration.

The style conversion buttons 1630 allow the user to perform style conversion operations on the input image by clicking any one of the style conversion buttons 1630. In some embodiments, the style conversion buttons 1630 may be pre-established. In other embodiments, the style conversion buttons 1630 may be automatically established based on at least one of the content of the character group corresponding to the input image and the user's historical editing record. In some embodiments, the number of style conversion buttons 1630 may be plural, so as to provide the user with a better user experience by clicking a suitable style conversion button 1630. In some embodiments, by editing the label content corresponding to each style conversion button 1630, the user can more intuitively know the expected results of the style conversion operation performed by each style conversion button 1630 on the input image based on the label content.

In some embodiments, the first display block 1610 may be configured to display a style conversion image. More specifically, after the user clicks any one of the style conversion buttons 1630, the user may be aware of the generated style conversion image through the first display block 1610. Thus, through the display content of the first display block 1610, the user may evaluate whether the current style conversion image meets the user's expectations, and decide whether it is necessary to regenerate a new style conversion image through the style conversion button 1630.

Please refer to FIG. 17, which is a flow chart of a method for generating an image according to the sixth embodiment of the present application. More specifically, the method shown in FIG. 17 can generate a style conversion image, and the method includes steps S1410, S1420, S1430, S1440, and S1710. Steps S1410, S1420, S1430, and S1440 are substantially the same as those shown in FIG. 14, that is, the method shown in FIG. 17 may include steps S1410, S1420, S1430, and S1440 substantially the same as those in FIG. 14, and further includes step S1710.

In step S1710, the style conversion image is transmitted to the image output device, and the style conversion image is physically outputted by the image output device. In some embodiments, step S1710 may be executed after step S1440. In some embodiments, step S1710 may be similar to step S1210 shown in FIG. 12 , and the difference between step S1710 and step S1210 shown in FIG. 12 is that the generated style conversion image is outputted by executing step S1710.

Thus, through step S1710 as shown in FIG. 17 , the style conversion image generated by the processing module 220B can be further output and printed on a physical object, so that the style conversion image can be more widely used in various physical printing demand fields such as garment printing, automobile paint printing, mobile phone case printing or sticker printing.

In some embodiments, each step of the method for generating an image as described above may be stored in a computer-readable recording medium, which may be, for example, a hard disk, an optical disk, a magnetic disk, a flash drive, or a database accessible via a network, but is not limited thereto. The computer-readable recording medium can implement any of the methods for generating an image as described above after the computer device loads the program code into the memory and executes the program code.

In some embodiments, a computer program product for generating an image may include the various steps of the method for generating an image as described above, so that after loading and executing the computer program product, a computer device can implement any of the methods for generating an image as described above.

This application has been further explained through the above-mentioned embodiments and the attached drawings, but those with ordinary knowledge in the technical field to which this application belongs can still make many modifications and changes without violating the scope and spirit proposed in the scope of the patent application of this application. Therefore, the scope of protection of this application should still be based on the scope of the patent application and should not be limited by the content disclosed in the specification.

110: User device 120: Server 200: Computer device 210: Image receiving module 220, 220A, 220B: Processing module 221A, 221B: Image receiving unit 222A: String generating unit 223A: Image editing request receiving unit 223B: Style conversion request receiving unit 224A: String editing unit 224B: Style conversion unit 225A: Image generating unit 226A, 226B: Image output unit 230: Storage module 240: Image output module 250: Image database 260: Editing database 270: Image description model 280: Image generating model 290: Style conversion model 310: Image output device 320: Server 510: Input image 520: Keyword character group 530: Edit character group 540: Artificial intelligence image 600: Display screen 610: First display block 620: Second display block 630: Image editing button 830: Edit character group 840: Artificial intelligence image 930: Edit character group 940: Artificial intelligence image 1501X~1508X: Input image 1501Y~1506Y: Learning image 15081Z~15086Z: Style conversion image 1600: Display screen 1610: First display area 1630: Style conversion button S310, S320, S330, S340: Step S410A, S420A, S430A : Step S410B, S420B, S430B : Step S710: Step S1010, S1020 : Step S1110, S1120, S1130: Step S1140, S1150: Step S1210: Step S1410, S1420, S1430, S1440: Step S1710: Step

FIG. 1 is a schematic diagram illustrating a connection relationship in which a computer device of an embodiment of the present application is signal-connected to at least one of a user device and a server and is signal-connected to at least one of an image output device and a server. FIG. 2 is a block diagram illustrating a processing module of an embodiment of the present application. FIG. 3 is a flow chart illustrating a method for generating an image of a first embodiment of the present application. FIG. 4A is a detailed flow chart illustrating a method of generating a keyword character group based on at least one input image of an example of the present application. FIG. 4B is a detailed flow chart illustrating a method of generating an artificial intelligence image based on an edited character group of an example of the present application. FIG. 5 is a schematic diagram illustrating a result generated after executing each step of an example of the present application. FIG6 is a schematic diagram of a display screen on a user device illustrating an embodiment of the present application. FIG7 is a flow chart illustrating a method for generating an image in a second embodiment of the present application. FIG8 is a schematic diagram illustrating the results generated after executing each step of another example of the present application. FIG9 is a schematic diagram illustrating the results generated after executing each step of another example of the present application. FIG10 is a flow chart illustrating a method for generating an image in a third embodiment of the present application. FIG11 is a detailed flow chart illustrating another example of the present application for generating a keyword character set based on at least one input image. FIG12 is a flow chart illustrating a method for generating an image in a fourth embodiment of the present application. FIG. 13 is a block diagram of a processing module of another embodiment of the present application. FIG. 14 is a flow chart of a method for generating an image of the fifth embodiment of the present application. FIG. 15 is a diagram of the result of converting at least one input image into different style conversion images of the present application. FIG. 16 is a diagram of a display screen on a user device of another embodiment of the present application. FIG. 17 is a flow chart of a method for generating an image of the sixth embodiment of the present application.

S310: Step

S320: Step

S330: Step

S340: Steps

Claims (22)

A method for generating an image, which is executed after a computer program product is loaded and executed by a computer device to generate an artificial intelligence image, the method comprising: receiving at least one input image; generating a keyword character set based on the at least one input image; automatically generating at least one editing button based on the content of the keyword character set and at least one of a historical editing record of a user; after receiving a editing request sent from any one of the at least one editing buttons, performing at least one string editing operation on the keyword character set based on an editing instruction set corresponding to the editing request, and generating an editing character set; and generating the artificial intelligence image based on the editing character set. The method of claim 1, wherein generating the keyword character set based on the at least one input image comprises: inputting the at least one input image into an image description model; and outputting the keyword character set through the image description model, wherein the image description model automatically generates the keyword character set corresponding to the at least one input image based on the at least one input image. The method as claimed in claim 1, wherein generating the artificial intelligence image based on the edited character set comprises: inputting the edited character set into an image generation model; and outputting the artificial intelligence image through the image generation model, wherein the image generation model automatically generates the artificial intelligence image corresponding to the edited character set based on the edited character set. The method as described in claim 1 further includes: Determining whether to stack the at least one string editing operation based on a stacking attribute value corresponding to each of the at least one photo editing button. A method as described in claim 4, wherein the stack attribute value is set to stackable or non-stackable. A method as described in claim 4, wherein the stack attribute value is set based on at least one of a preference setting of a user and a default value. The method as claimed in claim 1 further comprises: Determining the editing degree of the keyword character group by the at least one string editing operation based on an editing weight value corresponding to each of the at least one editing button. A method as described in claim 7, wherein the editing weight value is set based on at least one of the content of the keyword character group, a historical editing record of a user, and a default value. The method as described in claim 1 further includes: receiving object information corresponding to the physical size of an object; and determining the size of the artificial intelligence image based on the object information. The method of claim 1, wherein generating the keyword character set based on the at least one input image comprises: receiving at least one decorative image; performing at least one composition operation on the at least one input image and the at least one decorative image, and generating an arrangement image; inputting the arrangement image into an image description model; and outputting the keyword character set through the image description model, wherein the image description model automatically generates the keyword character set corresponding to the arrangement image based on the arrangement image. The method described in claim 1 further includes: Transmitting the artificial intelligence image to an image output device, and physically outputting the artificial intelligence image through the image output device. A method for generating an image, which is executed after a computer program product is loaded and executed by a computer device to generate an artificial intelligence image, the method comprising: receiving at least one input image; generating a keyword character set based on the at least one input image; after receiving a photo editing request sent from any one of at least one photo editing buttons, performing at least one string editing operation on the keyword character set based on an editing instruction set corresponding to the photo editing request, and generating an editing character set; generating the artificial intelligence image based on the editing character set; and determining whether to stack the at least one string editing operation based on a stacking attribute value corresponding to each of the at least one photo editing buttons. A method as claimed in claim 12, wherein the stack attribute value is set to stackable or non-stackable. A method as described in claim 12, wherein the stack attribute value is set based on at least one of a preference setting of a user and a default value. A method for generating an image, which is executed after a computer program product is loaded and executed by a computer device to generate an artificial intelligence image, the method comprising: receiving at least one input image; generating a keyword character group based on the at least one input image; after receiving a photo editing request sent from any one of at least one photo editing buttons, performing at least one string editing operation on the keyword character group based on an editing instruction set corresponding to the photo editing request, and generating an edited character group; generating the artificial intelligence image based on the edited character group; and determining the editing degree of the keyword character group by the at least one string editing operation based on an editing weight value corresponding to each of the at least one photo editing buttons, The editing weight value is set based on at least one of the content of the keyword character group, a historical editing record of a user, and a default value. A method for generating an image, the method is executed after a computer program product is loaded and executed by a computer device to generate an artificial intelligence image, the method comprising: receiving at least one input image; generating a keyword character set based on the at least one input image; after receiving a photo editing request sent from any one of at least one photo editing buttons, performing at least one string editing operation on the keyword character set based on an editing instruction set corresponding to the photo editing request, and generating an edited character set; and generating the artificial intelligence image based on the edited character set; wherein generating the keyword character set based on the at least one input image includes: receiving at least one decorative image; Perform at least one composition operation on the at least one input image and the at least one decorative image to generate an arrangement image; Input the arrangement image to an image description model; and Output the keyword character set through the image description model, Wherein, the image description model automatically generates the keyword character set corresponding to the arrangement image based on the arrangement image. A method for generating an image, the method is executed after a computer program product is loaded and executed by a computer device to generate a style conversion image, the method comprising: receiving at least one input image; and after receiving a style conversion request sent from any one of at least one style conversion buttons, inputting the at least one input image into a style conversion model based on the style conversion request, and outputting the style conversion image through the style conversion model, wherein the style conversion model performs at least one style conversion operation on the at least one input image based on the style learning result of at least one learning image, and generates the style conversion image, Wherein, the at least one learning image is generated by a method for generating an image as described in any one of claims 1 to 10 and 12 to 16. The method as described in claim 17 further includes: Transmitting the style conversion image to an image output device, and physically outputting the style conversion image through the image output device. A computer device for generating an image, adapted to be connected to a user terminal signal to receive at least one input image from the user terminal, and to generate an output image based on the at least one input image, the computer device comprising: an image receiving module, configured to be connected to the user terminal signal, adapted to receive the at least one input image; a processing module, configured to be connected to the image receiving module signal; and a storage module, configured to be connected to the processing module signal, wherein the storage module stores a program code, and After executing the program code stored in the storage module, the processing module can execute the method for generating an image as described in any one of claim items 1 to 10 and 12 to 16 to generate an artificial intelligence image, or can execute the method for generating an image as described in claim item 17 to generate a style conversion image, and use the artificial intelligence image or the style conversion image as the output image. The computer device as described in claim 19 further comprises: An image output module, configured to be signal-connected to the processing module and adapted to transmit the output image to an image output device. A computer-readable recording medium for generating an image, which can complete the method for generating an image as described in any one of claims 1 to 18 after a computer device loads a program code into a memory and executes the program code. A computer program product for generating an image. When a computer device loads and executes the computer program product, it can complete the method for generating an image as described in any one of claims 1 to 18.