JP2013222229A - Input operation device, image forming apparatus including the device, input operation method, and input operation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an input operation device that reduces the number of operation steps by eliminating the need of operations such as depression of a voice recognition key and preliminary level settings, and allows intuitive operation to improve operability.SOLUTION: An input operation device performs an operation set in advance by associating the operation with any of the type of a screen that is displayed on display means 171 when a touch operation is made on a touch panel, a position of the touch operation on the touch panel, and the type of the touch operation, and the content of voice input received by voice input reception means 11.

Description

  The present invention relates to an input operation device used for an operation panel or the like of an image forming apparatus for a user to perform an input operation, an image forming apparatus including the apparatus, an input operation method, and an input operation program.

  With the widespread use of smartphones, tablet PCs, and the like that enable intuitive operation, intuitive operability is required for various devices. Typical examples of operations include flicks and pinches called multi-touch operations.

  When previewing a saved document in an image forming apparatus or the like, when the page is fed by the flick operation, the page is basically page-by-page, and an arbitrary page is desired to be instantaneously accessed. However, there is a problem that a setting is required in advance or a bar display of page display is necessary, and the display area is narrowed. Also, when pinching in / out and reducing / enlarging the display, it is difficult to adjust to the optimum size at once, and there is a problem that the level setting key is required and the display is narrowed like the brick. .

  Also, basically, if you try to realize additional requests such as level setting and page designation by performing multi-touch operation, your finger is blocked in the first operation, so additional operation is required after that, and more than 2 steps There is also a problem that this operation becomes necessary.

  On the other hand, operation by voice recognition is one of the operation methods that can be intuitively operated. In speech recognition, if speech recognition processing is executed from beginning to end, the CPU is occupied, and other processing becomes heavy.

  In order to solve the above-described problem of operation by voice recognition, as described in Patent Document 1, it is common to input voice in a state in which voice input can be accepted by pressing a voice recognition key. After the voice recognition key is pressed, an intuitive operation can be performed by voice without performing a finger operation.

JP 2006-167916 A

  However, since it is necessary to press the voice recognition key before voice input, there is a problem that one-step operation increases. It is also possible to combine voice recognition with intuitive operation with a finger. In this case, too, after pressing the voice recognition key, voice input is performed, and specific numerical values such as level settings are input. There is a problem that the number of operation steps is similarly increased, and it is not an intuitive operation, and the operability is not good.

  The present invention has been made in view of such a technical background, and does not require operations such as pressing a voice recognition key or setting a level in advance, thereby reducing the number of operation steps and enabling an intuitive operation. It is an object of the present invention to provide an input operation device with improved operability, and to provide an image forming apparatus, an input operation method, and an input operation program including the device.

The above problem is solved by the following means.
(1) It is determined that the touch operation is performed by a display unit having a touch panel, a determination unit that determines whether or not a touch operation is performed by a user on the touch panel of the display unit, a voice input unit, and the determination unit. The voice input receiving means for receiving voice input by the voice input means; the type of screen displayed on the display means when the touch operation is performed; the position of the touch operation on the touch panel; An input operation device comprising: control means for executing a preset operation in association with at least one of the types and the content of the voice input received by the voice input receiving means.
(2) Storage means in which at least one of the screen type, the position of the touch operation, the type of touch operation, the keyword, and the operation content are associated and stored, and the control means includes the voice input means It is determined whether or not the content of the voice input by the word matches the keyword stored in the storage means, and if it matches, at least one of the keyword and the type of the screen, the position of the touch operation, the type of the touch operation 2. The input operation device according to item 1, wherein the operation associated with any one of the operations is executed.
(3) The type of touch operation may be flicking with a finger while touching the touch panel lightly, dragging with a finger sliding with a light touch on the touch panel, or touching the touch panel with a finger. A tap, a double tap that is a double-tapping operation with a finger, a multi-touch operation on the touch panel, and a pinch-in / pinch-out operation that slides the finger toward or away from the center. Rotation, which is the action of rotating around the center position of one or both fingers, or drag and drop, which is an action of sliding the finger with the touch panel lightly touched and releasing the finger at the target location 3. The input operation device according to item 1 or 2, which is at least one of them.
(4) The input operation device according to any one of the preceding items 1 to 3, wherein the voice input reception unit receives voice input until a preset time after the touch operation is completed.
(5) An image forming apparatus comprising the input operation device according to any one of items 1 to 4.
(6) An input operation method of an image forming apparatus including a display unit having a touch panel and a voice input unit, wherein a determination step of determining whether or not a touch operation is performed by a user on the touch panel; and the determination step The voice input receiving step for receiving voice input by the voice input means, the type of screen displayed on the display means when the touch operation is made, and the touch panel A control step of executing a preset operation in association with at least one of the position of the touch operation and the type of touch operation and the content of the voice input received in the voice input reception step. An input operation method characterized by that.
(7) An input operation program for causing a computer of an image forming apparatus including a display unit having a touch panel and an audio input unit to execute, and determining whether a touch operation by a user has been performed on the touch panel A voice input accepting step for accepting voice input by the voice input means when the touch operation is judged to have been displayed on the display means when the touch operation is made. Executes a preset operation in association with at least one of the type of screen, the position of the touch operation on the touch panel, the type of touch operation, and the contents of the voice input received in the voice input receiving step. And an input operation program for causing the computer to execute a control step. Lamb.

  According to the invention described in (1) above, when it is determined that a touch operation has been performed, voice input by the voice input unit is accepted, and the screen displayed on the display unit when the touch operation is performed is displayed. A preset operation is executed corresponding to at least one of the type, the position of the touch operation on the touch panel, the type of the touch operation, and the content of the voice input received by the voice input receiving unit.

  Therefore, since the user can execute the operation by touching the touch detection area on the touch panel of the display means and inputting the voice at the same time without pressing the voice recognition key as in the past, the user can press the voice recognition key. In addition, operations such as level setting and the like can be made unnecessary, the number of operation steps can be reduced, intuitive operation can be performed, and operability can be improved.

  According to the invention described in item (2) above, it is determined whether or not the content of the voice input by the voice input means matches the keyword stored in the storage means. Then, an operation associated with at least one of the position of the touch operation and the type of the touch operation is executed.

  According to the invention described in (3), the operation is executed based on at least one of the touch operations of flick, drag, tap, double tap, pinch in / pinch out, rotation, and drag and drop.

  According to the invention described in (4) above, since the voice input is accepted until a preset time after the touch operation is finished, the voice input is surely accepted even if the voice input is somewhat delayed from the touch operation. .

  According to the invention described in item (5), the number of operation steps can be reduced, an intuitive operation is possible, and an image forming apparatus that can improve operability can be obtained.

  According to the invention described in item (6) above, operations such as pressing of a voice recognition key and prior level setting can be eliminated, the number of operation steps can be reduced, and an intuitive operation of the image forming apparatus can be performed. Can be improved.

  According to the invention described in the preceding item (7), it is determined whether or not a touch operation has been performed by the user on the touch panel. When it is determined that the touch operation has been performed, the voice input by the voice input unit is accepted and the touch operation is performed. In association with at least one of the type of the screen displayed on the display means when it is made, the position of the touch operation on the touch panel, the type of the touch operation, and the content of the received voice input in advance It is possible to cause the computer of the image forming apparatus to execute processing for executing the set operation.

1 is a block diagram of an image forming apparatus including an input operation device according to an embodiment of the present invention. 3 is a flowchart illustrating an operation of the image forming apparatus. It is a figure which shows the relationship between the start and completion | finish timing of a touch operation, and an audio | voice input reception period. It is a flowchart which shows the type determination process of the touch operation in step S111 of FIG. It is a type explanatory view of touch operation. It is a table which shows the operation | movement preset by matching with the classification of the screen currently displayed on the display part when the touch operation to the touch panel was made, and the keyword input by voice. FIG. 7 is a table showing operations preset for the lower layer screen of one screen shown in the table of FIG. 6 in association with the type and the keyword input by voice. FIG. It is a table which shows a preset operation in association with a position of a touch operation when a touch operation on the touch panel is performed and a keyword input by voice. It is a table which shows the operation | movement set beforehand matched with the classification of the touch operation to a touchscreen, and the keyword input by voice. It is explanatory drawing in the case of performing different operation | movement according to the position in the display screen which performed the touch operation even if it is the same touch operation classification.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram illustrating an image forming apparatus 1 including an input operation device according to an embodiment of the present invention.

  In this embodiment, the image forming apparatus 1 uses an MFP (Multi Function Peripherals) which is a multifunction digital image forming apparatus that combines a plurality of functions such as a copy function, a printer function, a facsimile function, and a scan function. In the following description, the image forming apparatus is also referred to as an MFP.

  The MFP 1 includes a CPU 11, a ROM 12, a RAM 13, a scanner unit 14, a storage unit 15, a printer unit 16, an operation panel 17, a network controller 18, a voice input unit 19, and the like.

  The CPU 11 controls the entire MFP 1 in an integrated manner so that functions such as a copy function, a printer function, a facsimile function, and a scan function can be used. Further, in this embodiment, it is detected based on information from the operation panel that the user has touched the touch panel of the operation panel 17, and the voice input by the voice input unit can be accepted, or the received voice input can be received. The type of the screen displayed on the operation panel 17 when the voice recognition process is performed or the touch operation is performed, the position of the touch operation on the touch panel, the type of the touch operation, and the like are determined. Based on at least one set in advance and the content of the voice input, processing such as executing a preset operation or switching the screen of the operation panel 17 to another screen is executed. Will be described later.

  The ROM 12 is a memory that stores an operation program of the CPU 11, and the RAM 13 is a memory that provides a work area when the CPU 11 operates according to the operation program.

  The scanner unit 14 functions as a reading unit that reads an image of a document written on a sheet or the like and converts it into image data that is electronic data. Although not shown, the scanner unit 15 may include an automatic document feeder (ADF) that automatically conveys a plurality of documents to a reading position.

  The storage unit 15 is a memory that stores image data read by the scanner unit 15, other data, applications, and the like. For example, a hard disk device is used. In this embodiment, a first dictionary having a large number of keywords and a second dictionary further narrowing down keywords existing in the first dictionary are stored, and the type of touch operation and the like are registered. Yes. Further, at least one of the type of screen displayed on the operation panel 17, the position of the touch operation on the touch panel, the type of touch operation, and the keyword stored in the second dictionary are associated with each other. Thus, operations to be executed are preset and stored.

  The printer unit 16 prints image data of a document read by the scanner unit 15 or print data transmitted from an external device such as a personal computer on a sheet.

  The operation panel 17 is used for user operations, message display, and the like. The operation panel 17 includes a liquid crystal with a touch panel, a display unit 171 for displaying various messages, the state of the MFP 1, and the like, a numeric keypad, and a start key. A hard key unit 172 including keys and the like is provided.

  The network controller 18 functions as a communication unit for transmitting / receiving data to / from an external device such as a personal computer or a server.

  The voice input unit 19 is for inputting a voice uttered by a user, and is constituted by, for example, a microphone.

  FIG. 2 is a flowchart showing the operation of the MFP 1. This operation is executed by the CPU 11 operating in accordance with a program stored in the ROM 12.

  In step S11, it is detected that a touch operation on the touch panel of the display unit 171 is started, and in step S111, a touch operation type determination process is executed.

  Next, in step S12, it is determined based on the type determination result whether the touch operation is an operation type registered in the storage unit 15.

  If the touch operation is not a registered operation type (NO in step S12), an operation corresponding to the touch operation is executed in step S132, and then the process returns to step S11. If it is a touch operation of the registered operation type (YES in step S12), in step S131, the voice input unit 19 is turned on and reception of voice input is started.

  Therefore, as shown in FIG. 3, acceptance of voice input is started almost simultaneously with the start of the touch operation. Also, the voice input is accepted until a timing T2 that is later than the detection end timing T1 of the touch operation. In other words, the voice input acceptance period is longer than the touch operation detection period by (T2-T1), and this period is a grace period. This grace period (T2-T1) is a preset value.

  Next, in step S14, it is determined whether or not there is a voice input. If not (NO in step S14), the process proceeds to step S152. If there is a voice input (YES in step S14), in step S151, the voice recognition process is performed, and the content obtained by the process is stored in the storage unit 15 in the first dictionary. Determine if they match. If they do not match (NO in step S151), the process proceeds to step S152. If they match (YES in step S151), the process proceeds to step S16.

  In step S <b> 16, it is determined whether or not the content obtained by the voice recognition process matches the keyword described in the second dictionary stored in the storage unit 15. If they do not match (NO in step S16), the process proceeds to step S152. If they match (YES in step S16), in step S17, at least one of the current screen type displayed on the display unit 171, the position of the touch operation on the touch panel, and the type of touch operation is set in advance. A preset operation corresponding to the keyword is executed.

  On the other hand, in step S152, it is determined whether or not the touch operation with the finger is finished. If the touch operation has not ended (NO in step S152), the process returns to step S14. If completed (YES in step S152), the process returns to step S11.

  As described above, when it is detected that the touch operation has been performed, the voice input by the voice input unit 19 is accepted, and the type of screen displayed on the display unit 171 when the touch operation is performed, on the touch panel Since a preset operation is executed in accordance with at least one of the position of the touch operation and the type of touch operation and the content of the voice input received by the voice input unit 19, As described above, the operation can be performed by touching the touch panel of the display unit 171 and inputting the voice at the same time without pressing the voice recognition key. For this reason, operations such as pressing a voice recognition key and setting a prior level can be made unnecessary, the number of operation steps can be reduced, an intuitive operation of the MFP 1 can be performed, and operability can be improved.

  In the flowchart of FIG. 2, it is determined whether or not the content of the voice input matches the keyword of the first dictionary, and if it matches, it is determined whether or not it matches the keyword of the second dictionary. However, one dictionary corresponding to the second dictionary may be provided.

  FIG. 4 is a flowchart showing the touch operation type determination processing in step S111 of FIG. This process is determined based on detection outputs sequentially transmitted from the operation panel 17.

  In step S221, it is determined whether or not the first detection output from the operation panel 17 is a light touch on the touch panel, and if it is a light touch on the touch panel (YES in step S221), In step S2211, it is determined whether or not the second detection output is a detection of a finger being played. If it is detected that the finger has been played (YES in step S2211), the process returns to step S22111 after determining that the touch operation is a flick (swipe). As shown in FIG. 5A, this operation is used, for example, to change the one-page display to the two-page display, page feed, or the like. It should be noted that the flick operation may be divided into a plurality of types of operations by determining whether the flick operation finger is slow or fast.

  In step S2211, if the second detection output is not a detection that the finger has been flipped (NO in step S2211), it is determined in step S2212 whether the second detection output is a detection of slipping the finger. To do. If it is detected that the finger is slipped (YES in step S2212), it is determined in step S22121 that the touch operation is a drag (pan), and then the process proceeds to step S22122, where the third detection output is released. It is determined whether or not this is a detection. If it is not detected that the finger has been released (NO in step S22122), the process waits until it is detected that the finger has been released (YES in step S22122), and if detected, the touch operation is determined to be a drop in step S22123. Then return. That is, it is determined as drag and drop. This operation is used, for example, when a display document is stored in a predetermined folder icon as shown in FIG.

  In step S2212, if the second detection output is not detection of sliding the finger (NO in step S2212), it is determined in step S2213 whether the second detection output is detection of releasing the finger. To do. If it is not detected that the finger has been released (NO in step S2213), the process returns to step S2211. If it is detected that the finger has been released (YES in step S2213), the touch operation is determined to be single touch in step S2211, and then the process returns.

  If the first detection output is not a light touch on the touch panel in step S221 (NO in step S221), the first detection output is a detection that the touch panel is struck with a finger in step S222. If it is detected that a finger has been struck (YES in step S222), it is determined in step S2221 whether or not the second detection output is a struck finger. . If the second detection output is not a detection that the finger has been struck (NO in step S2221), it is determined in step S2211 that the touch operation is a single tap, and then the process returns. This operation is used for pressing various keys, for example, as shown in FIG.

  If it is detected in step S 2221 that the second detection output is a finger strike (YES in step S 2221), the process returns to step S 22212 after determining that the touch operation is a double tap. This operation is used, for example, for enlarging or reducing the screen as shown in FIG.

  If it is determined in step S222 that the first detection output is not a detection that the touch panel has been struck with a finger (NO in step S222), it is determined in step S223 whether the first detection output is multi-touch.

  If it is multi-touch (YES in step S223), it is determined in step S2231 whether or not the second detection output is detection of a finger contracted. If the second detection output indicates that the finger has been contracted (YES in step S2231), the process returns to step S22311 after determining that the touch operation is a pinch-in. On the other hand, if the second detection output is not a detection that the finger is contracted (NO in step S2231), it is determined in step S2232 whether or not the second detection output is a detection that the finger is expanded. If the second detection output is a detection that the finger is spread (YES in step S2232), the process returns to step S22321 after determining that the touch operation is a pinch out.

  These pinch-in / pinch-out operations are used, for example, to partially reduce or enlarge a part of the screen as shown in FIG.

  In step S2232, if the second detection output is not a detection that the finger is spread (NO in step S2232), it is determined in step S2233 whether the second detection output is a rotation detection. If rotation is detected (YES in step S2233), in step S22331, it is determined that the touch operation is rotation, and then the process returns. This operation is used, for example, to rotate a part of the screen as shown in FIG.

  If it is determined in step S2233 that the second detection output is not rotation detection (NO in step S2233), it is determined in step S2234 whether the second detection output is a detection that a finger has been released. If it is detected that the finger has been released (YES in step S2234), the process returns to step S22341 after determining that the touch operation is multi-touch. If it is not detected that the finger has been released (NO in step S2234), the process returns to step S2231.

  If the first detection output is not multi-touch in step S223 (NO in step S223), the touch operation is determined as another operation in step S224, and the process returns. In this case, it is determined that the operation type is not registered in step S12 of FIG.

  FIG. 6 is a table showing operations set in advance in association with the type of screen displayed on the display unit 171 when a touch operation on the touch panel is performed and a keyword input by voice.

  In the example shown in FIG. 6, the screen types are “copy screen”, “scan screen”, “box screen”, and “browser screen”, and the keywords input by voice are “start”, “last”, “1”, “2”, and “200%”. Is illustrated. In this example, the association by the type of touch operation is not made.

  For example, when the user touches an arbitrary area of the touch panel and inputs a voice “start” while the copy screen is displayed on the display unit 171, the CPU 11 displays the display unit based on the table in FIG. The screen of 171 is changed to the top screen. In addition, when the user touches an arbitrary area of the touch panel and inputs a voice “start” while the browser screen is displayed on the display unit 171, the screen of the display unit 171 is moved to the top of the page. .

  Similarly, when the user touches an arbitrary area of the touch panel and inputs a sound “1” in a state where the copy screen is displayed on the display unit 171, the number of copies is set to 1. In addition, when the user touches an arbitrary area of the touch panel and inputs the voice “200%” while the browser screen is displayed on the display unit 171, the screen display of the display unit 171 is enlarged to 200%. To do.

  Note that some keywords are not accepted depending on the combination of screen type and keyword. For example, when a copy screen is displayed on the display unit 171, even if the user touches an arbitrary area on the touch panel and inputs a voice “Last”, it is not accepted, and the screen of the display unit 171 remains as it is. Maintain the state.

  The keywords as described above are stored for each screen type and stored in the second dictionary. For example, the keyword related to the copy screen is the pattern A of the second dictionary, the keyword related to the scan screen is the pattern B of the second dictionary, the keyword related to the box screen is the pattern C of the second dictionary, Keywords related to the browser screen are stored in pattern D of the second dictionary, and keywords related to other screens are stored in pattern E. When the CPU 11 determines the type of the screen, the CPU 11 searches the second dictionary to search for a keyword input by voice to a pattern related to the screen, specifies the screen type and the operation associated with the keyword, and executes it. It has been made.

  In addition, each screen of the “copy screen”, “scan screen”, “box screen”, and “browser screen” has one or more lower layer screens, and each lower layer screen is also associated with each keyword. Is set.

  For example, as shown in FIG. 7, each operation is set in association with keywords such as “destination selection screen”, “thumbnail display screen” and “1”, “2”, etc., which are lower layer screens of the scan screen. Has been. In the example of FIG. 7, when the user touches an arbitrary area of the touch panel and inputs “1” and “2” by voice while the destination selection screen is displayed, the first and second destinations respectively. Is selected. In addition, when the user touches an arbitrary area of the touch panel and inputs “1” and “2” by voice while the thumbnail display screen is displayed, the first page and the second page are displayed, respectively. .

  FIG. 8 is a table showing operations set in advance in association with touch operation positions and keywords on the touch panel.

  In the example illustrated in FIG. 8, a case where the touch operation position (detection position) is a display portion of a key and a thumbnail, and the voice input keywords are “1” and “2” is illustrated.

  For example, when the user touches the display position of the thumbnail on the touch panel and inputs the sound “1” or “2”, the CPU 11 displays the first page or the second page based on the table of FIG. . Also in this case, no association is made according to the type of touch operation.

  Depending on the combination of the touched position and the keyword, there are keywords that are not accepted. For example, even if the user touches the display position of the key on the touch panel and inputs a voice of “1” or “2”, it is not accepted, and the screen of the display unit 171 is maintained as it is.

  In the case of FIG. 8 as well, the keywords are collected for each touch position and stored in the second dictionary. For example, a keyword related to the key position is the pattern a of the second dictionary, a keyword related to the thumbnail position is the pattern b of the second dictionary, and a keyword related to the page position that can be paged is the second dictionary. Keywords related to positions other than the dictionary pattern c are stored in the pattern d. When the CPU 11 determines the touched position on the display screen, the CPU 11 searches the second dictionary for a keyword input by voice by narrowing down the pattern related to the touch position, and performs the operation associated with the touched position and the keyword. It is supposed to be identified and executed.

  FIG. 9 is a table showing operations set in advance in association with types of touch operations on the touch panel and keywords input by voice.

  In the example illustrated in FIG. 9, a case where the touch operation type is flick, pinch-in, and rotation and the keyword input by voice is “1”, “2”, “50%”, and “90 degrees” is illustrated.

  For example, when the user inputs a voice “1” or “2” by flicking an arbitrary area of the touch panel while the destination selection screen which is a lower layer screen of the scan screen is displayed, the CPU 11 Based on the table of 9, the first or second destination is selected. When the user performs a touch operation by rotating the thumbnail display area of the touch panel while the thumbnail screen is displayed on the display unit 171, and inputs a voice “1”, “2”, or “90 degrees”, The thumbnail screen of the display unit 171 is rotated in the rotation direction by 1 degree, 2 degrees, or 90 degrees, respectively.

  Furthermore, if a pinch-out operation is performed and “2 ×” or “200%” is input as a voice, it is displayed in double size, and a pinch-in operation is performed and “1/2” “50%” is displayed as a voice. When “” is input, it is displayed in a half size, and when “screen width” and voice are input, it is possible to specify an arbitrary size such as displaying according to the horizontal width of the screen. In addition, by performing a pinch-in or pinch-out operation and inputting “favorite” as a voice, the display size can be changed to the set favorite size.

  In addition, there are keywords that are not accepted depending on the combination of the type of touch operation and the keyword. For example, when a destination selection screen is displayed, flicking an arbitrary area of the touch panel and inputting a voice of “50%” or “90 degrees” is not accepted, and the screen of the display unit 171 is not displayed. Keep it as it is.

  Also, depending on the type of touch operation and the displayed screen, even if the content of the voice input is the same, the operation to be executed is different. For example, if a voice is input as “2” by performing a flick operation, the second page is displayed. If a voice is input as “2” by a pinch operation, the second page is displayed. Also, when only one page is displayed on the page edit mode screen, if a flick operation is performed and “2” is input by voice, the page being edited is inserted into the second page or a pinch operation is performed. By inputting “2” as a voice, the page being edited may be divided into two copies, or the two pages may be combined into two in one. In addition, it is possible to discard to the trash box by performing a drag operation and inputting “discard” as a voice.

  Even with the same touch operation type, different operations may be performed according to the position on the display screen where the touch operation is performed. For example, as shown in FIG. 10A, when a rotation operation is performed in the thumbnail preview image display area and “90 degrees” is input as a voice, the thumbnail preview image is rotated by 90 degrees. As shown in FIG. 5, when a rotation operation is performed in the key area in the screen on which the thumbnail preview image is displayed and “90 degrees” is input as a voice, the screen is maintained without being accepted.

1 Image forming apparatus 11 CPU
12 ROM
14 Scanner unit 15 Storage unit 17 Operation panel 171 Display unit 19 Voice input unit

Claims (7)

Display means having a touch panel;
Determining means for determining whether or not a touch operation by a user has been performed on the touch panel of the display means;
Voice input means;
A voice input receiving means for receiving voice input by the voice input means when the judgment means determines that the touch operation has been performed;
At least one of the type of screen displayed on the display unit when the touch operation is performed, the position of the touch operation on the touch panel, the type of touch operation, and the voice received by the voice input receiving unit Control means for executing a preset operation in association with the content of the input;
An input operation device comprising: A storage unit that stores at least one of the type of the screen, the position of the touch operation, the type of the touch operation, the keyword, and the operation content in association with each other;
The control means determines whether or not the content of the voice input by the voice input means matches the keyword stored in the storage means, and if it matches, the keyword and the type of the screen, the position of the touch operation The input operation device according to claim 1, wherein an operation associated with at least one of the types of touch operations is executed.   The type of touch operation is a flick that is an action of flipping a finger while lightly touching the touch panel, a drag that is an action of sliding a finger while lightly touching the touch panel, or a tap that is an action of hitting the touch panel with a finger. Double-tap, which is the action of tapping with a finger twice, multi-touch on the touch panel, pinch-in / pinch-out which is the action of sliding the finger toward the center or sliding in the direction to expand, or multi-touch on the touch panel, At least one of rotation that rotates around the center position of the finger or both fingers or drag and drop that slides the finger while touching the touch panel lightly and releases the finger at the target location The input operation device according to claim 1 or 2.   The input operation device according to claim 1, wherein the voice input receiving unit receives a voice input until a preset time after the touch operation is finished.   An image forming apparatus comprising the input operation device according to claim 1. An input operation method for an image forming apparatus including a display unit having a touch panel and a voice input unit,
A determination step of determining whether a touch operation by the user has been performed on the touch panel;
In the determination step, when it is determined that the touch operation has been performed, a voice input reception step of receiving a voice input by the voice input unit;
At least one of the type of screen displayed on the display means when the touch operation is performed, the position of the touch operation on the touch panel, the type of touch operation, and the voice received in the voice input receiving step A control step for executing a preset operation in association with the content of the input;
An input operation method characterized by comprising: An input operation program for causing a computer of an image forming apparatus including a display unit having a touch panel and a voice input unit to execute the program,
A determination step of determining whether a touch operation by the user has been performed on the touch panel;
In the determination step, when it is determined that the touch operation has been performed, a voice input reception step of receiving a voice input by the voice input unit;
At least one of the type of screen displayed on the display means when the touch operation is performed, the position of the touch operation on the touch panel, the type of touch operation, and the voice received in the voice input receiving step A control step for executing a preset operation in association with the content of the input;
An input operation program for causing the computer to execute.

Images