JP2019113591A - Point rating support device, and control method and control program therefor - Google Patents

Abstract

[Problem] To provide a marking support device that can reduce the workload when marking descriptive or essay style questions and perform marking more efficiently, and a control method and control program for the same. [Solution] The marking support device 100 classifies answer data including answers to descriptive or essay style questions to be marked into groups of similar answer contents for each question, and sets marking criteria for each group of similar answer contents, which makes it easy to clarify the marking criteria and reduces the workload when marking. [Selected Figure] Figure 1

Description

  The present invention relates to a scoring support device that supports scoring of answers to descriptive questions and essay questions, and a control method and control program therefor.

  Conventionally, in university entrance examinations and various qualification tests, mock tests for exam preparation, etc., it is roughly divided into alternatives, which use a mark sheet to select an answer from a plurality of options, and words, phrases, and simple sentences. Description methods, essay methods such as essays, etc. are adopted. As for the scoring in these tests, an automatic scoring method is adopted in the alternative method using a mark sheet. On the other hand, with regard to the description method and the argument method, since the scoring work by one or more graders is required, the task time at the time of scoring increases, the task load increases, and the evaluation for the answer varies. Had.

  Therefore, various methods for solving such a problem have been proposed. For example, in Japanese Patent Application Laid-Open Publication No. 2008-101501, model questions are presented by presenting question answer data of model answerers and answer data of answerers to the scorer terminal of the person in charge of scoring for each question in the description method and the discussion method. It is described that, while the division of the same answer sheet is divided by a plurality of graders at the same time, one specific grader is in charge of the scoring for each question.

2004-069746

  In the scoring method as described above, basically, one or more scorers manually perform scoring based on the model answer etc., so the workload at the time of scoring can not be sufficiently reduced. There was room for improvement.

  In view of the above, the present invention provides a scoring support device capable of marking more efficiently by reducing the workload at the time of scoring with regard to questions in the description method and the discussion method, and providing a control method and control program therefor. To aim.

The scoring support device according to the present invention is
Classify the answer data including the answer of the question to be scored into a group of similar answer contents for each question,
A scoring standard is set for each group of similar answer contents.

Moreover, the control method of the scoring assistance apparatus according to the present invention
Classify the answer data including the answer of the question to be scored into a group of similar answer contents for each question,
A scoring standard is set for each group of similar answer contents.

The present invention is a control program of a scoring support device, and
On the computer
The answer data including the answer of the question to be scored is classified into groups of similar answer contents for each question,
A scoring standard is set for each group of similar answer contents.

  According to the present invention, with regard to questions in the description method and the discussion method, it is possible to reduce the work load at the time of scoring and perform scoring more efficiently.

It is a block diagram showing one embodiment of a scoring assistance device concerning the present invention. It is a flowchart which shows an example of the control method of the scoring assistance apparatus which concerns on one Embodiment. It is a flowchart which shows the registration method of the answer data in an example of the control method of the scoring assistance apparatus which concerns on one Embodiment. It is a flowchart which shows the setting of the scoring standard in an example of the control method of the scoring assistance apparatus which concerns on one Embodiment. It is the schematic which shows the modification of the scoring assistance apparatus which concerns on one Embodiment.

Hereinafter, a scoring support device according to the present invention, a control method therefor and a control program therefor will be described in detail by showing embodiments.
FIG. 1 is a block diagram showing an embodiment of a scoring support device according to the present invention.

  For example, as shown in FIG. 1, the scoring support device 100 according to the present invention is generally an image reading unit 110, an image processing unit 120, a similarity calculating unit 130, a determination information updating unit 140, and an achievement determining unit 150, a correctness determination processing unit 160, a display unit 170, an input operation unit 180, and a storage unit 190. Here, the scoring support device 100 is realized by a system including an information processing terminal such as a personal computer. In addition, at least the image processing unit 120, the similarity calculation unit 130, the determination information update unit 140, the achievement degree determination unit 150, and the correctness determination processing unit 160 perform operations such as CPU (central processing unit) or MPU (microprocessor) It is realized by a processing means or a computer.

  The image reading unit 110 reads an answer sheet as image data, and stores the read result in, for example, a predetermined storage area of the storage unit 190 in association with an attribute (examination number or the like) of an answerer. Here, the image reading unit 110 may be provided integrally with another configuration and the scoring support device 100, or may have a separate configuration such as an external scanner. It may be

  The image processing unit 120 performs predetermined image processing on the image data of the answer sheet read by the image reading unit 110, and character-recognizes the answer content for each question and acquires it as answer data subjected to text processing. Specifically, the image processing unit 120 disassembles the image data of the answer sheet read by the image reading unit 110 for each question, and extracts the image data of the question of the description method or the argument method to be scored. Then, the image processing unit 120 executes character recognition processing on each of the extracted image data, and converts the answer contents into text. The answer data of the answer contents to be graded, which are converted into text data, are rearranged in the order of the questions having a large number of answerers and stored in a predetermined storage area of the storage unit 190.

  The similarity calculation unit 130 calculates the similarity to the correct answer (correct answer) of the question and the similarity to the incorrect answer. Here, the degree of similarity is, for example, 100% of the degree of similarity A, which is the degree of similarity to the correct answer, and the degree of similarity, which is the degree of similarity to the incorrect answer, When the degree B is 0%, and the important word and the sentence structure to be synonymous are not used at all, the similarity A is 0% and the degree of similarity B is 100%. The similarity to the correct solution and the similarity to the incorrect solution are calculated according to the frequency of use.

  Specifically, the similarity calculation unit 130 first morphologically analyzes the answer data for the extracted question and vectorizes it according to the importance of each topic (noun) of the answer data. Assuming that the importance of all topics (nouns) in the model solution is vector A and the importance of all topics (nouns) in the answer data is vector B, the product of the size of vector A and the size of vector B is the denominator. A value with the numerator being the sum of products of the importance degree vector of each topic of A and the topic vector of the corresponding answer data is calculated as the correct similarity of the answer data to the model answer. Similarly, assuming that the importance of an incorrect answer topic different from all the topics (nouns) of model solution is vector C, the product of the magnitude of vector C and the magnitude of vector B is a denominator, and the importance vector of each topic of vector C A value with the numerator being the sum of the products of the corresponding answer data and the topic vector is calculated as the incorrectness similarity of the answer data to the model answer. The arithmetic processing means stores the correct similarity and the incorrect solution similarity of the answer data to the model answer in the storage unit 190.

  The determination information update unit 140 updates the parameter serving as the scoring standard based on the determination result of the correct / incorrect answer to the question answer. Specifically, the determination information update unit 140 compares the answer content converted into text data by the image processing unit 120 with the model answer of the question, or the similarity calculated by the similarity calculation unit 130. Based on the judgment result of the "correct / incorrect answer" judged by the person in charge of marking, the correct answer / incorrect answer list that indicates the correct answer / incorrect answer, which is the standard when scoring for each question Update ". In addition, the determination information update unit 140 updates the correctness / incorrectness determination rate calculated based on the determination of “correct / incorrect” by the above-mentioned person in charge of scoring. The updated “correct list / incorrect list” and the correct / incorrect determination rate are stored in a predetermined storage area of the storage unit 190.

  The achievement level determination unit 150 determines the degree of achievement of the threshold value of the parameter updated by the determination information update unit 140. Specifically, the achievement level determination unit 150 determines that the “correct list / not above” is satisfied when the updated correctness / incorrectness judgment rate reaches the level of the preset threshold (judgment threshold Ath, Bth). Based on the “correct list”, it is determined that a sufficiently reliable determination of correct / incorrect answers can be realized. The determination result is stored in a predetermined storage area of the storage unit 190.

  The correctness / incorrectness judgment processing unit 160 judges the correctness / incorrectness with respect to the answer of the question based on the parameter judged by the achievement degree judgment unit 150 as having reached the predetermined achievement degree. Specifically, the correctness / incorrectness judgment processing unit 160 corrects the answer to the question based on the “correct answer list / incorrect answer list” in which the achievement judgment unit 150 determines that the correctness / incorrect judgment rate has reached the predetermined threshold. / Automatically execute the judgment of incorrect answer, and determine the score based on the judgment. The determined score is stored in a predetermined storage area of the storage unit 190.

The display unit 170 has a display and a display panel, and displays various data and the like handled in the above-described units. The input operation unit 180 has an input device such as a keyboard and a mouse, for example, selects data displayed on the display unit 170, a menu, and the like, and executes predetermined processing. The storage unit 190 stores various data handled or generated in the above-described units in a predetermined storage area. In addition, the storage unit 190 stores a control program for realizing the processes and functions in the above-described units. Here, part or all of the storage unit 190 may have a form as a removable storage medium, and may be configured to be removable from the scoring support device 100.
The storage unit 190 is a memory for storing the answer contents, the number of answers, the answer contents vector, the normalized scoring result, and the like.

  In addition, although illustration is abbreviate | omitted in this embodiment, the scoring assistance apparatus 100 may be provided with the interface part (I / F part) which transmits / receives various data between external apparatuses. Here, the I / F unit may be connected directly to an external device, or may be connected via a network. In this case, a wired connection method connected via a communication cable or a wireless connection method based on a communication standard such as Wi-Fi (registered trademark) can be applied.

(Control method of scoring support device)
Next, a control method and a control program of the scoring support device according to the present embodiment will be described with reference to the drawings.
FIG.2, FIG.3, FIG.4 is a flowchart which shows an example of the control method of the scoring assistance apparatus which concerns on this embodiment.

  In the control method of the scoring support apparatus according to the present embodiment, processing for generating a correct answer list / incorrect answer list for each question in a description method or an argument method to be evaluated, and the generated correct answer list / incorrect answer list Then, the judgment of the correct / incorrect answer to the answer of the question is automatically executed, and the actual scoring process of deciding the scoring based on the judgment is executed. These processings are applied to the scoring support device 100, operation processing means including the image processing unit 120, the similarity calculation unit 130, the determination information update unit 140, the achievement degree determination unit 150, and the correctness determination processing unit 160, or This is realized by executing a specific program on a computer.

  As shown in the flowcharts of FIG. 2, FIG. 3 and FIG. 4, the control method of the scoring support apparatus 100 according to the present embodiment first performs various operations such as scoring of handwritten answer sheets by writing instruments and In a company or an educational institution that comprehensively manages information, the answer sheets of all the solvers are collectively read (scanned) by the image reading unit 110 of the scoring support apparatus 100 after the test is completed (step S102). Here, the image reading unit 110 may read the entire answer sheet, or may specify and read a specific area.

  Next, in the image processing unit 120 of the scoring support device 100, the image data of the answer sheet read by the image reading unit 110 is disassembled for each question, and the image data of the question of the description method or argument method to be scored is extracted. (Step S104). The image processing unit 120 executes character recognition processing on each of the extracted image data, and performs text conversion for converting the contents of the answer into a text (step S106). Although the above answer is based on the answer sheet, the answer contents may be converted into text by converting the contents of the answer by voice recognition into text, or the answer of the analog data answered by other means may be converted into text. Here, data of answers converted into texts are collectively referred to as answer data including unanswered data. After step S106, the arithmetic processing means records the unregistered data in the storage unit 190 in association with the question.

  Next, as shown in FIGS. 2 and 3, the process proceeds to step S107. Step S107 executed by the arithmetic processing means includes steps S202 to S222. First, the arithmetic processing means specifies the total number of answerers corresponding to the answer data, and registers the total number of answerers in the storage unit 190 (step S202). Then, one question whose number of registered answer persons has not reached all the answer persons is extracted (step S204), and a plurality of answer data corresponding to the extracted one question is extracted from the storage unit 190 (step S206). When the total number of registered answer persons of the plurality of answer data corresponding to one extracted question does not reach the total number of answer persons (NO in step S208), among the plurality of answer data corresponding to the extracted one question One unregistered answer data is extracted (step S210).

  If the registered answer data determined to be substantially the same content as the extracted unregistered answer data is stored in storage unit 190 (step S 212 YES), the unregistered answer data has already been registered. It is stored in the storage unit 190 as data of a group of data, and the number of registered and answered persons of the corresponding question is counted up to +1 and stored in the storage unit 190 (step S214), and the process returns to step S208.

  On the other hand, when the answer data having substantially the same content as the extracted unregistered answer data is not stored in the storage unit 190 as registered (NO in step S212), the unregistered answer data is already registered. The data is stored in the storage unit 190 as data of a group of new registered answer data different from the existing answer data (step S216), and the number of registered answer persons of the corresponding question is set to +1 and stored in the storage unit 190 (step S218) , And return to step S208.

  In step S208, when the total number of registered answer number in step S214 and step S218 reaches all the answer number (YES in step S208), the answer number corresponding to all the questions does not reach all the answer number (NO in step S220). Returning to step S204, if reached (step S220 YES), the group of answer data of the answer contents to be graded that have been converted into digital data is rearranged in the order of the question having a large number of answers, and the predetermined storage of the storage unit 190 It stores in the area (step S222).

  Next, as shown in FIG. 2 and FIG. 4, the process shifts to setting of a scoring standard (step S108). Step S108 executed by the arithmetic processing means includes steps S302 to S332. If the scoring criteria for all the questions have not been completed (NO in step S302), the arithmetic processing means extracts the questions for which the scoring criteria have not been completed and reads one or more groups of corresponding answer data from the storage unit 190 ( Step S304) Among these, the group having the largest number of registered answer persons in the group of the answer data for which the normalized scoring result is not registered is extracted (step S306).

  When automatic scoring of the group of the extracted answer data is set to be effective (YES in step S308), the degree of correctness similarity and the degree of incorrectness of each answer data are calculated by the similarity calculation unit 130 (step S310).

  If the calculated correct similarity is equal to or higher than the set correct similarity threshold and the calculated incorrect similarity is equal to or higher than the set incorrect similarity threshold, the manual scoring by the grader ( Normalized scoring result determination is performed (step S312 YES), and if the result of the manual scoring is considered valid (step S316 YES), the process proceeds to step S318. If the result of the manual scoring is not regarded as valid (NO in step S316), the process proceeds to step S314, and the normalized scoring result of the answer data to the model solution is registered based on the degree of similarity by automatic scoring. In this case, the higher one of the correct similarity and the incorrect similarity is registered as a normalized scoring result. Here, the higher the degree of similarity with the model solution, the closer the correct degree of similarity to “1”, and the higher the degree of similarity with the model solution, the closer to “0”.

  On the other hand, if the calculated correct similarity is less than the set correct similarity threshold, manual scoring by the grader is not carried out (NO in step S312), and automatic scoring is performed, based on the similarity by automatic scoring. The normalized scoring result of the answer data to the model answer is registered with the incorrect answer similarity (for example, "0") (step S314). Also, if the calculated correct similarity is equal to or higher than the set correct similarity threshold and the calculated incorrect similarity is less than the set incorrect similarity threshold, manual scoring is performed. Instead (NO in step S312), automatic scoring is performed, and the normalized scoring result of the answer data to the model answer is registered as the correct similarity (for example, "1") based on the similarity by the automatic scoring (step S314).

  In step S308, when automatic scoring of the group of the extracted answer data is not set effective (NO in step S308), or when it is determined in step S316 that manual scoring by the grader is valid (step S308) The answer data group is presented to the grader (step S318).

  The grader appropriately marks the presented answer data according to the model answer and inputs the graded result (step S320), and the arithmetic processing means normalizes the graded result of the answer data to the model answer based on the similarity based on the input result. It registers (step S322).

  Next, among the groups of extracted answer data, the number of persons in the normalized scoring result (step S314) registered by giving priority to automatic scoring, and the normalized scoring result registered with priority to manual scoring (step S322) If the calculated value is equal to or higher than the automatic marking start threshold value (YES in step S326), the automatic marking is validated (step S328), and the calculated value is calculated. Is less than the automatic marking start threshold (NO in step S326), the process proceeds to step S330.

  In step 330, if the calculated value is equal to or more than the manual marking start threshold (YES in step S330), the manual marking is invalidated (step S332), and the process proceeds to step S302, and the calculated value is less than the manual marking start threshold If it is (step S326 NO), it will transfer to step S302.

  And if the scoring criteria of all the questions are completed (Step S302 YES), it will shift to adjustment of the scoring criteria of Step S110. In step S110, correct similarity and incorrect solution similarity between the answer vector of the model solution and the solution vector of the solution data group are calculated, and the normalized scoring result obtained in step S108 is correct similarity, incorrect solution similarity Check if they match the higher one, and if they do not match, notify the grader or the grade manager who manages the grade that they do not match.

  The grader or the grader compares the answer contents determined to have a high degree of correct similarity with the answer contents determined to have a high degree of incorrect similarity, and determines the necessity of changing the graded result. If the grader or the grader decides that the change is necessary, the grader adds a correction to the normalized graded result, repeats step S110 from the beginning, and if it is determined that the change is not necessary, shifts to the grade of step S112. In step S112, the scorer obtains points according to the score given by the normalized score result for each group of answer data, and markings of all the solvers by completing scoring of all groups of all questions. Ends.

  Step S107 can be omitted because the scorer manually marks all the answers without the operation processing means performing. In this case, all items may be registered and the process may proceed to step S108.

  In addition, step S107 can be omitted by causing the same grader to score similar answer content. In this case, it is sufficient to create a scored correct answer list / incorrect answer list of each grader and cause the grader to score an answer similar to the correct answer list / incorrect answer list.

(Modification)
Next, a modification of the scoring support device according to the present embodiment will be described.
FIG. 5 is a schematic view showing a modification of the scoring support device according to the present embodiment.

  The scoring support device 100 according to the above-described embodiment can be applied to a center device installed in an enterprise, an educational institution, or the like. This invention is not limited to such a form, For example, you may have a modification which performs a part of function of the scoring assistance apparatus 100 with the other apparatus and terminal outside.

  Specifically, for example, as shown in FIG. 5, the scoring support device according to the modification includes at least a center device 210 installed in a business entity or the like, a grader terminal 220 assigned to a plurality of graders, And a network 300 to which the center device 210 and the grader terminal 220 are connected. Here, the network 300 may be a network that can be used publicly, such as the Internet, or may be a network that can be limitedly used by a specific group such as a company or an educational institution.

  The center device 210 has, for example, a configuration equivalent to that of the scoring support device 100 described in the above-described embodiment, and the grader terminal 220 includes at least a display unit, an input operation unit, a storage unit, an I / F unit, and the like. And a CPU for controlling each of these units. Here, the grader terminal 220 receives the answer data and the correct answer data of the question to be graded provided from the center device 210 via the network 300 and displays it on the display unit, whereby the grader in charge of the answer is Determine the correct / incorrect answer. The determination result is transmitted to the center device 210 via the network 300.

  As a result, since it is possible to execute processing including steps requiring a judgment by the person in charge of grading among the control methods shown in the embodiment described above on the grader terminal 220, a plurality of operators in charge of each task can be performed. Can be performed simultaneously in parallel, reducing the burden due to the increase in working time at the time of scoring, and suppressing variation in evaluation for answers even when scoring by multiple graders, more efficiently Scoring can be done.

  In the configuration shown in FIG. 5, the grader terminal 220 receives (downloads) answer data and correct answer data of a question to be scored from the center device 210 via the network 300 (corrected) / incorrect answer It is not limited to the form which makes a judgment. For example, only when the grader terminal 220 is accessed as a cloud-type terminal and the center device 210 is accessed through the network 300, it is possible to use correct answer data and answer data of questions to be scored, so that scoring can be performed. The data may not be downloaded or stored in the user terminal 220. According to this aspect, it is possible to prevent the outflow of personal data and important data related to the scoring operation, including the scoring results and attributes of the answerer.

In the above-described embodiment, it is preferable to perform manual scoring with invalid automatic scoring at the initial stage, accumulate data, and then switch to automatic scoring in step S308.
In the above embodiment, the points of incorrect answers may be classified into zero points of complete wrong answers and partial points higher than 0 points, and may be set as points of incorrect answers with the highest degree of similarity. If there are a plurality of similarities exceeding the specified threshold, the average value of the distribution of the plurality of similarities exceeding the threshold may be used as the distribution.

Although some embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and the respective configurations of the above-described plurality of embodiments are combined within the feasible range. And the scope of the invention described in the claims and the equivalents thereof.
In the following, the invention described in the original claims of the present application is appended.

(Supplementary note)
[1]
Classify the answer data including the answer of the question to be scored into a group of similar answer contents for each question,
A scoring support device characterized by setting a scoring standard for each group of similar answer contents.

[2]
The setting of the scoring standard includes switching from automatic scoring to manual scoring by a grader based on correct similarity and incorrect similarity of each answer content to the model answer content [1]. Scoring support device.

[3]
It is characterized in that at least one of the automatic marking validation and the manual marking invalidation is set according to the marking ratio according to the normalization marking result by the automatic marking or the normalization marking result by the manual marking according to the marking criteria set. The scoring support device according to [1] or [2].

[4]
Classify the answer data including the answer of the question to be scored into a group of similar answer contents for each question,
The control method of the scoring support apparatus, wherein a scoring standard is set for each group of similar answer contents.

[5]
On the computer
The answer data including the answer of the question to be scored is classified into groups of similar answer contents for each question,
A control program for a scoring support apparatus, characterized in that a scoring standard is set for each group of similar answer contents.

100 Scoring Support Device 110 Image Reading Unit 120 Image Processing Unit 130 Similarity Calculation Unit 140 Judgment Information Update Unit 150 Achievement Degree Judgment Unit 160 Correctness Judgment Processing Unit 170 Display Unit 180 Input Operation Unit 190 Storage Unit

Claims (5)

Classify the answer data including the answer of the question to be scored into a group of similar answer contents for each question,
A scoring support device characterized by setting a scoring standard for each group of similar answer contents.   The setting of the scoring criteria includes switching from automatic scoring to manual scoring by a grader based on correct similarity and incorrect similarity of each answer content to model answer content. Scoring support device.   It is characterized in that at least one of the automatic marking validation and the manual marking invalidation is set according to the marking ratio according to the normalization marking result by the automatic marking or the normalization marking result by the manual marking according to the marking criteria set. The marking support device according to claim 1 or 2. Classify the answer data including the answer of the question to be scored into a group of similar answer contents for each question,
The control method of the scoring support apparatus, wherein a scoring standard is set for each group of similar answer contents. On the computer
The answer data including the answer of the question to be scored is classified into groups of similar answer contents for each question,
A control program for a scoring support apparatus, characterized in that a scoring standard is set for each group of similar answer contents.

Images