TWI745995B - Teaching Assistant System - Google Patents

Abstract

A teaching aid system. A smart glasses continuously shoot an image, generate a face image data according to the image, and continuously capture a current voice segment of a teacher, and then the face image data and the current voice The fragment is sent to the cloud server via a smart device. The cloud server stores the current voice segment, obtains a target identification code according to the face image data and a face recognition model, and generates and sends a data request to the teaching database server. The teaching database server sends a target basic data to the cloud server according to the data request, and the cloud server sends the target basic data and the target learning status data corresponding to the target ID to the smart device to display it The basic information of the target and the learning status information of the target.

Description

Teaching Assistant System

The present invention relates to an auxiliary system, in particular to a teaching auxiliary system that can improve teaching quality.

Teachers usually teach multiple classes at the same time, and when they come into contact with many students, they often call the students' names when they are called or interacted with each other in teaching.

However, the teacher cannot remember the names of the students one by one. Therefore, when the call is made, they only use the roll call method according to the name of the class, which is very time-consuming, and sometimes students will be called on behalf of the students; and When teaching, the teacher can only call out the names of a few impressed students, so that they can only interact with specific students, although the Republic of China Publication No. 201312478 discloses a portable face recognition device for users to see For someone, the display unit can display the name information at the same time, but only the name information can make it easier for the teacher to identify the students in class, but the teacher still has difficulty grasping the class status of each student.

Therefore, the purpose of the present invention is to provide a teaching assistance system that allows the teacher to grasp the class status of each student.

Therefore, the teaching assistance system of the present invention is suitable for assisting a teaching person, and includes a smart glasses, a smart device, a cloud server, and a teaching database server.

The smart glasses include an image capturing unit, a voice capturing unit, a glasses communication unit, and a glasses processing unit.

The image shooting unit is used for continuously shooting an image.

The voice capturing unit is used for continuously capturing a current voice segment of the teacher.

The glasses communication unit is connected to a first communication network.

The glasses processing unit is electrically connected to the image capturing unit, the voice capturing unit, and the glasses communication unit for determining whether the image captured by the image capturing unit has a human face, and when it is determined that the image has a human face, Generate a face image data based on the image.

The smart device includes a display unit, a device communication unit, and a device processing unit.

The device communication unit is connected to the first communication network and a second communication network, and is connected to the glasses communication unit via the first communication network.

The device processing unit is electrically connected to the display unit and the device communication unit.

The cloud server is connected to the second communication network, connected to the device communication unit via the second communication network, and stores a face recognition model related to a plurality of students and a plurality of face recognition models corresponding to the identification codes of the students, respectively , And multiple pieces of learning status data corresponding to the identification codes.

The teaching database server is connected to the cloud server via the second communication network, and stores a plurality of basic data corresponding to the identification codes.

Wherein, the glasses processing unit transmits the face image data and the current voice segment to the device communication unit via the glasses communication unit, and the device processing unit transmits the face image data and the current voice segment via the device communication unit To the cloud server, the cloud server stores the current voice segment, and obtains a target identification code corresponding to the face image data according to the face image data and the face recognition model. The cloud server generates and passes The second communication network transmits a data request including the target identification code to the teaching database server, and the teaching database server transmits the target identification code corresponding to the data request via the second communication network according to the data request The target basic data of to the cloud server, the cloud server transmits the target basic data and the target learning status data corresponding to the target identification code to the device communication unit via the second communication network, and the device processing unit controls the display The unit displays the basic information of the target and the learning status data of the target.

The effect of the present invention is that the smart device can instantly display the basic data and learning status data of the students photographed by the smart glasses, so that the teacher can grasp the class status of each student at any time.

Before the present invention is described in detail, it should be noted that in the following description, similar elements are denoted by the same numbers.

Referring to FIG. 1, a first embodiment of the teaching assistance system of the present invention includes a smart glasses 1, a smart device 2, a cloud server 3, and a teaching database server 4.

The smart glasses 1 includes an image capturing unit 11, a voice capturing unit 12, a glasses communication unit 13, and a glasses processing unit 14.

The image capturing unit 11 is used for continuously capturing an image.

The voice capturing unit 12 is used to continuously capture a current voice segment of one of the teachers. In this embodiment, the voice capturing unit 12 is, for example, a microphone, but it is not limited to this.

The glasses communication unit 13 is connected to a first communication network 91. In this embodiment, the first communication network 91 is a short-range wireless communication network, such as bluetooth or wifi, but it is not limited thereto.

The glasses processing unit 14 is electrically connected to the image capturing unit 11, the voice capturing unit 12, and the glasses communication unit 13, for determining whether the image captured by the image capturing unit 11 has a human face, and when it is determined that the image When there is a human face, a human face image data is generated according to the image. It is worth noting that, in this embodiment, when it is determined that the image has a face, the glasses processing unit 14 performs preprocessing on the image to generate the face image data, but it is not limited to this. .

The smart device 2 includes an input unit 21, a display unit 22, a device communication unit 23, and a device processing unit 24.

In this embodiment, the input unit 21 and the display unit 22 constitute a touch screen, but it is not limited thereto.

The device communication unit 23 is connected to the first communication network 91 and a second communication network 92, and is connected to the glasses communication unit 13 via the first communication network 91. In this embodiment, the second communication network 92 is the Internet, but it is not limited to this.

The device processing unit 24 is electrically connected to the input unit 21, the display unit 22 and the device communication unit 23.

The cloud server 3 is connected to the second communication network 92, and connected to the device communication unit 23 via the second communication network 92, and a plurality of mobile devices 93 respectively related to a plurality of students, and stores a data related to the students The face and a plurality of face recognition models corresponding to the identification codes of the students, and a plurality of learning status data corresponding to the identification codes respectively. It should be noted that, in this embodiment, the identification codes are, for example, the student numbers of the students, and the face recognition model is based on multiple face training image data related to the faces of the students and the The identification code is established by learning using a machine learning algorithm, such as a Convolutional Neural Network (CNN), and each learning status data includes an attendance rate, an exam score, and an assignment A score, and an impression score. The mobile devices 93 are, for example, smart phones, but not limited to this.

The teaching database server 4 is connected to the cloud server 3 via the second communication network 92, and stores a plurality of basic data corresponding to the identification codes. It is worth noting that in this embodiment, each basic information includes a name, a grade, a department, and a student type. The student type includes general students or special students (such as autistic students, listening Students with disabilities, or students with learning disabilities), but not limited to this.

The operation flow of the first embodiment of the teaching aid system of the present invention will be described below.

The glasses processing unit 14 transmits the face image data and the current voice segment to the device communication unit 23 via the glasses communication unit 13.

The device processing unit 24 transmits the face image data and the current voice segment to the cloud server 3 via the device communication unit 23.

The cloud server 3 stores the current voice segment, and performs voice-to-text conversion on the current voice segment to obtain text data corresponding to the current voice segment, and then translates the text according to multiple translation requests from the mobile devices Data to generate a plurality of translated text data corresponding to the translation requests, wherein each translation request includes a desired translation language, and the translated text data are respectively transmitted to the mobile devices via the second communication network 92 93. It should be particularly noted that in other embodiments, the cloud server can directly translate the current speech segment according to the translation requests to generate a plurality of translated speech segments corresponding to the translation requests, and pass the translation request. The second communication network 92 transmits the translated speech fragments to the mobile devices 93, but it is not limited to this.

The cloud server 3 obtains a target identification code corresponding to the face image data according to the face image data and the face recognition model, and the cloud server 3 generates and transmits a message including the target through the second communication network 92 The information of the identification code is requested to the teaching database server 4. The teaching database server 4 transmits the target basic data corresponding to the target identification code of the data request to the cloud server 3 via the second communication network 92 according to the data request. The cloud server 3 transmits the target basic data and target learning status data corresponding to the target identification code to the device communication unit 23 via the second communication network 92, and the device processing unit 24 controls the display unit 22 to display the target basic data. Information and information on the learning status of the target.

The input unit 21 updates the impression score of the target learning status data according to an input operation of the teacher, and the device processing unit 24 sends the updated target learning status data to the cloud server 3 via the device communication unit 23, The cloud server 3 updates the stored target learning status data.

In this embodiment, the cloud server 3 stores the current voice segment to record the lesson process, and the cloud server 3 converts the current voice segment into the text data, and then based on multiple translations from the mobile devices Request to translate the text data to generate the translated text data, and then send the translated text data to the mobile devices 93, so that students can learn in their native language, so that the teaching content is not restricted by language , And the smart device 2 can instantly display the basic information and learning status data of the students photographed by the smart glasses 1, so that the teacher can keep track of the class status of each student and the student’s attendance status at any time, and The teacher can use the smart device 2 to add or subtract the student's impression score according to the student's class attitude (for example: sleeping, playing mobile phone, daze, chatting, listening to music, etc.), and mark the reason for the addition or subtraction.

Referring to FIG. 2, a second embodiment of the teaching aid system of the present invention is similar to the first embodiment, except that the second embodiment further includes a video playback device 5.

The video playback device 5 includes a video playback communication unit 51, a video playback display unit 52, and a video playback processing unit 53.

The video playback communication unit 51 is connected to the second communication network 92 and connected to the cloud server 3 via the second communication network 92.

The video playback processing unit 53 is electrically connected to the video playback communication unit 51 and the video playback display unit 52.

The cloud server 3 transmits the text data to the video playback communication unit 51 via the second communication network 92, and the video playback processing unit 53 controls the video playback display unit 52 to display the text data.

In this embodiment, the video playback device 5 can display the text data in real time to assist students in learning.

It should be noted that in other embodiments, the cloud server 3 may also first translate the text data into a preset language, and then send the translated text data to the image via the second communication network 92 The playback communication unit 51, the video playback processing unit 53 controls the video playback display unit 52 to display the translated text data, but not limited to this.

Referring to FIG. 2, a third embodiment of the teaching aid system of the present invention is similar to the first embodiment, except that the third embodiment further includes a voice playback device 6.

The voice playback device 6 includes a voice playback communication unit 61, a speaker 62, and a voice playback processing unit 63.

The voice broadcast communication unit 61 is connected to the first communication network 91 and is connected to the glasses communication unit 13 via the first communication network 91.

The voice playback processing unit 63 is electrically connected to the voice playback communication unit 61 and the speaker 62.

The glasses processing unit 14 transmits the current voice segment to the voice playback communication unit 61 via the glasses communication unit 13, and the voice playback processing unit 63 controls the speaker 62 to play the current voice segment.

In this embodiment, the voice playback device 6 can instantly play the current voice segment to assist students in learning.

To sum up, in the teaching aid system of the present invention, the cloud server 3 stores the current voice segment to record the lesson process, and the cloud server 3 converts the current voice segment into the text data, and then according to the translation requests, Translate the text data to generate the translated text data, and then send the translated text data to the mobile devices 93, so that students can learn in their native language, so that the teaching content is not restricted by language, and The smart device 2 can instantly display the basic data and learning status data of the students photographed by the smart glasses 1, so that the teacher can grasp the class status of each student at any time, and grasp the student’s attendance status, and the teaching The user can use the smart device 2 to add or subtract the student’s impression score according to the student’s class attitude (such as sleeping, playing mobile phone, listening to music, etc.). In addition, the video playback device 5 can display the text data in real time, and the voice playback device 6 The current voice segment can be played in real time to assist students in learning, so it can indeed achieve the purpose of the invention.

However, the above are only examples of the present invention. When the scope of implementation of the present invention cannot be limited by this, all simple equivalent changes and modifications made in accordance with the scope of the patent application of the present invention and the content of the patent specification still belong to Within the scope covered by the patent of the present invention.

1: Smart glasses 11: Image capture unit 12: Voice capture unit 13: Glasses communication unit 14: Glasses processing unit 2: Smart device 21: Input unit 22: display unit 23: Device communication unit 24: Device processing unit 3: Cloud server 4: Teaching database server 5: Video playback device 51: Video playback communication unit 52: Video playback display unit 53: video playback processing unit 6: Voice playback device 61: Voice playback communication unit 62: Horn 63: Voice playback processing unit 91: The first communication network 92: The second communication network 93: mobile device

Other features and effects of the present invention will be clearly presented in the embodiments with reference to the drawings, in which: Figure 1 is a block diagram illustrating a first embodiment of the teaching aid system of the present invention; Figure 2 is a block diagram illustrating a second embodiment of the teaching aid system of the present invention; and Fig. 3 is a block diagram illustrating a third embodiment of the teaching aid system of the present invention.

1: Smart glasses

11: Image capture unit

12: Voice capture unit

13: Glasses communication unit

14: Glasses processing unit

2: Smart device

21: Input unit

22: display unit

23: Device communication unit

24: Device processing unit

3: Cloud server

4: Teaching database server

91: The first communication network

92: The second communication network

Claims (10)

A teaching assistance system, suitable for assisting a teacher, includes: a smart glasses, including an image shooting unit, which continuously shoots an image, and a voice capture unit, which continuously captures a current voice segment of one of the teachers, A glasses communication unit, connected to a first communication network, and a glasses processing unit, electrically connected to the image capturing unit, the voice capturing unit, and the glasses communication unit, and determining whether the image captured by the image capturing unit has a human Face, when it is determined that the image has a human face, a face image data is generated according to the image; a smart device, including a display unit, a device communication unit, connected to the first communication network and a second communication network, and through The first communication network is connected to the glasses communication unit, and a device processing unit is electrically connected to the display unit and the device communication unit; a cloud server is connected to the second communication network, and the device communicates via the second communication network Unit, and store a plurality of student faces and a plurality of face recognition models corresponding to the identification codes of the students, and a plurality of learning status data corresponding to the identification codes; and a teaching database server The device is connected to the cloud server via the second communication network, and stores multiple pieces of basic data corresponding to the identification codes; Wherein, the glasses processing unit transmits the face image data and the current voice segment to the device communication unit via the glasses communication unit, and the device processing unit transmits the face image data and the current voice segment via the device communication unit To the cloud server, the cloud server stores the current voice segment, and obtains a target identification code corresponding to the face image data according to the face image data and the face recognition model. The cloud server generates and passes The second communication network transmits a data request including the target identification code to the teaching database server, and the teaching database server transmits the target identification code corresponding to the data request via the second communication network according to the data request The target basic data of to the cloud server, the cloud server transmits the target basic data and the target learning status data corresponding to the target identification code to the device communication unit via the second communication network, and the device processing unit controls the display The unit displays the basic information of the target and the learning status data of the target. The teaching assistance system according to claim 1, wherein the cloud server is connected to a plurality of mobile devices respectively related to the students via the second communication network, and the cloud server also performs voice-to-text conversion on the current voice segment To obtain text data corresponding to the current voice segment, and then translate the text data according to multiple translation requests from the mobile devices to generate multiple translated text data corresponding to the translation requests, each translation request It includes a desired translation language, and transmits the translated text data to the mobile devices via the second communication network. The teaching assistance system according to claim 2, further comprising a video playback device, and the video playback device includes A video playback communication unit connected to the second communication network and connected to the cloud server via the second communication network; a video playback display unit; and a video playback processing unit electrically connected to the video playback communication unit and the image Play and display unit; wherein, the cloud server also transmits the text data to the video playback communication unit via the second communication network, and the video playback processing unit controls the video playback and display unit to display the text data. The teaching assistance system according to claim 2, further comprising a video playback device including a video playback communication unit, connected to the second communication network, and connected to the cloud server via the second communication network; A video playback and display unit; and a video playback processing unit, electrically connected to the video playback communication unit and the video playback and display unit; wherein the cloud server translates the text data and translates it via the second communication network The text data of is sent to the video playback communication unit, and the video playback processing unit controls the video playback display unit to display the translated text data. The teaching assistance system according to claim 1, wherein the smart device further includes an input unit electrically connected to the device processing unit, the input unit updates the target learning condition data according to an input operation of the teacher, and the device processes The unit sends the updated target learning status data to the cloud server via the device communication unit, and the cloud server updates the stored The target learning status data. The teaching assistance system according to claim 5, wherein each learning status data includes an attendance rate, an examination score, an assignment score, and an impression score, and the input unit updates the target learning status data according to the input operation Impression score. The teaching assistance system according to claim 1, wherein when it is determined that the image has a human face, the glasses processing unit performs pre-processing on the image to generate the human face image data. The teaching assistance system according to claim 1, wherein the face recognition model stored in the cloud server is based on a plurality of face training image data related to the faces of the students and the identification codes. Established by a machine learning algorithm. The teaching assistance system according to claim 1, further comprising a voice playback device including a voice playback communication unit, connected to the first communication network, and connected to the glasses communication unit via the first communication network; A speaker; and a voice playback processing unit electrically connected to the voice playback communication unit and the speaker; wherein, the glasses processing unit transmits the current voice segment to the voice playback communication unit via the glasses communication unit, and the voice playback processing unit Control the speaker to play the current voice segment. The teaching assistance system according to claim 1, wherein the cloud server is connected to a plurality of rows respectively related to the students via the second communication network Mobile device, the cloud server translates the current voice segment according to multiple translation requests from the mobile devices to generate multiple translated voice segments corresponding to the translation requests, and pass the second communication network The translated speech fragments are respectively transmitted to the mobile devices.

Images