TWM664095U - A system for music teaching - Google Patents

Abstract

This invention discloses a music teaching system, including an image module, a musical instrument recognition module, and a human posture estimation module; the image module is used to obtain the practice image data of the practitioner; the musical instrument recognition module is used to obtain musical instrument data information and human body data information related to the practitioner from the practice image data; the human body posture estimation module is used to determine the practice type of the practitioner based on the human body data information and the musical instrument data information; the human body data information is marked according to the practice type to obtain the practice posture of the practitioner. This creation adopts a method of combining human body posture information and musical instrument information to solve the problem that the existing technology cannot present the practitioner's human body posture during the practice process, and thus realizes the correction of human body posture during the practice process.

Description

A system for music teaching

This work belongs to the field of music teaching, specifically involving a system used for music teaching.

Current music teaching equipment, such as keyboard instruments, orchestral instruments, percussion instruments, and plucked instruments, uses sensors to detect the position and force of the strings and keys plucked, the drums pressed or struck, and the like, and compares the student's performance with the music score to determine whether the student is playing well or not. This method is actually based on the fact that the practitioner uses the body (such as fingers) or other instruments to contact the instrument, and then applies force to the instrument, causing the instrument to make a sound. It is to compare the timbre, scale and notes on the score. This comparison method cannot fully reflect whether the position of the practitioner's body posture relative to the instrument during the practice process is correct, so it is impossible to find the fundamental reasons why the practitioner makes playing errors or inadequate movements. The playing posture of students is often ignored when playing, and the wrong playing posture will cause great damage to the student's body. For the problems that the practitioner encounters during the playing process, the teacher or equipment needs to give effective correction instructions to improve the performance.

In order to solve the above problems, it is urgent to provide a system for music teaching so as to realize the combination of musical instrument types and present the body posture of the practitioner corresponding to the musical instrument type during the practice process.

In view of this, the author of this invention has specially researched and improved the music teaching system that can adjust the practitioner's practice posture, hoping to improve the above problems with a better design. After long-term research and development and continuous testing, this creation was finally released.

In order to at least solve the above problems, this invention proposes a music teaching system, including: an image module, a musical instrument recognition module, and a human posture estimation module; the image module is used to obtain the practice image data of the practitioner; the musical instrument recognition module is used to obtain one or more musical instrument data information in the practice image data; the human posture estimation module is used to obtain human data information related to the practitioner from the practice image data; the practice type of the practitioner is determined according to the human data information and the musical instrument data information; the human data information is marked according to the practice type to obtain the practice posture of the practitioner.

In the first aspect, the invention provides a music teaching system, including an image module, a musical instrument identification module, and a human posture estimation module; the image module is used to obtain the practice image data of the practitioner; the musical instrument identification module is used to obtain one or more musical instrument data information in the practice image data; the human posture estimation module is used to obtain human data information related to the practitioner from the practice image data; the practice type of the practitioner is determined according to the human data information and the musical instrument data information; the human data information is marked according to the practice type to obtain the practice posture of the practitioner.

Optionally, the exercise posture of the practitioner is compared with the image reference data or the exercise template image data to determine whether the exercise posture of the practitioner is correct; and in response to determining that the exercise posture of the practitioner is incorrect, a correction instruction for correcting the exercise posture of the practitioner is output.

Optionally, it further includes: an image reference module and a judgment module; the image reference module is used to obtain image reference data or training template image data of the trainee; the judgment module is used to compare the training posture of the trainee with the image reference data or the training template image data to judge whether the training posture of the trainee is correct; and in response to judging that the training posture of the trainee is incorrect, output a correction instruction for correcting the training posture of the trainee.

Optionally, the training image data includes at least one or more of visible light image data, invisible light image data or sonar image data.

Optionally, the types of musical instruments are classified according to the musical instrument data information to form a musical instrument classification library; and the musical instrument data information is stored in the corresponding musical instrument classification library according to the musical instrument classification information.

Optionally, judging the practice type of the practitioner according to the human body data information and the musical instrument data information further includes: determining according to the musical instrument data information and the human body data information, locating the joint positions of the practitioner related to the practice posture; and judging the type of the practitioner based on the joint position.

Optionally, the training image data also includes phased training template image data, wherein the phases are related to time and/or training difficulty.

Optionally, the comparison result of the training image data and the image reference data is obtained by executing artificial intelligence continuous image comparison; the training quality of the trainee is determined based on the comparison result; and posture correction suggestions for the trainee are output based on the training quality.

Optionally, it also includes outputting the correction instruction in a visual and/or auditory manner.

In the second aspect, the invention provides a method for music teaching, including obtaining a practitioner's practice video data; obtaining musical instrument data information and human body data information related to the practitioner from the practice video data; judging the practice type of the practitioner based on the human body data information and the musical instrument data information; marking the human body data information according to the practice type to obtain the practice posture of the practitioner.

Optionally, the exercise posture of the practitioner is compared with the image reference data or the exercise template image data to determine whether the exercise posture of the practitioner is correct; and in response to determining that the exercise posture of the practitioner is incorrect, a correction instruction for correcting the exercise posture of the practitioner is output.

Optionally, the training image data includes at least one or more of visible light image data, invisible light image data or sonar image data.

Optionally, the types of musical instruments are classified according to the musical instrument data information to form a musical instrument classification library; and the musical instrument data information is stored in the corresponding musical instrument classification library according to the musical instrument classification information.

Optionally, judging the practice type of the practitioner according to the human body data information and the musical instrument data information further includes: determining according to the musical instrument data information and the human body data information, locating the joint positions of the practitioner related to the practice posture; and judging the type of the practitioner based on the joint position.

Optionally, the training image data also includes phased training template image data, wherein the phases are related to time and/or training difficulty.

Optionally, the comparison result of the training image data and the image reference data is obtained by executing artificial intelligence continuous image comparison; the training quality of the trainee is determined based on the comparison result; and posture correction suggestions for the trainee are output based on the training quality.

Optionally, it also includes outputting the correction instruction in a visual and/or auditory manner.

Compared with the existing technology, this invention has the following advantages: This invention provides a system for music teaching. This invention obtains the practice image data of the practitioner during the practice process, extracts the instrument data information and the body data information of the practitioner from the practice image data; judges the practice type of the practitioner by combining the body data information and the instrument data information, and marks the body data information according to the practice type to obtain the practice posture of the practitioner corresponding to the practice instrument that the practitioner is using, so as to facilitate the storage, comparison, or presentation of the image of the practice posture to a remote teacher through data transmission. Furthermore, by comparing the exercise posture of the practitioner with the image reference data or the exercise template image data, it is determined whether the exercise posture of the practitioner is correct; and in response to determining that the exercise posture of the practitioner is incorrect, a correction instruction for correcting the exercise posture of the practitioner is output, thereby realizing the intelligent and intuitive provision of the judgment and correction suggestions corresponding to the exercise posture of the equipment used by the practitioner.

[Original work]

100, 200: Method

300:System

301: Image module

302: Musical instrument identification module

303: Human body posture estimation module

304: Image reference module

305: Judgment module

500: Equipment composition

501:Host

502: First Camera

503: Second camera

504: piano surface

505:Fixed rod

S101, S102, S103, S104, S201, S202, S203, S204, S205: Steps

By reading the detailed description below with reference to the attached figures, the above and other purposes, features and advantages of the exemplary implementation of the present invention will become easy to understand. In the attached figures, several implementations of the present invention are shown in an exemplary and non-restrictive manner, and the same or corresponding numbers represent the same or corresponding parts, among which: [Figure 1] shows an exemplary flow chart of the method for music teaching of the first embodiment of the present invention.

[Figure 2] shows an exemplary flow chart of a method for music teaching in the second embodiment of the present invention.

[Figure 3] shows a block diagram of a system for music teaching in the third embodiment of the present invention.

[Figure 4] shows a demonstration of the image features of the joint position recognition of a music practitioner's human body posture in the present invention embodiment.

[Figure 4a] shows an exemplary example of a key node bitmap extracted according to Figure 4 of the present invention.

[Figure 5a] shows an exemplary example of the device composition of the imaging module of one embodiment of the present invention.

[Figure 5b] shows an exemplary example of the device composition of the imaging module of another embodiment of the present invention.

[Figure 5c] shows an exemplary example of an electronic device for music teaching in another embodiment of the present invention.

This creation is a system for music teaching. Its implementation methods, features and effects are described in detail below with several preferred feasible embodiments and diagrams, so that you can have a deeper understanding and agree with this creation.

The following will combine the attached figures in the embodiments of this invention to clearly and completely describe the technical solutions in the embodiments of this invention. Obviously, the described embodiments are part of the embodiments of this invention, not all of them. Based on the embodiments in this invention, all other embodiments obtained by a person with ordinary knowledge in the technical field to which this invention belongs without making progressive labor are within the scope of protection of this invention.

It should be understood that the terms "include" and "comprising" used in the description and patent application of this invention indicate the existence of the described features, wholes, steps, operations, elements and/or components, but do not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and/or their collections.

It should also be understood that the terms used in this invention specification are only for the purpose of describing specific embodiments and are not intended to limit the invention. As used in this invention specification and the scope of the patent application, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and/or" used in this invention specification and the scope of the patent application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

As used in this specification and claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]", depending on the context.

The following is a description of the preferred embodiments of this invention in conjunction with the attached figures. It should be understood that the preferred embodiments described here are only used to illustrate and explain this disclosure, and are not used to limit this disclosure.

In view of the problem that the existing technology cannot intuitively present the body posture of the practitioner, the first embodiment of this creation provides a method 100 for music teaching, which is described in detail below with reference to Figure 1.

As shown in FIG1 , a method 100 for music teaching includes: in step S101: obtaining the practice image data of the practitioner.

Preferably, the training image data includes at least one or more of visible light image data, invisible light image data or sonar image data. The visible light image may be an image taken by a photographic device such as an ordinary camera or video camera, and the invisible light image data refers to an image taken by an infrared camera, infrared video camera, etc.

In step S102: musical instrument data information and human body data information related to the practitioner are obtained from the practice image data.

Preferably, the types of musical instruments are classified according to the musical instrument data information to form a musical instrument classification library; and the musical instrument data information is stored in the corresponding musical instrument classification library according to the musical instrument classification information.

The acquired instrument data information may include one or more instruments. The instruments in the practice image data are classified according to the appearance or timbre information of the instruments to form an instrument classification library. For example, the instrument classification library may be defined as orchestral instruments, keyboard instruments, percussion instruments, etc.

Furthermore, it also includes adding a practitioner-defined instrument classification library according to the practitioner's needs. For example, the practitioner can improve the instrument according to the needs, form a new instrument type, and then form a new instrument classification library. The practitioner can also re-classify the existing instrument classification library according to the needs.

In step S103: judging the practitioner's practice type based on the human body data information and the musical instrument data information.

Preferably, the position of the joints of the practitioner related to the practice posture is located based on the musical instrument data information and the human body data information; and the type of the practitioner is determined based on the joint position.

Among them, after locating the joint positions, an identification image of the joint points is formed. Please refer to Figure 4 for the demonstration of the characteristics of the identification image of the joint points. The identification image of the joint points is obtained based on human posture estimation. Human posture estimation refers to locating key points of the human body (such as shoulders, elbows, wrists, hips, knees, ankles, fingers, etc.) in images or videos through computer algorithms.

In step S104: the human body data information is marked according to the exercise type to obtain the exercise posture of the practitioner; the specific implementation process of the above method 100 is illustrated by combining the image of the little girl practicing ukulele in FIG4 and the joint node identification of the little girl shown in FIG4a: according to step S101, the image or video of the ukulele practitioner or the training process is obtained through an imaging device. The image can be obtained by shooting with a camera or mobile phone connected to the network, and the shot image is uploaded to the cloud server through the network for subsequent data analysis, or the subsequent data analysis is performed on the local device, for example, through an APP on a mobile phone.

According to step S102, musical instrument data information and human body data information related to the practitioner are obtained from the practice image data. Specifically, in an actual scene such as FIG4 , it is first necessary to locate the human body in the image information (image or video frame) of the above-mentioned little girl practicing the ukulele. For example, in the process of the little girl playing the ukulele, by locating the positions of the key nodes of the little girl's body parts, that is, in the human body area, first identify the positions of the key nodes, such as the major nodes of the main body trunk, such as the head, neck, shoulders, elbows, wrists, hands, hips, knees and ankles, etc., and mark the positions of the key nodes of the body trunk posture that can clearly reflect the ukulele playing process, such as the hand nodes are identified in detail. Then, images of musical instruments are extracted from the video information of the practitioner's practice process, and the extracted images of musical instruments are compared with the existing musical instrument database to directly identify the instrument used by the practitioner, or to identify the instrument used by the practitioner based on the aforementioned information of the key nodes of the body, and then obtain relevant musical instrument data information, such as roughly distinguishing whether it is a keyboard instrument or a string instrument through the hand posture, for example, there is a relatively obvious difference between the hand postures of playing the piano and playing the ukulele. Furthermore, the performance scene of the practitioner can also be identified based on the musical instrument data information and the aforementioned human body posture of the practitioner. The performance scene includes the data information of the musical instrument (specifically, whether it is a ukulele or a guitar, or a piano or an electronic keyboard, etc.), multiple shooting angles, etc. The musical instrument data information can also be identified by sound recognition.

According to step S103, the practice type of the practitioner is determined according to the human body data information and the musical instrument data information; according to the identification information of the body trunk posture and hand joints and the image of the musical instrument in FIG4a, it can be determined that the musical instrument played by the little girl is ukulele, and further according to the hand posture, the practice type of the little girl can be obtained, that is, the little girl is a beginner or a professional performer, etc., and the position of each key node of the hand has the greatest impact on the quality of the action during the playing process. Therefore, the identification and presentation of the key nodes of the hand can clearly present the problems existing in the playing action. Similarly, combined with actual situations, for practitioners playing other types of musical instruments, the above method can be used to derive the practitioner's practice type or practice stage. Among them, the specific recognition technology of human regional nodes can adopt computer vision and deep learning technology, such as convolutional neural network, etc.

To identify the location of key nodes, an automatic key point detection algorithm can be used, which usually requires the use of computer vision and deep learning technology to automatically identify the location of key nodes in images or video frames. The automatic detection algorithm requires a large amount of annotated data for training in order to achieve accurate key point recognition on new images. A large amount of annotated data is input into the existing pre-trained pose recognition model (such as OpenPose, PoseNet or AlphaPose and other basic models) for training, and the pre-trained model is used to extract features, thereby obtaining the identification and marking of the key joint locations. These pre-trained pose recognition models perform well in general human pose recognition and can handle a variety of scenes and poses. It is understood that the above description is merely exemplary and not restrictive, and a person with ordinary knowledge in the technical field to which this creation belongs can modify the above method according to the guidance of this creation without departing from the spirit and essence of this creation.

According to step S104, when the image or video of the music performance practitioner is collected, the image or video data captured by the camera or photographic equipment is analyzed, and the key point coordinates of the current practitioner are extracted according to the aforementioned practice type, and the key node coordinate positions are marked, where the coordinate positions can be relative values. Then, the marking of the human body data information is integrated to obtain the practitioner's practice posture. Furthermore, the result of the practice posture is used to analyze the practitioner's skills, evaluate the quality of the movement, and other tasks.

According to a preferred embodiment of the present invention, the practice image data in step S101 also includes staged practice template image data, wherein the stage is related to time and/or practice difficulty. The practice image data can be used to store the practice data of each practice stage in the practice process of the practitioner in the past, so that the practitioner himself or the teacher can compare the practice data of each stage in the practice process to determine whether the practicer's performance level has improved or has any defects.

In order to solve the problem that the existing technology cannot analyze the body posture of the practitioner in the form of images or audio and provide correction suggestions, the second embodiment of this invention provides a method for music teaching.

As shown in FIG2 , a method 200 for music teaching includes: in step S201: obtaining the practice image data of the practitioner.

In step S202: musical instrument data information and human body data information related to the practitioner are obtained from the practice image data.

In step S203: judging the practitioner's practice type based on the human body data information and the musical instrument data information.

In step S204: mark the human body data information according to the exercise type to obtain the exercise posture of the practitioner.

In step S205: the exercise posture of the practitioner is compared with the image reference data or the exercise template image data to determine whether the exercise posture of the practitioner is correct; and in response to determining that the exercise posture of the practitioner is incorrect, a correction instruction for correcting the exercise posture of the practitioner is output.

The image reference data is generated from the image of a famous musician or a music leader of a certain age group playing, or is a training template image data for a training stage of a trainee made by a teacher according to the trainee's progress. The training template image data can be a teacher's demonstration or a standard movement made by the trainee under the guidance of the teacher.

Furthermore, the comparison result of the training image data and the image reference data is obtained by executing artificial intelligence continuous image comparison; the training quality of the trainee is determined based on the comparison result; and posture correction suggestions for the trainee are output based on the training quality.

In actual application, the practitioner obtains an evaluation of his/her own practice process at a certain stage by using the continuous image comparison function of artificial intelligence. Based on the aforementioned use of computer vision and deep learning technology, the positions of key nodes in the continuous images or video frames of the practitioner at a certain stage are automatically identified and annotated, and the image annotation and comparison results of the practitioner at multiple practice stages are obtained. Through the comparison results, the evaluation of the practice process is obtained. For example, whether there is improvement or regression, and the more serious problems that have always existed in the practice process.

As shown in FIG3 , the third embodiment of the present invention also provides a system 300 for music teaching, including: an image module 301, a musical instrument recognition module 302, a human posture estimation module 303, an image reference module 304 and a judgment module 305.

The image module 301 is used to obtain and store the practice image data of the practitioner; the musical instrument identification module 302 is used to obtain one or more musical instrument data information in the practice image data; the human body posture estimation module 303 obtains human body data information related to the practitioner from the practice image data; determines the practice type of the practitioner according to the human body data information and the musical instrument data information; and marks the human body data information according to the practice type to obtain the practice posture of the practitioner.

Among them, human posture estimation includes judging body movements, including the positions of important joints and finger joints in different instrument practices, especially the movements of all finger joints (human posture estimation in other fields only has 17 or 33 points, and rarely uses all finger joints as parameters.)

For example, when practicing the piano, you need to raise your arms, swing your arms, stand your fingers, slide your fingers, and fix your wrists and elbows.

For example, for violin: holding the bow, holding the bow, stringing the bow, staccato bow, keeping the base of the thumb relaxed, etc.

For example, for percussion instruments, the palms are bent and raised to the chest.

In order to achieve human posture estimation, human posture recognition technology may be used. Human posture recognition technology may include the following steps: Human body detection: locate the human body in the image or video frame to provide a spatial range for subsequent posture recognition.

Key node detection: Identify the locations of key nodes in the human body, such as the head, neck, shoulders, elbows, wrists, hips, knees, and ankles. Human posture recognition algorithms include OpenPose or DeepLabCut. These algorithms can output the two-dimensional or three-dimensional coordinates of key points on a given demonstration image, which can be further used to analyze human posture.

Posture construction: Connect the detected key nodes to construct a human skeleton model to describe the human posture.

Analyze the images or video data captured by the camera or depth camera, extract the key coordinates of the practitioner, and mark the key nodes.

With the rapid development of deep learning and convolutional neural networks (CNN), human posture recognition technology has made significant progress, usually using deep learning methods. Some typical deep learning methods, such as HourglassNetwork, PoseNet and AlphaPose, have achieved high accuracy and real-time performance in human posture recognition tasks.

Through the above human posture recognition, we can get a description of the human posture, that is, realize human posture estimation.

Specifically, for example, in Figure 4, human posture recognition for piano players or keyboard users usually focuses on key areas such as hands, fingers, wrists, and upper limbs. The following is an exemplary method for human posture estimation for the above two types of players.

1) Clarify the key joint set: For the posture recognition task of the above-mentioned practitioners, clarify the key joint set. For piano practitioners, this usually includes the wrist, palm, and finger joints. Make sure that the key joints marked in the image can cover the main movements in the piano playing process.

2) Select appropriate images and videos: In order to obtain high-quality annotation data, clear, high-resolution images and videos should be selected to avoid motion blur and occlusion. In addition, in order to improve the accuracy of recognition, try to select data from multiple angles, multiple scenes, and multiple types of practitioners.

3) Use a specialized hand posture recognition model: You can use a pre-trained model specifically for hand posture recognition, such as Hand3D, DeepHPS, or OpenPose hand module models. These models perform well in hand posture recognition tasks and can be used as a basic model to learn from the images of the above practitioners.

4) Apply the pre-trained model to the pose recognition task of the above-mentioned practitioner. First, extract features using the pre-trained model, and then use the specially collected above-mentioned practitioner dataset for training and fine-tuning to better adapt to the above-mentioned practice scenarios.

5) Data augmentation: In order to improve the generalization ability of the model, data augmentation techniques can be used, such as image rotation, flipping, cropping, scaling, brightness and contrast adjustment, etc., to increase the diversity of training data. This helps the model to perform better in different scenes and perspectives.

6) Use the trained model to identify and annotate new images. Fuse the musical instrument features and human posture data to form a comprehensive dataset. You can also use timestamps to synchronize the music features and posture data so that their relationship can be considered in the analysis.

The image reference module 304 is used to obtain image reference data or the image data of the trainee's training template; the judgment module 305 is used to compare the trainee's training posture with the image reference data or the training template image data to judge whether the trainee's training posture is correct; and in response to judging that the trainee's training posture is incorrect, output a correction instruction for correcting the trainee's training posture.

Among them, all or part of the functions of each step in the above three embodiments can be completed by using the API of the local device to call the computing power of the cloud device or other distributed devices.

FIG4 shows an example of an image feature demonstration of a joint point position recognition method used in music teaching according to an embodiment of the present invention.

FIG5a shows a device composition 500 of an imaging module of an embodiment of the present invention. The imaging module of the present invention may include a sensor 501, a first camera 502, and a second camera 503, which are sequentially mounted on a fixed rod 505 from bottom to top. Furthermore, the sensor 501 may be a laser that generates infrared rays or a laser that generates other color lights (e.g., green light). Preferably, the laser may be, for example, an infrared single-beam laser. It should be noted that the imaging module of the present invention selects infrared rays as the light source for signal detection because infrared lasers have the advantages of low power consumption, good integration effect, and a single luminous frequency, and because the playing interface of a musical instrument (e.g., a keyboard) is usually a plane, a traditional single-beam laser cannot be used as a light source. Based on the above principle, this invention uses an infrared straight laser as the detection light source. In particular, the infrared straight laser can be configured to have an adjustable emission angle and direction. If the instrument being played is a piano, the infrared rays emitted can be parallel to the piano surface (504), and its installation position is close to the piano surface so that the laser plane it emits is close to the piano surface.

The first camera 502 and the second camera 503 may be wide-angle cameras, which are configured to collect the musical instrument data information and human body data information respectively. Specifically, when the performer is playing, his fingers or striking instruments will block and reflect the infrared rays at the position of the pressed or struck keys, and the pressed or struck keys will be triggered at this time, so the camera will take real-time images of the fingers or striking instruments blocking the infrared rays, and transmit the images to the human body posture estimation module and/or judgment module for further analysis and processing. In addition, the installation heights of the first camera 502 and the second camera 503 can be adjusted according to their shooting ranges, and their shooting ranges should be able to cover the overall situation of the musical instrument and the key joints of the human body respectively.

As shown in FIG. 5b, the device composition of the imaging module of another embodiment of the present invention, the imaging module of the present invention may include a first camera 502 and a second camera 503 sequentially mounted on a fixing rod 505 from bottom to top.

The first camera 502 and the second camera 503 may be wide-angle cameras, which are configured to collect the musical instrument data information and human body data information respectively. Specifically, when the performer is playing, the camera captures the image of the fingers or the striking instruments in real time, and transmits the image to the human body posture estimation module and/or judgment module for further analysis and processing. In addition, the installation height of the first camera 502 and the second camera 503 can be adjusted according to the range of their shooting, and the range of their shooting should be able to cover the overall situation of the musical instrument and the key joints of the human body respectively.

As shown in FIG. 5c, another embodiment of the present invention provides an electronic device for music teaching, including: a host 501, a first camera 502 and a second camera 503; the host 501 is used to receive images taken by the first camera 502 and the second camera 503; in this embodiment, the first camera 502 and the second camera 503 are only schematically shown to be installed on a fixed rod 505, and the installation method and position thereof are not limited; the host 501 also includes a processor; and a memory, which is used to store program instructions for music teaching, and when the processor runs the program instructions, the electronic device executes any method according to the first embodiment.

The first camera, the second camera, and the sensor described in the above two embodiments are in an "or" relationship and can be used together or independently, and can be set on different modules or devices (such as different computer hosts or mobile phones). In the specific implementation process, the functions of the first camera and the second camera are not limited.

The technical solution of this creation is particularly suitable for processing the image data of practitioners in AI chips.

In summary, this creation obtains image data during the practice process and performs process control on the practice process, stores and marks images of the practice posture, and directly presents them in the teaching equipment or presents them to remote teachers through data transmission. Furthermore, through the analysis and comparison of the image data of the practitioners, it can intelligently and intuitively give judgments and correction suggestions on the practice posture corresponding to the equipment used by the practitioners.

In summary, the technical means disclosed in this creation are not only unprecedented, but can also achieve the expected purpose and effect. Therefore, it is both novel and progressive, and is truly a new type as defined by the Patent Law. In terms of its overall structure, it has indeed met the statutory requirements of the Patent Law, and therefore a new type patent application is filed in accordance with the law.

However, the above is only the best implementation example of this creation, and it cannot be used to limit the scope of implementation of this creation. In other words, all equivalent changes and modifications made according to the scope of the patent application of this creation and the content of the specification should still fall within the scope of the patent of this creation.

300:System

301: Image module

302: Musical instrument identification module

303: Human body posture estimation module

304: Image reference module

305: Judgment module

Claims (9)

A system for music teaching, characterized in that it includes: an image module, a musical instrument recognition module, and a human posture estimation module; the image module is coupled to the musical instrument recognition module and the human posture estimation module, and the musical instrument recognition module is coupled to the human posture estimation module; the image module is used to obtain the practice image data of the practitioner; the musical instrument recognition module is used to obtain One or more musical instrument data information in the practice image data; the human body posture estimation module is used to obtain human body data information related to the practitioner from the practice image data; according to the human body data information and the musical instrument data information, the practice type of the practitioner is determined; and the human body data information is marked according to the practice type to obtain the practice posture of the practitioner. The system as described in claim 1, further comprising: an image reference module and a judgment module; the image reference module and the human posture estimation module are coupled to the judgment module; the image reference module is used to obtain image reference data or the training template image data of the practitioner; the judgment module is used to compare the training posture of the practitioner with the image reference data or the training template image data to determine whether the training posture of the practitioner is correct; and in response to determining that the training posture of the practitioner is incorrect, output a correction instruction for correcting the training posture of the practitioner. A system as described in claim 1, wherein the training image data includes at least one or more of visible light image data, invisible light image data, or sonar image data. A system as described in claim 1, wherein the training image data further includes phased training template image data, wherein the phases are related to time and/or training difficulty. The system as described in claim 2, further comprising: Classifying the types of musical instruments according to the musical instrument data information to form a musical instrument classification library; and storing the musical instrument data information in the corresponding musical instrument classification library according to the musical instrument classification information. The system as described in claim 5, further comprising adding a practitioner-defined instrument classification library according to the practitioner's needs. The system as described in claim 1, wherein determining the type of the practitioner's practice based on the human body data information and the musical instrument data information further includes: determining based on the musical instrument data information and the human body data information, locating the joint positions of the practitioner related to the practice posture; and determining the type of the practitioner based on the joint position. The system as described in claim 2, further comprising: obtaining the comparison result of the training image data and the image reference data by executing artificial intelligence continuous image comparison; determining the training proficiency of the trainee based on the comparison result; and outputting posture correction suggestions for the trainee based on the training proficiency. A system as described in any one of claim 2 to claim 8, wherein the system further includes outputting the correction instruction in a visual and/or auditory manner.