[go: up one dir, main page]

CN106340286B - Universal real-time musical instrument playing evaluation system - Google Patents

Universal real-time musical instrument playing evaluation system Download PDF

Info

Publication number
CN106340286B
CN106340286B CN201610859490.XA CN201610859490A CN106340286B CN 106340286 B CN106340286 B CN 106340286B CN 201610859490 A CN201610859490 A CN 201610859490A CN 106340286 B CN106340286 B CN 106340286B
Authority
CN
China
Prior art keywords
module
score
musical
performance
musical instrument
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201610859490.XA
Other languages
Chinese (zh)
Other versions
CN106340286A (en
Inventor
许炜
程文青
杨楠
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huazhong University of Science and Technology
Original Assignee
Huazhong University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huazhong University of Science and Technology filed Critical Huazhong University of Science and Technology
Priority to CN201610859490.XA priority Critical patent/CN106340286B/en
Publication of CN106340286A publication Critical patent/CN106340286A/en
Application granted granted Critical
Publication of CN106340286B publication Critical patent/CN106340286B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10H—ELECTROPHONIC MUSICAL INSTRUMENTS; INSTRUMENTS IN WHICH THE TONES ARE GENERATED BY ELECTROMECHANICAL MEANS OR ELECTRONIC GENERATORS, OR IN WHICH THE TONES ARE SYNTHESISED FROM A DATA STORE
    • G10H1/00—Details of electrophonic musical instruments
    • G10H1/0008—Associated control or indicating means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/60—Information retrieval; Database structures therefor; File system structures therefor of audio data
    • G06F16/68—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/683—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B15/00—Teaching music
    • G09B15/02—Boards or like means for providing an indication of notes
    • G09B15/023—Electrically operated
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10G—REPRESENTATION OF MUSIC; RECORDING MUSIC IN NOTATION FORM; ACCESSORIES FOR MUSIC OR MUSICAL INSTRUMENTS NOT OTHERWISE PROVIDED FOR, e.g. SUPPORTS
    • G10G1/00—Means for the representation of music
    • G10G1/02—Chord or note indicators, fixed or adjustable, for keyboard of fingerboards

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Library & Information Science (AREA)
  • Acoustics & Sound (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • Data Mining & Analysis (AREA)
  • Auxiliary Devices For Music (AREA)

Abstract

本发明公开了一种通用的实时乐器演奏评价系统，属于音乐信号处理领域。该发明系统包括主控模块、数据库、乐曲选择模块、电子乐谱解析模块、乐谱排版显示模块、音频采集模块、乐谱跟踪模块、乐器演奏正确性评价模块和评分计算模块；通过解析电子乐谱、对乐器演奏进行实时的乐谱跟踪和自动音乐转录，得到演奏的正确性评价结果并实时显示，并对用户的演奏水平计算评分。本发明系统能够根据乐器演奏的正确性，对其进行客观地评价，帮助乐器学习者了解自己的掌握程度，提高学习效率，可以广泛应用于专业的乐器教学和等级考试中。

Figure 201610859490

The invention discloses a general real-time musical instrument performance evaluation system, which belongs to the field of music signal processing. The inventive system includes a main control module, a database, a music selection module, an electronic music score analysis module, a music score typesetting display module, an audio collection module, a music score tracking module, a musical instrument performance correctness evaluation module and a score calculation module; Perform real-time music score tracking and automatic music transcription, get the correctness evaluation results of the performance and display it in real time, and calculate the score for the user's performance level. The system of the invention can objectively evaluate the musical instrument performance according to its correctness, help musical instrument learners to understand their mastery level, improve learning efficiency, and can be widely used in professional musical instrument teaching and grade examinations.

Figure 201610859490

Description

Universal real-time musical instrument playing evaluation system
Technical Field
The invention belongs to the field of music signal processing, and particularly relates to a universal real-time musical instrument playing evaluation system.
Background
Music is inherited from ancient times by its unique artistic charm and value. Musical instrument performance is one of the main expression forms of music, and can be used for mastering temperament and practice and promoting the improvement of aesthetic consciousness and musical literacy. With the improvement of living standard, people's enthusiasm for learning musical instrument playing is increasing, and parents pay more and more attention to the cultivation of child's music literacy. However, learning musical instrument performance requires a professional teacher to perform guidance and a large accumulation of exercises.
With the development of computer technology and music digitization technology, many music-related problems can be handled by computers. Music Extensible Markup Language (MusicXML) is an electronic score exchange and distribution format. MusicXML overcomes the disadvantage that the mature MIDI (Musical Instrument digital interface) format does not support some structural information of music (such as bars, repeats, linguistics, etc.) and other information important to the representation of music.
At present, in patent specification CN104183171A, there is disclosed a system and method for accurate judgment of the performance level of an electronic score-based musical instrument, in which a learner is required to start a performance from the starting position of a score and follow the score to perform at a designated speed. Since the frequency spectrum calculated from the music score and the frequency spectrum calculated from the collected audio information of the acoustic musical instrument are directly compared to obtain a judgment result, it is impossible to clearly judge which note is played more or less when a performance error occurs.
Disclosure of Invention
In view of the above defects or improvement requirements of the prior art, the present invention provides a universal real-time musical instrument playing evaluation system, which aims to obtain and display the performance correctness evaluation result in real time by analyzing the electronic music score, performing real-time music score tracking and automatic music transcription on the musical instrument playing, and calculating and scoring the performance level of the user, thereby solving the problems of player playing inflexibility and low evaluation accuracy in the prior art.
To achieve the above object, the present invention provides a universal real-time musical instrument performance evaluation system, comprising: the system comprises a main control module, a database, a music selection module, an electronic music score analysis module, a music score typesetting display module, an audio acquisition module, a music score tracking module, a musical instrument playing correctness evaluation module and a grading calculation module;
the main control module (10) is used for regulating and controlling each module, transmitting data among the modules and enabling the modules to work coordinately under a correct time sequence;
the database (11) is used for storing a music score file (111), a music score audio file (112), a music score information file (113), a performance audio file (114), a spectrum template file (115) and an evaluation result (116) in MusicXML format;
the music selection module (12) is used for inquiring the database (11) and displaying a music score list, and a player can select a music to be played from the music score list; determining a music score file (111) and a music score audio file (112) in MusicXML format for playing music;
the electronic music score analysis module (13) is used for converting the music score file (111) in the musicXML format selected by the music selection module (12) into a music score information file (113);
the music score typesetting display module (14) is used for displaying the received music score information in display equipment with various sizes, calculating the appropriate number of lines and the number of small sections of each line according to the size of the display equipment, displaying in lines, automatically sliding the music score upwards by one line after playing the middle displayed line, and marking the received playing progress and correctness to the corresponding position in the music score in real time;
the audio acquisition module (15) is used for acquiring sound signals played by a user in real time, storing the sound signals played by the user as a playing audio file (114) into the database (11) by adopting Pulse Code Modulation (PCM) coding;
the music score tracking module (16) is used for converting a music score audio file into a frequency spectrum of a music score audio after receiving the music score audio file, and calculating a corresponding audio fingerprint library; converting the received current performance sound signal into a current performance frequency spectrum, and converting the current performance frequency spectrum into a corresponding audio fingerprint; searching fingerprints matched with the playing audio fingerprints in a music score audio fingerprint library in real time, and counting the positions of playing sound signals corresponding to the music score;
the musical instrument playing correctness evaluating module (17) is used for deducing musical instruments and notes to be played at the current playing moment according to the received music score information and the music score tracking result of the music score tracking module (16); detecting the musical instruments and notes of the currently played sound signals in real time; comparing the detected musical instruments and notes of the current playing sound signals, and evaluating whether the playing is correct or not in real time; and transmitting the evaluation result to a music score typesetting display module (14) in real time; after the user finishes playing, all evaluation results are transmitted to a score calculation module (18);
and the score calculating module (18) is used for summarizing the evaluation result of the musical instrument playing correctness and calculating the score when the playing is finished.
Further, the main control module (10) is specifically configured to:
before a user starts playing the musical instrument, transmitting a music score information file (113) obtained by analysis to a music score typesetting display module (14) and a musical instrument playing correctness evaluation module (17), and transmitting a music score audio file (112) selected by the user to a music score tracking module (16);
when a user starts playing the musical instrument, a playing sound signal collected by the audio collection module (15) is received, and a music score tracking module (16) and a musical instrument playing correctness evaluation module (17) are synchronously started;
when a user plays an instrument, sound signals collected by the audio acquisition module (15) to be played are transmitted to the music score tracking module (16) and the instrument playing correctness evaluation module (17) in real time; transmitting the performance sound signals obtained by real-time statistics of the music score tracking module (16) to a music score display module (14) and a musical instrument performance correctness evaluation module (17) corresponding to the positions in the music score; transmitting the evaluation result of the musical instrument playing correctness evaluation module (17) to a music score typesetting display module (14) in real time; transmitting the evaluation result of the musical instrument playing correctness evaluation module (17) to a score calculation module (18);
when the playing of the user musical instrument is finished, the notification score calculation module (18) collects the received evaluation results of the musical instrument playing correctness and calculates the score.
Further, the score tracking module (16) preferably uses Shazam algorithm to convert the frequency spectrum of the score audio into an audio fingerprint library of the score; and converting the frequency spectrum of the playing sound signal into a corresponding audio fingerprint.
Further, the musical instrument performance correctness evaluating module (17) includes the following sub-modules:
an automatic music transcription sub-module (171) for reading a spectrum template file (115) of a musical instrument corresponding to the score information of the current performance from the database; converting the currently received playing sound signal into a playing frequency spectrum, and then calculating the currently played musical instrument and note by the current playing frequency spectrum and a frequency spectrum template file (115), preferably adopting a non-negative matrix decomposition algorithm
V＝WH，
Substituting the frequency spectrum V and the frequency spectrum template W of the played sound signal to obtain H so as to deduce the played musical instrument or note;
and the comparison evaluation sub-module (172) is used for comparing the currently played musical instruments and notes with the musical instruments and notes which should be played in the music score information at the current moment, and evaluating whether the elements in the music score information are correctly played in real time, wherein the conditions comprise 3 conditions of correct playing, more plays and missed plays.
Generally, compared with the prior art, the technical scheme of the invention has the following technical characteristics and beneficial effects:
(1) after the music score is selected, a player can play from any music score paragraph at will without playing from the starting position of the music score or playing at the designated speed of the music score, and the playing freedom degree is higher;
(2) the evaluation of the playing correctness directly compares the notes played by the musical instrument with the musical instrument, and the evaluation result can more clearly indicate the error reasons.
Drawings
FIG. 1 is a block diagram of the overall architecture of the system of the present invention;
fig. 2 is a functional block diagram of a musical instrument performance correctness evaluating module.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention. In addition, the technical features involved in the embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.
As shown in fig. 1, the system of the present invention comprises: the system comprises a main control module, a database, a music selection module, an electronic music score analysis module, a music score typesetting display module, an audio acquisition module, a music score tracking module, a musical instrument playing correctness evaluation module and a grading calculation module;
the main control module (10) is used for regulating and controlling each module, transmitting data among the modules and enabling the modules to work coordinately under a correct time sequence; the method is specifically used for:
before a user starts playing the musical instrument, transmitting a music score information file (113) obtained by analysis to a music score typesetting display module (14) and a musical instrument playing correctness evaluation module (17), and transmitting a music score audio file (112) selected by the user to a music score tracking module (16);
when a user starts playing the musical instrument, a playing sound signal collected by the audio collection module (15) is received, and a music score tracking module (16) and a musical instrument playing correctness evaluation module (17) are synchronously started;
when a user plays an instrument, sound signals collected by the audio acquisition module (15) to be played are transmitted to the music score tracking module (16) and the instrument playing correctness evaluation module (17) in real time; transmitting the performance sound signals obtained by real-time statistics of the music score tracking module (16) to a music score display module (14) and a musical instrument performance correctness evaluation module (17) corresponding to the positions in the music score; transmitting the evaluation result of the musical instrument playing correctness evaluation module (17) to a music score typesetting display module (14) in real time; transmitting the evaluation result of the musical instrument playing correctness evaluation module (17) to a score calculation module (18);
when the playing of the user musical instrument is finished, the notification score calculation module (18) collects the received evaluation results of the musical instrument playing correctness and calculates the score.
The database (11) is used for storing a music score file (111), a music score audio file (112), a music score information file (113), a performance audio file (114), a spectrum template file (115) and an evaluation result (116) in MusicXML format;
the music selection module (12) is used for inquiring the database (11) and displaying a music score list, and a player can select a music to be played from the music list and determine a music score file (111) and a music score audio file (112) in MusicXML format for playing music;
the electronic music score analysis module (13) is used for converting the music score file (111) in the musicXML format selected by the music selection module (12) into a music score information file (113);
the music score typesetting display module (14) is used for displaying the received music score information in display equipment with various sizes, calculating the appropriate number of lines and the number of small sections of each line according to the size of the display equipment, displaying in lines, automatically sliding the music score upwards by one line after playing the middle displayed line, and marking the received playing progress and correctness to the corresponding position in the music score in real time;
the audio acquisition module (15) is used for acquiring sound signals played by a user in real time, storing the sound signals played by the user as a playing audio file (114) into the database (11) by adopting Pulse Code Modulation (PCM) coding;
the music score tracking module (16) is used for converting a music score audio file into a frequency spectrum of a music score audio after receiving the music score audio file, and calculating a corresponding audio fingerprint library; converting the received current performance sound signal into a current performance frequency spectrum, and converting the current performance frequency spectrum into a corresponding audio fingerprint; searching fingerprints matched with the playing audio fingerprints in a music score audio fingerprint library in real time, and counting the positions of playing sound signals corresponding to the music score; the score tracking module (16) preferably adopts Shazam algorithm to convert the frequency spectrum of the score audio into an audio fingerprint library of the score; converting the frequency spectrum of the playing sound signal into a corresponding audio fingerprint;
the musical instrument playing correctness evaluating module (17) is used for deducing musical instruments and notes to be played at the current playing moment according to the received music score information and the music score tracking result of the music score tracking module (16); detecting the musical instruments and notes of the currently played sound signals in real time; comparing the detected musical instruments and notes of the current playing sound signals, and evaluating whether the playing is correct or not in real time; and transmitting the evaluation result to a music score typesetting display module (14) in real time; after the user finishes playing, all evaluation results are transmitted to a score calculation module (18);
and the score calculating module (18) is used for summarizing the evaluation result of the musical instrument playing correctness and calculating the score when the playing is finished.
The musical instrument performance correctness evaluating module (17) shown in fig. 2 includes the following sub-modules:
an automatic music transcription sub-module (171) for reading a spectrum template file (115) of a musical instrument corresponding to the score information of the current performance from the database; converting the currently received playing sound signal into a playing frequency spectrum, and then calculating the currently played musical instrument and note by the current playing frequency spectrum and a frequency spectrum template file (115), preferably adopting a non-negative matrix decomposition algorithm
V＝WH，
Substituting the frequency spectrum V and the frequency spectrum template W of the played sound signal to obtain H so as to deduce the played musical instrument or note;
and the comparison evaluation sub-module (172) is used for comparing the currently played musical instruments and notes with the musical instruments and notes which should be played in the music score information at the current moment, and evaluating whether the elements in the music score information are correctly played in real time, wherein the conditions comprise 3 conditions of correct playing, more plays and missed plays.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.

Claims (3)

1.一种通用的实时乐器演奏评价系统，其特征在于，本发明系统包括：主控模块(10)、数据库(11)、乐曲选择模块(12)、电子乐谱解析模块(13)、乐谱排版显示模块(14)、音频采集模块(15)、乐谱跟踪模块(16)、乐器演奏正确性评价模块(17)和评分计算模块(18)；1. a general real-time musical instrument performance evaluation system is characterized in that, the system of the present invention comprises: main control module (10), database (11), musical composition selection module (12), electronic musical score analysis module (13), musical score layout a display module (14), an audio acquisition module (15), a score tracking module (16), a musical instrument performance correctness evaluation module (17) and a score calculation module (18); 所述主控模块(10)，用于调控各个模块，在各模块间传递数据，并使它们在正确的时序下协调工作；The main control module (10) is used to regulate and control each module, transfer data between the modules, and make them work in coordination under correct timing; 所述数据库(11)，用于存储MusicXML格式的乐谱文件(111)、乐谱音频文件(112)、乐谱信息文件(113)、演奏音频文件(114)、频谱模板文件(115)和评价结果(116)；The database (11) is used to store music score files (111), music score audio files (112), music score information files (113), performance audio files (114), spectrum template files (115) and evaluation results ( 116); 所述乐曲选择模块(12)，用于查询数据库(11)，显示乐谱列表，演奏者可以从中选择一首将要演奏的乐曲，确定演奏曲目的MusicXML格式的乐谱文件(111)和乐谱音频文件(112)；Described musical piece selection module (12), is used for inquiring database (11), displays musical score list, performer can select a musical piece to be played therefrom, determine musical score file (111) and musical score audio file ( 112); 所述电子乐谱解析模块(13)，用于将乐曲选择模块(12)选取的MusicXML格式的乐谱文件(111)转换为乐谱信息文件(113)；The electronic musical score analysis module (13) is used to convert the musical score file (111) in the MusicXML format selected by the musical composition selection module (12) into a musical score information file (113); 所述乐谱排版显示模块(14)，用于将收到的乐谱信息在各种尺寸大小的显示设备中显示，根据显示设备的尺寸大小，计算出合适的行数和各行小节数，分行显示，当演奏完显示的中间一行后，乐谱自动向上滑动一行，并将收到的演奏的进度和正确性实时地标记到乐谱中的相应位置；The musical score typesetting display module (14) is used to display the received musical score information in display devices of various sizes, and according to the size of the display device, calculate the appropriate number of lines and the number of bars in each line, and display them in separate lines, When the middle row displayed is finished, the score will automatically slide up a row, and the progress and correctness of the received performance will be marked to the corresponding position in the score in real time; 所述音频采集模块(15)，用于实时采集用户演奏的声音信号，采用脉冲编码调制(PCM)编码，将用户演奏的声音信号保存为演奏音频文件(114)存入数据库(11)；Described audio frequency acquisition module (15), is used for collecting the sound signal of user's performance in real time, adopts pulse code modulation (PCM) encoding, and the sound signal of user's performance is saved as performance audio file (114) and is stored in database (11); 所述乐谱跟踪模块(16)，用于收到乐谱音频文件后，转化为乐谱音频的频谱，计算对应的音频指纹库；将接收到的当前演奏声音信号转换为当前演奏频谱，再由当前演奏频谱转换为对应的音频指纹；实时查找乐谱音频指纹库中与演奏音频指纹相匹配的指纹，统计演奏声音信号对应于乐谱中的位置；The musical score tracking module (16) is used to convert the musical score audio frequency spectrum into the musical score audio frequency spectrum after receiving the musical score audio file, and calculate the corresponding audio fingerprint library; the received current performance sound signal is converted into the current performance spectrum, and then the current performance is performed by the current performance spectrum. The spectrum is converted into the corresponding audio fingerprint; the fingerprint matching the performance audio fingerprint in the music score audio fingerprint database is searched in real time, and the position in the music score corresponding to the performance sound signal is counted; 所述乐器演奏正确性评价模块(17)，用于根据接收到的乐谱信息文件(113)和乐谱跟踪模块(16)的乐谱跟踪结果，推求当前演奏时刻应该被演奏的乐器和音符；实时检测当前演奏声音信号的乐器和音符；对比检测到的当前演奏声音信号的乐器和音符，实时评价演奏是否正确；并将评价结果实时传递给乐谱排版显示模块(14)；用户演奏结束后，将全部评价结果传递给评分计算模块(18)；Described musical instrument performance correctness evaluation module (17), is used for according to the musical score tracking result of received musical score information file (113) and musical score tracking module (16), infers the musical instrument and note that should be played at the current performance moment; real-time detection The musical instruments and notes of the currently playing sound signal; compare the detected musical instruments and notes of the currently playing sound signal, and evaluate whether the performance is correct in real time; and transmit the evaluation result to the music score typesetting display module (14) in real time; The evaluation result is passed to the score calculation module (18); 所述乐器演奏正确性评价模块(17)包括以下子模块：The musical instrument performance correctness evaluation module (17) includes the following sub-modules: 自动音乐转录子模块(171)，用于从数据库中读取当前演奏的乐谱信息对应乐器的频谱模板文件(115)；将当前收到的演奏声音信号转化为演奏频谱，再由当前演奏频谱和频谱模板文件(115)推算得到当前演奏的乐器和音符，优选非负矩阵分解算法The automatic music transcription submodule (171) is used to read the spectrum template file (115) of the musical instrument corresponding to the musical score information of the current performance from the database; the currently received performance sound signal is converted into a performance spectrum, and then the current performance spectrum and The spectrum template file (115) is calculated to obtain the currently played musical instrument and note, preferably a non-negative matrix decomposition algorithm V＝WH，V=WH, 其中，代入演奏的声音信号的频谱V和频谱模板W，求出H从而推断出演奏的乐器或音符；Wherein, substitute the frequency spectrum V and frequency spectrum template W of the sound signal of performance, obtain H so as to infer the musical instrument or note of performance; 对比评价子模块(172)，用于将当前演奏的乐器和音符对比当前时刻乐谱信息中应该被演奏的乐器和音符，实时评价乐谱信息中的元素是否被正确演奏，包括演奏正确、多弹、漏弹3种情况；The comparative evaluation sub-module (172) is used to compare the currently played musical instruments and notes with the musical instruments and notes that should be played in the musical score information at the current moment, and to evaluate in real time whether the elements in the musical score information are played correctly, including playing correctly, playing multiple times, 3 situations of missing bombs; 所述评分计算模块(18)用于在演奏结束时，汇总乐器演奏正确性评价结果，计算评分。The score calculation module (18) is used for summarizing the evaluation results of the correctness of musical instrument performance and calculating the score at the end of the performance. 2.根据权利要求1所述的一种通用的实时乐器演奏评价系统，其特征在于，所述主控模块(10)具体用于：2. a kind of general real-time musical instrument performance evaluation system according to claim 1, is characterized in that, described main control module (10) is specially used for: 当用户开始乐器演奏前，将解析得到的乐谱信息文件(113)传递给乐谱排版显示模块(14)和乐器演奏正确性评价模块(17)，将用户选择的乐谱音频文件(112)传递给乐谱跟踪模块(16)；Before the user starts to play the musical instrument, the musical score information file (113) obtained by analysis is passed to the musical score typesetting display module (14) and the musical instrument performance correctness evaluation module (17), and the musical score audio file (112) selected by the user is passed to the musical score. tracking module (16); 当用户开始乐器演奏时，收到音频采集模块(15)采集的演奏声音信号，同步启动乐谱跟踪模块(16)和乐器演奏正确性评价模块(17)；When the user starts to play the musical instrument, receives the performance sound signal collected by the audio collection module (15), and starts the musical score tracking module (16) and the musical instrument performance correctness evaluation module (17) synchronously; 当用户乐器演奏过程中，将音频采集模块(15)收集到演奏的声音信号实时传递到乐谱跟踪模块(16)和乐器演奏正确性评价模块(17)；将乐谱跟踪模块(16)实时统计得到的演奏声音信号对应于乐谱中的位置传递到乐谱显示模块(14)和乐器演奏正确性评价模块(17)；将乐器演奏正确性评价模块(17)的评价结果实时传递给乐谱排版显示模块(14)；将乐器演奏正确性评价模块(17)的评价结果传递给评分计算模块(18)；During the performance of the user's musical instrument, the sound signal collected by the audio acquisition module (15) is transmitted to the musical score tracking module (16) and the musical instrument performance correctness evaluation module (17) in real time; The performance sound signal corresponding to the position in the musical score is transmitted to the musical score display module (14) and the musical instrument performance correctness evaluation module (17); the evaluation result of the musical instrument performance correctness evaluation module (17) is transmitted to the musical score typesetting display module (17) in real time. 14); pass the evaluation result of the musical instrument performance correctness evaluation module (17) to the score calculation module (18); 当用户乐器演奏结束后，通知评分计算模块(18)汇总收到的乐器演奏正确性评价结果，计算评分。When the user's musical instrument performance is finished, the scoring calculation module (18) is notified to summarize the received evaluation results of the correctness of the musical instrument performance, and calculate the score. 3.根据权利要求1所述的一种通用的实时乐器演奏评价系统，其特征在于，所述乐谱跟踪模块(16)优选Shazam算法将乐谱音频的频谱转化为这首乐谱的音频指纹库；将当前演奏频谱，转化为对应的音频指纹。3. a kind of general real-time musical instrument performance evaluation system according to claim 1 is characterized in that, the preferred Shazam algorithm of described musical score tracking module (16) converts the frequency spectrum of musical score audio into the audio fingerprint library of this musical score; The current performance spectrum is converted into the corresponding audio fingerprint.
CN201610859490.XA 2016-09-27 2016-09-27 Universal real-time musical instrument playing evaluation system Expired - Fee Related CN106340286B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201610859490.XA CN106340286B (en) 2016-09-27 2016-09-27 Universal real-time musical instrument playing evaluation system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201610859490.XA CN106340286B (en) 2016-09-27 2016-09-27 Universal real-time musical instrument playing evaluation system

Publications (2)

Publication Number Publication Date
CN106340286A CN106340286A (en) 2017-01-18
CN106340286B true CN106340286B (en) 2020-05-19

Family

ID=57840462

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201610859490.XA Expired - Fee Related CN106340286B (en) 2016-09-27 2016-09-27 Universal real-time musical instrument playing evaluation system

Country Status (1)

Country Link
CN (1) CN106340286B (en)

Families Citing this family (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106935236A (en) * 2017-02-14 2017-07-07 复旦大学 A kind of piano performance appraisal procedure and system
CN106951457A (en) * 2017-02-24 2017-07-14 北京银河润泰科技有限公司 The processing method and processing device of the music score of Chinese operas
CN107316536A (en) * 2017-06-22 2017-11-03 华中师范大学 A kind of virtual dulcimer system based on musical instrument digital interface
JP6708180B2 (en) * 2017-07-25 2020-06-10 ヤマハ株式会社 Performance analysis method, performance analysis device and program
CN107666511B (en) * 2017-08-31 2020-09-29 王奕斌 Method, device and system for remote music examination level
CN107831950A (en) * 2017-12-13 2018-03-23 刘智亮 A kind of music for piano display device of automatic page turning
CN108364661B (en) * 2017-12-15 2020-11-24 海尔优家智能科技（北京）有限公司 Visual voice performance evaluation method, device, computer equipment and storage medium
CN108281127A (en) * 2017-12-29 2018-07-13 王楠珊 A kind of music practice auxiliary system, method, apparatus and storage device
US10861428B2 (en) * 2018-01-10 2020-12-08 Qrs Music Technologies, Inc. Technologies for generating a musical fingerprint
CN108389468A (en) * 2018-03-06 2018-08-10 安徽华熊科技有限公司 A kind of error correction method and device that note is played
CN108805000B (en) * 2018-04-09 2019-12-17 平安科技（深圳）有限公司 Electronic device, music score recognition method and storage medium based on deep learning
CN109686376B (en) * 2019-01-08 2020-06-30 北京雷石天地电子技术有限公司 Song singing evaluation method and system
CN110111761B (en) * 2019-03-28 2022-03-11 深圳市芒果未来科技有限公司 Method for real-time following musical performance and related product
CN110223709B (en) * 2019-05-31 2021-08-27 维沃移动通信有限公司 Recorded audio spectrum display method and terminal equipment
CN110299049B (en) * 2019-06-17 2021-12-17 韶关市启之信息技术有限公司 Intelligent display method of electronic music score
CN113744764B (en) * 2019-09-02 2024-04-26 深圳市平均律科技有限公司 Method for obtaining optimal comparison path of performance time value information and score time value information
CN111128097A (en) * 2019-12-30 2020-05-08 休止符科技深圳有限公司 Electronic music score device and using method thereof
CN111223355A (en) * 2020-01-16 2020-06-02 深圳大学 Musical instrument fingering correction device, system and method and musical instrument
CN115210803B (en) * 2020-03-04 2025-03-28 雅马哈株式会社 Information processing method, information processing system and recording medium
CN111415688A (en) * 2020-04-03 2020-07-14 北京乐界乐科技有限公司 Intelligent recording method for musical instrument
CN111292708A (en) * 2020-04-03 2020-06-16 北京乐界乐科技有限公司 An intelligent management system for musical instruments
CN111653153A (en) * 2020-06-24 2020-09-11 周洁 Music teaching system based on-line card punching scoring
CN111968674B (en) * 2020-07-06 2021-08-10 南京邮电大学 Online instrument fighting platform and ranking list system based on audio signal transcription
CN111737515B (en) * 2020-07-22 2021-01-19 深圳市声扬科技有限公司 Audio fingerprint extraction method and device, computer equipment and readable storage medium
CN111976329B (en) * 2020-08-31 2022-06-21 深圳市月白电子科技有限公司 Staff automatic following method and automatic tracking system in musical instrument playing
CN112735360B (en) * 2020-12-29 2023-04-18 玖月音乐科技(北京)有限公司 Electronic keyboard instrument replay method and system
CN113516999A (en) * 2021-07-05 2021-10-19 武汉市景明云乐网络科技有限公司 Method for demonstrating instrumental fingering and intelligently scoring
CN114399986B (en) * 2022-01-10 2025-07-18 北京有竹居网络技术有限公司 Musical instrument auxiliary exercise method, device, medium and electronic equipment
CN114288653B (en) * 2022-01-21 2025-03-28 腾讯科技（深圳）有限公司 A game control method and related device
CN114758639A (en) * 2022-03-18 2022-07-15 上海妙克信息科技有限公司 Guzheng audio playing method and device, electronic equipment and computer readable medium
CN117863175A (en) * 2023-12-25 2024-04-12 之江实验室 A piano-playing robot offline evaluation system and method

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102959544A (en) * 2010-05-04 2013-03-06 沙扎姆娱乐有限公司 Method and system for synchronizing media
CN105070298A (en) * 2015-07-20 2015-11-18 科大讯飞股份有限公司 Polyphonic musical instrument scoring method and device
CN105280170A (en) * 2015-10-10 2016-01-27 北京百度网讯科技有限公司 Method and device for playing music score
CN105304073A (en) * 2014-07-09 2016-02-03 中国科学院声学研究所 Method and system for estimating multiple music notes of music played by percussion string instruments

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100412196B1 (en) * 2001-05-21 2003-12-24 어뮤즈텍(주) Method and apparatus for tracking musical score
CN100363981C (en) * 2003-10-23 2008-01-23 凌阳科技股份有限公司 electronic music score device
CN1953044B (en) * 2006-09-26 2011-04-27 中山大学 A musical instrument performance prompt and detection system and method based on MIDI files
CN101226526A (en) * 2007-01-17 2008-07-23 上海怡得网络有限公司 Method for searching music based on musical segment information inquest
CN201294089Y (en) * 2008-11-17 2009-08-19 音乐传奇有限公司 Interactive music playing equipment
CN101800045B (en) * 2010-01-21 2012-10-10 冷明 Electronic device for displaying and playing music book
CN102142247A (en) * 2011-03-30 2011-08-03 东南大学 Multifunctional electronic score
CN103514866B (en) * 2012-06-28 2018-01-16 深圳市咪发发科技有限公司 A kind of method and device of instrument playing scoring
CN104183171B (en) * 2013-05-23 2016-10-19 杭州灵机科技有限公司 The system and method that a kind of instrument playing level based on electronic music is precisely passed judgment on
CN103354092B (en) * 2013-06-27 2016-01-20 天津大学 A kind of audio frequency music score comparison method with error detection function
CN103871295B (en) * 2014-03-31 2016-03-16 王紫颐 A kind of multi-functional Zheng electronic score device based on screen display

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102959544A (en) * 2010-05-04 2013-03-06 沙扎姆娱乐有限公司 Method and system for synchronizing media
CN105304073A (en) * 2014-07-09 2016-02-03 中国科学院声学研究所 Method and system for estimating multiple music notes of music played by percussion string instruments
CN105070298A (en) * 2015-07-20 2015-11-18 科大讯飞股份有限公司 Polyphonic musical instrument scoring method and device
CN105280170A (en) * 2015-10-10 2016-01-27 北京百度网讯科技有限公司 Method and device for playing music score

Also Published As

Publication number Publication date
CN106340286A (en) 2017-01-18

Similar Documents

Publication Publication Date Title
CN106340286B (en) Universal real-time musical instrument playing evaluation system
CN107767847B (en) Intelligent piano playing evaluation method and system
CN107967827A (en) A kind of music education exercise system and its method
CN101853650B (en) Music processing and output system and method thereof
CN107369359A (en) A kind of vocal music pronunciation training system
CN112258932B (en) Auxiliary exercise device, method and system for musical instrument playing
CN108182838A (en) A kind of real-time evaluation system of Piano lesson
CN103354092A (en) Audio music-score comparison method with error detection function
CN111968675A (en) Stringed instrument note comparison system based on hand recognition and use method thereof
JP2009047861A (en) Device and method for assisting performance, and program
CN106971645A (en) A kind of interactive teaching instrument for music teaching
CN102103857A (en) Singing scoring system
CN108039180A (en) Method for learning achievement of children language expression exercise and microphone equipment
CN108038146A (en) Musical performance artificial intelligence analysis method, system and equipment
US9092992B2 (en) System and method for music education
CN113126761A (en) Teaching material interaction system based on AR augmented reality and pattern recognition
CN111414841A (en) Automatic video evaluation method based on eye tracker and computer readable storage medium
CN114067767A (en) Music score marking method, system, medium and device based on playing information
CN111554257A (en) Note comparison system of traditional Chinese national musical instrument and use method thereof
CN111179691A (en) Note duration display method and device, electronic equipment and storage medium
Asahi et al. Toward a high performance piano practice support system for beginners
CN111653153A (en) Music teaching system based on-line card punching scoring
TWI768589B (en) Deep learning rhythm practice system
LU503185B1 (en) Intelligent teaching system and method for pipa performance
US20220415289A1 (en) Mobile App riteTune to provide music instrument players instant feedback on note pitch and rhythms accuracy based on sheet music

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20200519

Termination date: 20200927