CN105095069B - A kind of artificial intelligence response system detection method and system - Google Patents
A kind of artificial intelligence response system detection method and system Download PDFInfo
- Publication number
- CN105095069B CN105095069B CN201510346495.8A CN201510346495A CN105095069B CN 105095069 B CN105095069 B CN 105095069B CN 201510346495 A CN201510346495 A CN 201510346495A CN 105095069 B CN105095069 B CN 105095069B
- Authority
- CN
- China
- Prior art keywords
- test
- carousel
- answer
- artificial intelligence
- case
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000013473 artificial intelligence Methods 0.000 title claims abstract description 43
- 230000004044 response Effects 0.000 title claims abstract description 42
- 238000001514 detection method Methods 0.000 title claims abstract description 11
- 238000012360 testing method Methods 0.000 claims abstract description 176
- 238000004321 preservation Methods 0.000 claims description 7
- 238000010586 diagram Methods 0.000 description 4
- 235000013399 edible fruits Nutrition 0.000 description 3
- 238000000034 method Methods 0.000 description 3
- BUGBHKTXTAQXES-UHFFFAOYSA-N Selenium Chemical compound [Se] BUGBHKTXTAQXES-UHFFFAOYSA-N 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 229910052711 selenium Inorganic materials 0.000 description 2
- 239000011669 selenium Substances 0.000 description 2
- 241001269238 Data Species 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000008520 organization Effects 0.000 description 1
- 238000010998 test method Methods 0.000 description 1
Landscapes
- Debugging And Monitoring (AREA)
- Electrically Operated Instructional Devices (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
Abstract
The present invention discloses a kind of artificial intelligence response system detection method and system, including：Acquisition include at least one problem and with the associated multiple carousel answers of each problem；The problem of obtaining test case, selecting to be associated with the test case is as test problem and exports；Obtaining artificial intelligence response system, art is as test words art to the test problem, if the test words art and one of them described associated carousel answer matches of test problem, judge that the testing case passes through.The present invention by obtain a problem and with the relevant multiple carousel answers of the problem, and when test artificial intelligence response system, test words art is matched with multiple carousel answers, to be tested under realization dialogue art carousel scene.
Description
Technical field
The present invention relates to artificial intelligence correlative technology field, particularly a kind of artificial intelligence response system detection method and it is
System.
Background technology
In artificial intelligence field, many outstanding products have been emerged at present, these products are all based at natural language
The automaticmanual intelligent response system of reason has the automated test frame that some increase income, in order to solve at present for such system
The automatic test problem of automaticmanual intelligent response system based on natural language processing, introduce selenium frames+
The automation solutions of python can be realized and put question to input for system in web terminal and answer the automation of output judgement
Test, since most of automatic answering system is for uniquely having inputted unique answer, can directly be matched with result
Mode judge the result of test execution.
Existing automatic technology has the automatic test that web is realized based on selenium frames, in the defeated of web page
Enter frame input problem, and capture returning the result for input frame, problem and result are saved in file, then carry out result judgement,
Next problem is performed again.
The data of the problem of due to intelligent response system and answer are very big, by test script and test data point
From, test data is split in the form of reading and control preservation are convenient in certain structure preservation in the table with behavior unit,
Data line is a session.Different column datas represent different session informations, and following content is included per a line：
1 problem of the prior art of table and answer store method
| Problem | Scene identity | Test account | Predict answer |
However, for high frequency problem, it is possible to multiple answers occur, that is, words art carousel scene occur.However it is existing from
The result that dynamicization testing process and test data preserving type can not meet under words art carousel scene judges.
Invention content
Based on this, it is necessary to which automatic testing process and test data preserving type for the prior art can not meet words
The technical issues of result under art carousel scene judges provides a kind of artificial intelligence response system detection method and system.
A kind of artificial intelligence response system detection method, including：
Test data obtaining step, including：Acquisition include at least one problem and with the associated multiple wheels of each problem
Broadcast answer；
Test case obtaining step, including：The problem of obtaining test case, selecting to be associated with the test case is as survey
Why inscribe and export；
Comparison step, including：Obtaining artificial intelligence response system, art talks about art as test to the test problem, such as
Test words art described in fruit and one of them described associated carousel answer matches of test problem, then judge that the testing case leads to
It crosses.
A kind of artificial intelligence response system test system, including：
Test data acquisition module, is used for：Acquisition include at least one problem and with the associated multiple wheels of each problem
Broadcast answer；
Test case acquisition module, is used for：The problem of obtaining test case, selecting to be associated with the test case is as survey
Why inscribe and export；
Comparison module is used for：Obtaining artificial intelligence response system, art talks about art as test to the test problem, such as
Test words art described in fruit and one of them described associated carousel answer matches of test problem, then judge that the testing case leads to
It crosses.
The present invention by obtain a problem and with the relevant multiple carousel answers of the problem, and test artificial intelligence should
When answering system, test words art with multiple carousel answers is matched, is tested under art carousel scene with realizing to talk with.
Description of the drawings
Fig. 1 is a kind of work flow diagram of artificial intelligence response system detection method of the present invention；
Fig. 2 is the work flow diagram of preferred embodiment；
Fig. 3 is a kind of function structure chart of artificial intelligence response system test system of the present invention.
Specific embodiment
The present invention will be further described in detail in the following with reference to the drawings and specific embodiments.
It is as shown in Figure 1 a kind of work flow diagram of artificial intelligence response system detection method of the present invention, including：
Step S101, including：Acquisition include at least one problem and with the associated multiple carousel answers of each problem；
Step S102, including：Obtain test case, the problem of selecting to be associated with the test case as test problem simultaneously
Output；
Step S103, including：Obtaining artificial intelligence response system, art talks about art as test to the test problem, such as
Test words art described in fruit and one of them described associated carousel answer matches of test problem, then judge that the testing case leads to
It crosses.
The each problem got in step S101 is associated with corresponding carousel answer.Since carousel answer is multiple, because
This, in step s 103, when art if artificial intelligence response system provides difference, test method of the invention also can be well
It adapts to.In art carousel scene is talked about, artificial intelligence response system may provide different answers for same problem, therefore same
One test case, the prior art are also likely to will appear test for the first time by and the situation of second of test crash, and this
Invention can overcome the prior art institute problem, make well since multiple carousel answers being associated with same problem
It obtains test result and stablizes unification in the case where talking about art carousel scene.
Solve the problems, such as a result decision problem for corresponding to multiple answers, this scheme in automatic test by the present invention
It can be adapted for all similar scenes, can significantly promote the automatic test of the automatic answering system based on artificial intelligence
Use-case coverage rate reduces testing cost.
In one of the embodiments, described problem also with for representing the carousel answer of associated carousel answer quantity
Item number is associated preservation.
Carousel answer item number can in step S101 by analyze obtain the problem of and with each problem it is associated more
A carousel answer is calculated, and can also together be provided in offer problem and carousel answer by user.
The step S103 in one of the embodiments, specifically includes：Artificial intelligence response system is obtained to the survey
Art obtains the associated carousel answer item number of the test problem as test carousel answer item as test words art if why inscribing
Number, the test answers with the associated carousel answer of the test problem are compared and matchs, if it is described test talk about art and
One of carousel answer matches, then judge the testing case by and terminate the comparison match, the otherwise test
Words art is compared with another carousel answer to be matched, if the number of test the words art and carousel answer comparison match is more than
The test carousel answer item number, then judge that the testing case fails.
Since problem is associated with preservation with carousel answer item number, it can therefore be appreciated that there is a problem how many carousels to answer
Case matches so that test is more accurate so as to which test words art be compared with all carousel answers.
Described problem and associated carousel answer and carousel answer item number are using two dimension in one of the embodiments,
Table preserves, and every a line of the bivariate table preserves a problem and associated carousel answer and carousel answer item number.
In one of the embodiments：
The step S101, specifically includes：Obtain test account, acquisition include at least one problem and with each problem
Associated multiple carousel answers, described problem and the carousel answer and the test account relating；
The step S102, specifically includes：Artificial intelligence response system is logged in by testing account, obtains test case,
The problem of selecting to be associated with the test case is as test problem and exports.
As highly preferred embodiment of the present invention, problem, carousel answer, carousel answer item number preserving type such as the following table 2 institute
Show：
The problem of 2 preferred embodiment of table and answer preserving type
Carousel words art data organization form is solved by way of increasing carousel answer item number and multiple carousel answers
Problem, when n=1, can be compatible with pervious scheme, n>Answer words art carousel is supported when 1.Wherein, scene end of identification is used to represent
End for a test case.
The work flow diagram of preferred embodiment is illustrated in figure 2, including：
Step S201, read test use-case；
Step S202 opens browser；
Step S203 inputs the URL addresses of artificial intelligence response system to be tested；
Step S204 obtains test account from the bivariate table shown in table 2, and artificial intelligence response is logged in by testing account
System；
Step S205 obtains problem from the bivariate table shown in table 2, and inputs artificial intelligence response system；
Step S206 captures the result of artificial intelligence response system；
Step S207, the carousel answer item number n corresponding to acquisition problem, by the obtained results of step S206 in n item wheels
It broadcasts and matching comparison is carried out in answer, if successful match, perform step S208, else if n carousel answer mismatches
Then perform step S209；
Step S208, implementing result are recorded as, by (PASS), performing step S210；
Step S209, implementing result are recorded as failure (FAIL), perform step S210；
Step S210 determines whether the last one problem in table 2, if it is terminates, and otherwise performs step S205.
A kind of function structure chart of artificial intelligence response system test system of the present invention is illustrated in figure 3, including：
Test data acquisition module 301, is used for：Acquisition includes at least one problem and associated multiple with each problem
Carousel answer；
Test case acquisition module 302, is used for：Test case is obtained, selects to make the problem of association with the test case
For test problem and export；
Comparison module 303, is used for：Obtaining artificial intelligence response system, art is talked about as test to the test problem
Art, if the test words art and one of them described associated carousel answer matches of test problem, judge the test case
Test passes through.
In one of the embodiments, described problem also with for representing the carousel answer of associated carousel answer quantity
Item number is associated preservation.
The comparison module in one of the embodiments, is specifically used for：Artificial intelligence response system is obtained to the survey
Art obtains the associated carousel answer item number of the test problem as test carousel answer item as test words art if why inscribing
Number, the test answers with the associated carousel answer of the test problem are compared and matchs, if it is described test talk about art and
One of carousel answer matches, then judge the testing case by and terminate the comparison match, the otherwise test
Words art is compared with another carousel answer to be matched, if the number of test the words art and carousel answer comparison match is more than
The test carousel answer item number, then judge that the testing case fails.
Described problem and associated carousel answer and carousel answer item number are using two dimension in one of the embodiments,
Table preserves, and every a line of the bivariate table preserves a problem and associated carousel answer and carousel answer item number.
The test data acquisition module in one of the embodiments, is specifically used for：Test account is obtained, obtains packet
Include at least one problem and with the associated multiple carousel answers of each problem, described problem and the carousel answer with it is described
Test account relating；
The test case acquisition module, is specifically used for：Artificial intelligence response system is logged in by testing account, obtains and surveys
The problem of trying out example, selecting to be associated with the test case is as test problem and exports.
Embodiment described above only expresses the several embodiments of the present invention, and description is more specific and detailed, but simultaneously
Cannot the limitation to the scope of the claims of the present invention therefore be interpreted as.It should be pointed out that for those of ordinary skill in the art
For, without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to the guarantor of the present invention
Protect range.Therefore, the protection domain of patent of the present invention should be determined by the appended claims.
Claims (6)
1. a kind of artificial intelligence response system detection method, which is characterized in that including：
Test data obtaining step, including：Acquisition includes at least one problem and is answered with the associated multiple carousels of each problem
Case；
Described problem also with for representing that the carousel answer item number of associated carousel answer quantity is associated preservation；
Described problem and associated carousel answer and carousel answer item number are preserved using bivariate table, the bivariate table it is each
Row preserves a problem and associated carousel answer and carousel answer item number；
Test case obtaining step, including：The problem of obtaining test case, selecting to be associated with the test case is asked as test
It inscribes and exports；
Comparison step, including：Obtaining artificial intelligence response system, art is as test words art to the test problem, if institute
Test words art and one of them described associated carousel answer matches of test problem are stated, then judge that the testing case passes through.
2. artificial intelligence response system detection method according to claim 1, which is characterized in that the comparison step, tool
Body includes：Obtaining artificial intelligence response system, art as test words art, obtains the test problem to the test problem
Associated carousel answer item number is as test carousel answer item number, by test answers and the associated carousel answer of the test problem
Matching is compared, if the test words art and one of carousel answer matches, judge that the testing case passes through
And terminate the comparison match, otherwise the test words art is compared with another carousel answer matches, if the test
The number for talking about art and carousel answer comparison match is more than the test carousel answer item number, then judges that the testing case loses
It loses.
3. artificial intelligence response system detection method according to claim 1, it is characterised in that：
The test data obtaining step, specifically includes：Obtain test account, acquisition include at least one problem and with it is each
The associated multiple carousel answers of problem, problem and the carousel answer and the test account relating；
The test case obtaining step, specifically includes：Artificial intelligence response system is logged in by testing account, test is obtained and uses
Example, the problem of selecting to be associated with the test case, is as test problem and exports.
4. a kind of artificial intelligence response system test system, which is characterized in that including：
Test data acquisition module, is used for：Acquisition includes at least one problem and is answered with the associated multiple carousels of each problem
Case；
Described problem also with for representing that the carousel answer item number of associated carousel answer quantity is associated preservation；
Described problem and associated carousel answer and carousel answer item number are preserved using bivariate table, the bivariate table it is each
Row preserves a problem and associated carousel answer and carousel answer item number；
Test case acquisition module, is used for：The problem of obtaining test case, selecting to be associated with the test case is asked as test
It inscribes and exports；
Comparison module is used for：Obtaining artificial intelligence response system, art is as test words art to the test problem, if institute
Test words art and one of them described associated carousel answer matches of test problem are stated, then judge that the testing case passes through.
5. artificial intelligence response system test system according to claim 4, which is characterized in that the comparison module, tool
Body is used for：Obtaining artificial intelligence response system, art as test words art, obtains the test problem to the test problem
Associated carousel answer item number is as test carousel answer item number, by test answers and the associated carousel answer of the test problem
Matching is compared, if the test words art and one of carousel answer matches, judge that the testing case passes through
And terminate the comparison match, otherwise the test words art is compared with another carousel answer matches, if the test
The number for talking about art and carousel answer comparison match is more than the test carousel answer item number, then judges that the testing case loses
It loses.
6. artificial intelligence response system test system according to claim 4, which is characterized in that
The test data acquisition module, is specifically used for：Obtain test account, acquisition include at least one problem and with it is each
The associated multiple carousel answers of problem, problem and the carousel answer and the test account relating；
The test case acquisition module, is specifically used for：Artificial intelligence response system is logged in by testing account, test is obtained and uses
Example, the problem of selecting to be associated with the test case, is as test problem and exports.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510346495.8A CN105095069B (en) | 2015-06-19 | 2015-06-19 | A kind of artificial intelligence response system detection method and system |
| HK16105721.7A HK1217790B (en) | 2016-05-18 | Test method and system of artificial intelligence answering system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510346495.8A CN105095069B (en) | 2015-06-19 | 2015-06-19 | A kind of artificial intelligence response system detection method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105095069A CN105095069A (en) | 2015-11-25 |
| CN105095069B true CN105095069B (en) | 2018-06-08 |
Family
ID=54575564
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510346495.8A Active CN105095069B (en) | 2015-06-19 | 2015-06-19 | A kind of artificial intelligence response system detection method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105095069B (en) |
Families Citing this family (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106202417A (en) * | 2016-07-12 | 2016-12-07 | 北京光年无限科技有限公司 | A kind of man-machine interaction method for intelligent robot and system |
| CN108427685B (en) * | 2017-02-15 | 2022-03-04 | 北京京东尚科信息技术有限公司 | Automatic response obtaining method of intelligent response system |
| US20180357240A1 (en) * | 2017-06-08 | 2018-12-13 | Facebook, Inc. | Key-Value Memory Networks |
| CN110399469B (en) * | 2018-04-23 | 2022-02-15 | 中国电信股份有限公司 | Customer service robot understanding performance detection fusion method and device |
| CN109933477A (en) * | 2019-03-25 | 2019-06-25 | 北京六行君通信息科技股份有限公司 | Verifying automatic answering telephone set device people should answer art logic matching accuracy device |
| CN111988195B (en) * | 2019-05-24 | 2023-04-07 | 北京京东尚科信息技术有限公司 | Response scheme determination method, device, equipment and medium for packet test |
| CN110908913B (en) * | 2019-11-26 | 2024-03-29 | 阳光保险集团股份有限公司 | Test method and device of return visit robot, electronic equipment and storage medium |
| CN110908919B (en) * | 2019-12-02 | 2021-03-26 | 上海市软件评测中心有限公司 | Response test system based on artificial intelligence and application thereof |
| CN111225114A (en) * | 2020-01-14 | 2020-06-02 | 深圳壹账通智能科技有限公司 | Dial testing method and device, computer equipment and storage medium |
| CN113434383A (en) * | 2020-03-23 | 2021-09-24 | 北京奇虎科技有限公司 | Evaluation method and system for automatically responding APP, computer equipment and storage medium |
| CN111881251B (en) * | 2020-07-23 | 2023-10-13 | 平安科技（深圳）有限公司 | AI telephone sales testing method and device, electronic equipment and storage medium |
| CN116453655B (en) * | 2022-01-07 | 2025-02-25 | 腾讯科技（深圳）有限公司 | Interactive system testing method, device, electronic device and storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1744056A (en) * | 2004-09-04 | 2006-03-08 | 华为技术有限公司 | An Automatic Testing Method for Application |
| CN101226522A (en) * | 2008-02-04 | 2008-07-23 | 黄伟才 | Request-answering system and method supporting interaction between users |
| CN103186466A (en) * | 2011-12-31 | 2013-07-03 | 中国银联股份有限公司 | Information interaction device and method based on automatic generation of associated test cases |
| CN103853842A (en) * | 2014-03-20 | 2014-06-11 | 百度在线网络技术（北京）有限公司 | Automatic question and answer method and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7480899B2 (en) * | 2004-03-22 | 2009-01-20 | International Business Machines Corporation | Method and apparatus for autonomic test case feedback using hardware assistance for code coverage |
-
2015
- 2015-06-19 CN CN201510346495.8A patent/CN105095069B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1744056A (en) * | 2004-09-04 | 2006-03-08 | 华为技术有限公司 | An Automatic Testing Method for Application |
| CN101226522A (en) * | 2008-02-04 | 2008-07-23 | 黄伟才 | Request-answering system and method supporting interaction between users |
| CN103186466A (en) * | 2011-12-31 | 2013-07-03 | 中国银联股份有限公司 | Information interaction device and method based on automatic generation of associated test cases |
| CN103853842A (en) * | 2014-03-20 | 2014-06-11 | 百度在线网络技术（北京）有限公司 | Automatic question and answer method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105095069A (en) | 2015-11-25 |
| HK1217790A1 (en) | 2017-01-20 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105095069B (en) | A kind of artificial intelligence response system detection method and system | |
| CN110348439B (en) | Method, computer readable medium and system for automatically identifying price tags | |
| CN103824055B (en) | A kind of face identification method based on cascade neural network | |
| CN104809062B (en) | A kind of method of testing and system of artificial intelligence response system | |
| CN105488185B (en) | A kind of optimization method and device of knowledge base | |
| US12250279B2 (en) | Method and apparatus for moderating abnormal users, electronic device, and storage medium | |
| CN107204956B (en) | Website identification method and device | |
| CN109525480B (en) | User question collection system and method | |
| CN110222791A (en) | Sample labeling information auditing method and device | |
| CN104504077B (en) | The statistical method and device of web page access data | |
| CN105631486A (en) | image character recognition method and device | |
| CN107948255A (en) | The method for pushing and computer-readable recording medium of APP | |
| CN114840853B (en) | Digital business analysis method based on big data and cloud server | |
| CN111488847B (en) | Sports game video ball-feeding segment acquisition system, method and terminal | |
| CN103823852B (en) | The treating method and apparatus of database server daily record | |
| CN106708841A (en) | Website access path aggregation method and apparatus | |
| CN111832657A (en) | Text recognition method and device, computer equipment and storage medium | |
| CN117789723B (en) | Video content processing method and system based on artificial intelligence | |
| CN106933919A (en) | The connection method of tables of data and device | |
| CN107967641A (en) | Method of Commodity Recommendation, device and computer-readable recording medium | |
| CN107622766A (en) | Method and apparatus for searching for information | |
| CN108154177B (en) | Service identification method, device, terminal equipment and storage medium | |
| CN110516056A (en) | Interactive autonomous learning method, autonomous learning systems and storage medium | |
| CN117319552A (en) | Abnormal number monitoring method and device, storage medium and electronic equipment | |
| KR20200087325A (en) | Method, apparatus and computer program for detecting abusing activity in online service |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: DE Ref document number: 1217790 Country of ref document: HK |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| REG | Reference to a national code |
Ref country code: HK Ref legal event code: GR Ref document number: 1217790 Country of ref document: HK |