CN110309391A

CN110309391A - Scheduling, processing method, device, storage medium and the equipment of content crawl task

Info

Publication number: CN110309391A
Application number: CN201810235800.XA
Authority: CN
Inventors: 王炼; 吕远方; 林佳涛; 尹光宗
Original assignee: Tencent Technology Shenzhen Co Ltd
Current assignee: Tencent Technology Shenzhen Co Ltd
Priority date: 2018-03-21
Filing date: 2018-03-21
Publication date: 2019-10-08

Abstract

The present invention relates to scheduling, processing method, device, storage medium and the computer equipment of a kind of content crawl task, the dispatching method of content crawl task includes: the application content classification for obtaining target application and being configured；The corresponding content crawl task of each application content classification is generated respectively；Task, which is grabbed, for each content configures corresponding task configuration information；Each content crawl task is dispatched to the mobile terminal identification corresponding to idle state respectively；Issue each content crawl task and corresponding task configuration information；The content crawl task issued is used to indicate the corresponding mobile terminal of the mobile terminal identification being dispatched to according to corresponding task configuration information and grabs application content, and the application content belongs to application content classification corresponding with the content crawl task issued.Scheme provided by the present application improves application content crawl efficiency.

Description

Scheduling, processing method, device, storage medium and the equipment of content crawl task

Technical field

The present invention relates to field of computer technology, scheduling, processing method, dress more particularly to a kind of content crawl task It sets, storage medium and equipment.

Background technique

With the development of network technology, installable application is more and more on mobile terminal, these are answered using included It is also more also enriched with content.For user when facing numerous applications selection, it is suitable to select generally according to the evaluating result of application Application installed and used.

Evaluating result usually is analyzed to obtain according to the application content grabbed from application, and in traditional technology, usually Application program is run by mobile terminal self and stores application content, then grabs the application content of mobile terminal storage to carry out It using assessment, takes a long time in this way, leads to the crawl low efficiency of application content.

Summary of the invention

Based on this, it is necessary to for the relatively low problem of content crawl efficiency, provide a kind of content crawl task scheduling, Processing method, device, storage medium and equipment.

A kind of dispatching method of content crawl task, comprising:

Obtain the application content classification that target application is configured；

The corresponding content crawl task of each application content classification is generated respectively；

Task, which is grabbed, for each content configures corresponding task configuration information；

Each content crawl task is dispatched to the mobile terminal identification corresponding to idle state respectively；

Issue each content crawl task and corresponding task configuration information；The content crawl task issued, for referring to Show that the corresponding mobile terminal of the mobile terminal identification being dispatched to grabs application content according to corresponding task configuration information, it is described to answer Belong to application content classification corresponding with the content crawl task issued with content.

A kind of dispatching device of content crawl task, comprising:

Obtain module, the application content classification configured for obtaining target application；

Generation module, for generating the corresponding content crawl task of each application content classification respectively；

Configuration module configures corresponding task configuration information for grabbing task for each content；

Scheduler module, for each content crawl task to be dispatched to the mobile terminal mark corresponding to idle state respectively Know；

Module is issued, for issuing each content crawl task and corresponding task configuration information；The content issued is grabbed Task is taken, the corresponding mobile terminal of the mobile terminal identification being dispatched to is used to indicate and is answered according to the crawl of corresponding task configuration information With content, the application content belongs to application content classification corresponding with the content crawl task issued.

A kind of computer readable storage medium is stored with computer program on the computer readable storage medium, described When computer program is executed by processor, so that the processor executes following steps:

A kind of computer equipment, including memory and processor store computer program, the meter in the memory When calculation machine program is executed by the processor, so that the processor executes following steps:

Above content grabs dispatching method, device, storage medium and the computer equipment of task, for carrying out application content The target application of crawl obtains the application content classification that the target application is configured automatically, and then generates each application respectively The corresponding content of content type grabs task, then each content crawl task is dispatched to the mobile end corresponding to idle state respectively The content of a certain application can be grabbed be separated into multiple tasks in this way by end mark, and cooperated completion by multiple mobile terminals, To which after issuing each content crawl task and corresponding task configuration information, each mobile terminal can share out the work and help one another, and grab respectively The application content for belonging to different application content type is taken, the crawl efficiency of application content is greatly improved.

A kind of processing method of content crawl task, comprising:

It obtains and corresponds to the content crawl task that application content classification generates；

Determine the mobile terminal identification that each content crawl task is respectively dispatched to；

When the corresponding mobile terminal of the mobile terminal identification and locality connection, then obtains and be dispatched to the mobile terminal The content of mark grabs the corresponding task configuration information of task；

The control instruction generated according to the task configuration information is transmitted to the mobile terminal；

Receive the application content that the mobile terminal is grabbed and fed back according to the control instruction；The application content belongs to Application content classification corresponding with the content crawl task of the mobile terminal identification is dispatched to.

A kind of processing unit of content crawl task, comprising:

Module is obtained, for obtaining the content crawl task for corresponding to application content classification and generating；

Determining module, the mobile terminal identification being respectively dispatched to for determining each content crawl task；

The acquisition module is also used to, and when the corresponding mobile terminal of the mobile terminal identification and locality connection, is then obtained Take the content crawl corresponding task configuration information of task for being dispatched to the mobile terminal identification；

Transmission module, for transmitting the control instruction generated according to the task configuration information to the mobile terminal；

Receiving module, the application content for grabbing and feeding back according to the control instruction for receiving the mobile terminal；Institute It states application content and belongs to application content classification corresponding with the content crawl task of the mobile terminal identification is dispatched to.

Above content grabs processing method, device, storage medium and the computer equipment of task, is getting content crawl After task, the mobile terminal identification that each content crawl task is respectively dispatched to is automatically determined, and then in determining mobile terminal mark When knowing corresponding mobile terminal and locality connection, i.e., the content crawl task that acquisition is dispatched to the mobile terminal identification is appointed accordingly Business configuration information can transmit the control instruction that generate according to task configuration information to the mobile terminal in this way, so that the terminal Application content is grabbed according to control instruction.Since content crawl task corresponds to the generation of application content classification, in this way The content crawl of a certain application is separated into multiple tasks, is cooperated by multiple mobile terminals, crawl belongs to difference and answers respectively With the application content of content type, the crawl efficiency of application content is greatly improved.

Detailed description of the invention

Fig. 1 is the applied environment figure for the dispatching method that content grabs task in one embodiment；

Fig. 2 is the flow diagram for the dispatching method that content grabs task in one embodiment；

Fig. 3 is the schematic diagram of task queue in one embodiment；

Fig. 4 is the flow diagram for the processing method that content grabs task in one embodiment；

Fig. 5 is the interface schematic diagram that mobile terminal grabs application content in one embodiment；

Fig. 6 is the interface schematic diagram that mobile terminal grabs application content in another embodiment；

Fig. 7 is the flow diagram of mobile terminal registration in one embodiment；

Fig. 8 is the schematic diagram of mobile terminal state variation in one embodiment；

Fig. 9 is task schedule and the stream that interacts between server, terminal and mobile terminal in treatment process in one embodiment Journey block diagram；

Figure 10 is the function structure chart for the dispatching device that content grabs task in one embodiment；

Figure 11 is the function structure chart for the dispatching device that content grabs task in another embodiment；

Figure 12 is the function structure chart for the processing unit that content grabs task in one embodiment；

Figure 13 is the function structure chart for the processing unit that content grabs task in another embodiment；

Figure 14 is the internal structure chart of computer equipment in one embodiment.

Specific embodiment

In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and It is not used in the restriction present invention.

Fig. 1 is the applied environment figure for the dispatching method that content grabs task in one embodiment.Referring to Fig.1, this applies ring Border includes server 110, terminal 120 and mobile terminal 130.Server 110 is connect by network with terminal 110, terminal 120 and Mobile terminal 130 connects.Wherein, terminal 120 can pass through the connection types such as universal serial bus, wireless network or bluetooth and shifting Dynamic terminal 130 connects.Server 110 is used to execute the dispatching method of content crawl task.Server 110 specifically can be solely Vertical server is also possible to the server cluster of multiple independent server compositions.Terminal 120 specifically can be terminal console Or mobile terminal, mobile terminal specifically can be with tablet computer or laptops etc..Mobile terminal 130 specifically can be mobile phone Or electronics wearable device etc..Wherein, terminal 120 and the quantity of mobile terminal 130 are multiple.

In one embodiment, the processing method of content crawl task can also be applied in application environment as shown in Figure 1. Wherein, terminal 120 is used to execute the processing method of content crawl task.

Fig. 2 is the flow diagram for the dispatching method that content grabs task in one embodiment.The present embodiment is mainly with this Method is applied to the server 110 in above-mentioned Fig. 1 to illustrate.Referring to Fig. 2, the dispatching method which grabs task is specific Include the following steps:

S202 obtains the application content classification that target application is configured.

Wherein, target application is the application program that pending application content crawls.Application program (APP, Application) Be in order to obtain certain result and by computer etc. have information processing capability terminal execute code instruction sequence, or The symbolism instruction sequence or symbolism statement sequence of code instruction sequence can be automatically converted into.Application program is such as Social application program, information application program, service for life application program or game application etc..

Application content is data included by application program.Application content may include that visual application page shows number According to, such as can be in picture, text or the audio-video etc. that application page is shown.Application content, which may also comprise, generates application page institute The source codes data such as the style sheet code or page structure code of foundation, such as HTML (HyperText Markup Language, HyperText Markup Language) source code etc..

Application content classification is the class categories that application content is classified according to certain classification foundation.According to different Classification foundation, available different application content classification.For example, application content can be answered according to data type classifications With content type such as picture classification, video classification or text categories etc..For another example, by application content according to content topic point Application content classification such as Sport Class, political situation of the time classification or financial classification etc. can be obtained in class.For example, Tencent's news application It include multiple columns such as " important news " " video " " two Conferences " " Shenzhen " and " amusement " in program, these columns are by Tencent's news The application content classification configured.

In one embodiment, the application content classification that target application is configured can be classifies according to a kind of classification foundation Obtained class categories are also possible to the class categories classified according to a variety of classification foundations.

In one embodiment, the application content classification that target application is configured, can be the developer by target application It is configured during development goal application；It is also possible to determining unified when practicing content crawl configure by server 's；It can also be and be respectively configured according to the application type of target application when practicing content crawl by server determining.

Specifically, server can obtain the application identities for the application program that pending application content crawls, to determine these Target application corresponding to application identities, then obtain the application content classification that target application is configured.

In a specific embodiment, service implement body can be server corresponding with using application program is downloaded. Using downloading application program it is to provide the flatbed application program using download function, user can be by installing on mobile terminal The application program of application downloading application program is uploaded to using downloading application program downloading.It is such as answered using downloading application program With precious or application shop etc..

Further, application developer can upload to the application of exploitation using downloading application program, such user The application of its exploitation is downloaded in application downloading application program.In the present embodiment, server can then be obtained and be uploaded under application The application for carrying application program is used as target application, and then obtains the application content classification that target application is configured.It is understood that It is in the present invention, not to be defined to the source of target application, does not upload to the other application using downloading application program Also it can be used as target application.

S204 generates the corresponding content crawl task of each application content classification respectively.

Wherein, content crawl task is to carry out the task of application content crawl.Specifically, server is obtaining target application It, can one content crawl task of self-generating each for each application content classification after the application content classification configured.So just The application content of a certain application can be grabbed and be separated into multiple subtasks, and completed each subtask and only need to grab the target Using the application content for belonging to some application content classification.

For example, including that " important news " " video " " two Conferences " " Shenzhen " and " amusement " etc. are more in Tencent's news application program A column, these columns are the application content classification for being configured Tencent's news.In this way server can " column " be dimension Degree, crawls task as a content for crawling for the application content of each column.

S206 grabs task for each content and configures corresponding task configuration information.

Wherein, task configuration information is the data configured to task execution logic.Task configuration information specifically can be with Including application logon data, application operating instruction or application content acquisition modes etc..It is concretely logged in using logon data The account based on.When application content could be obtained by needing to log in application, task configuration information then should include that application is stepped on Record data, such as the social application account of social application program etc..Application operating instruction is the codeization instruction of operation application, than Such as application page rolls instruction, page control click commands and application page jump instruction.Application content acquisition modes are Obtain the mode of application content, such as page screenshot mode or data extracting mode etc..

Each content crawl task is dispatched to the mobile terminal identification corresponding to idle state by S208 respectively.

Wherein, idle state is can to transfer content crawl task and not yet dispatch the state for having content crawl task.? That is expression can grab task to the mobile terminal mark when mobile terminal identification corresponds to idle state with Scheduling content Know.Specifically, server can correspond to each mobile terminal identification difference recording status information.Status information is mobile terminal mark Know current corresponding state.In the present embodiment, service implement body can record mobile terminal identification pair by equipment state list The status information answered.

Specifically, task schedule condition can be arranged in server in advance, appointed the crawl of each content according to task schedule condition Business is dispatched to the mobile terminal identification corresponding to idle state respectively.Task schedule condition may include the constraint item of task quantity Part.The constraint condition of task quantity, for example it is only capable of dispatching N number of (such as 1) content crawl task every time to a mobile terminal mark Know.Task can be grabbed to avoid a large amount of contents in this way and concentrate on a mobile terminal, so that each mobile terminal has task that can locate Reason, improves the utilization efficiency of mobile terminal.Task schedule condition may also include the constraint condition of task schedule sequence, such as according to The generation time of content crawl task successively dispatches, and the content crawl task being previously generated formerly is dispatched, in rear generation Hold crawl task in rear scheduling.The timeliness of task processing can be improved in this way.

It in one embodiment, can be one or more corresponding to the quantity of the mobile terminal identification of idle state. When corresponding to the quantity of mobile terminal identification of idle state is one, the content of preset quantity can be grabbed task schedule extremely The mobile terminal identification；Task is grabbed for remaining content, then waits and inquires shifting that is newly-increased and corresponding to idle state It is dispatched again after dynamic terminal iidentification.Wherein, preset data is the task the upper limit of the number that mobile terminal can be dispatched to.

When it is multiple for corresponding to the quantity of mobile terminal identification of idle state, each content can be grabbed task and adjusted respectively One of them in mobile terminal identification for corresponding to idle state is spent to these, and is dispatched to the interior of each mobile terminal identification The quantity for holding crawl task is no more than preset quantity.Remaining content grabs task if it exists, then wait inquire it is newly-increased and Corresponding to being dispatched again after the mobile terminal identification of idle state.

In a specific embodiment, each content can be grabbed task and be dispatched to different and correspond to respectively by server In the mobile terminal identification of idle state.Task is grabbed for remaining content, then is waited and is inquired newly-increased and correspond to sky It is dispatched again after the mobile terminal identification of not busy state.

S210 issues each content crawl task and corresponding task configuration information；The content crawl task issued, for referring to Show that the corresponding mobile terminal of the mobile terminal identification being dispatched to grabs application content according to corresponding task configuration information, using interior Hold and belongs to application content classification corresponding with the content crawl task issued.

Specifically, task configuration information can be the data that mobile terminal can be handled directly, then server can directly by Each content crawl task and corresponding task configuration information, are issued to corresponding with the mobile terminal identification being accordingly dispatched to respectively Mobile terminal.Mobile terminal each in this way can grab application content according to the task configuration information issued, to execute content crawl Task.

For example, content crawl task includes content crawl task A, content crawl task B and content crawl task C； Wherein content crawl task A is scheduled to mobile terminal identification a, and content crawl task B is scheduled to mobile terminal identification b, interior Hold crawl task C and is scheduled to mobile terminal identification c.So content directly can be grabbed task A and corresponding task by server Configuration information is issued to the corresponding mobile terminal of mobile terminal identification a, and content is grabbed task B and corresponding task configuration information It is issued to the corresponding mobile terminal of mobile terminal identification b, content is grabbed into task C and corresponding task configuration information is issued to shifting The dynamic corresponding mobile terminal of terminal iidentification c.

In one embodiment, task configuration information can be the data that mobile terminal can not be handled directly, then servicing Device each content can be grabbed task and corresponding task configuration information is issued to intermediate equipment, is configured task by intermediate equipment Information, which is converted into after the executable control instruction of mobile terminal, is issued to mobile terminal, mobile terminal execute again the control instruction with Grab application content.Wherein, intermediate equipment is the equipment that can directly handle task configuration information.Specifically it can be shown in Fig. 1 Terminal 120.Specifically, terminal can generate control instruction according to task configuration information by the debugging tool installed thereon.It adjusts Trial work tool such as Android debugs bridge (Android Debug Bridge) or Appium (mobile terminal automation framework) etc..

In one embodiment, content is grabbed task by server and corresponding task configuration information is issued to centre simultaneously Content first can also be grabbed mission dispatching to intermediate equipment, grabbed in intermediate equipment with the content issued by equipment (terminal 120) When the corresponding mobile terminal identification of the mobile terminal identification of task schedule extremely connects, then the content issued crawl task is corresponding Task configuration information is issued to intermediate equipment.

Above content grabs the dispatching method of task, and for carrying out the target application of application content crawl, obtaining automatically should The application content classification that target application is configured, and then generate the corresponding content crawl of each application content classification respectively and appoint Business, then each content crawl task is dispatched to the mobile terminal identification corresponding to idle state respectively, it can be answered in this way by a certain Content crawl is separated into multiple tasks, and cooperated completion by multiple mobile terminals, to appoint issuing each content crawl After business and corresponding task configuration information, each mobile terminal can share out the work and help one another, and crawl belongs to different application content type respectively Application content, greatly improve the crawl efficiency of application content.

In one embodiment, when target application belongs to open access style in application, the scheduling of content crawl task Method further include: obtain the public identity of access target application；The corresponding content crawl task of each public identity is generated respectively；Its In, application content belongs to application content classification corresponding with the content crawl task issued, alternatively, grabbing with the content issued The corresponding public identity of task is corresponding.

Wherein, open access style application is the application used by open interface for third party.Third party can be by opening Interface registration public identity is put, then promotional content is issued by public identity.It is understood that since the promotional content can be The application page of target application is shown, then the promotional content also belongs to application content.Promotional content such as public's article or Advertisement information etc..

Public identity can be public service account, be also possible to third party's account associated with public service account. Public identity is the character string of unique identification public service account.For example, open access style application can be wechat application, connect The public identity for entering wechat application can be wechat public platform；For another example, open access style application can be Alipay application, The public identity of access Alipay application can be life number.

Specifically, server can also check whether the target application is that open access style is answered when determining target application With not being open access style in application, the step of then directly executing S202 to S210 in previous embodiment in the target application. It is open access style in application, in the application content classification that acquisition target application is configured in the target application, generates respectively When the corresponding content crawl task of each application content classification, the public identity of access target application can also be obtained；It generates respectively The corresponding content of each public identity grabs task.In this way, the content crawl task generated can be corresponding to application content classification , then the application content for executing task acquisition is to belong to the application content of the application content classification；The content of generation grabs Task is also possible to corresponding to public identity, then the application content for executing task acquisition is to belong to answering for the public identity Use content.

It is appreciated that for the target application for belonging to open access style application, server can also only obtain access target The public identity of application generates the corresponding content crawl task of each public identity respectively；And it no longer obtains target application and is configured Application content classification, generate each application content classification corresponding content crawl task respectively.

In the present embodiment, belong to open access style in target application to correspond in application, generating each public identity respectively Content grab task, in this way can to each public identity generate a task, cooperated completion by multiple mobile terminals, Each mobile terminal grabs the application content for belonging to different public identities respectively, greatly improves the crawl efficiency of application content. But also a mobile terminal can be concentrated on to avoid the application content crawl for belonging to a large amount of public identities, so that each mobile terminal There is task that can handle, improves the utilization efficiency of mobile terminal.

In one embodiment, the dispatching method of content crawl task further include: each content of generation is grabbed into task, is pressed The corresponding task generation time sequentially adds task queue.Each content crawl task is dispatched to respectively corresponding to idle state Mobile terminal identification, comprising: inquiry currently corresponds to the mobile terminal identification of idle state；It is successively taken from the head of the queue of task queue Content grabs task out；The content successively taken out crawl task is dispatched to mobile terminal marks inquire and different respectively Know.

Wherein, task queue is a single-entry single-out queue, is mainly used for cache contents crawl task.Specifically, it takes Content crawl task can be added task queue, will generated each by business device in one content crawl task of every generation Content grabs task, sequentially adds task queue by the corresponding task generation time.

Further, server can not be empty in task queue and there is the mobile terminal identification for corresponding to idle state When, content crawl task is successively taken out from the head of the queue of task queue, then the content successively taken out crawl task is dispatched to respectively The mobile terminal identification inquired.Server can also grab task to empty task queue in new content and presence corresponds to When the mobile terminal identification of idle state, content crawl task is successively taken out from the head of the queue of task queue, then will successively take out Content crawl task is dispatched to mobile terminal identifications inquire and different respectively.

Task is grabbed for remaining content, then waits and inquires mobile terminal mark that is newly-increased and corresponding to idle state It is dispatched again after knowledge.It, then, will when new content being waited to grab task for the remaining mobile terminal identification corresponding to idle state Newly-increased content grabs task schedule to these mobile terminal identifications.

In the present embodiment, each content is grabbed into task, sequentially adds task queue by the corresponding task generation time, presses According to the first in first out of task queue, the content crawl task being previously generated formerly is dispatched, and is appointed in the content crawl of rear generation Business only dispatches a content every time and grabs task to a mobile terminal identification in rear scheduling.That is, each mobile whole A task is only distributed at end every time, and after the completion of task, new task of reallocating improves the timeliness of task processing in this way.

In one embodiment, each content crawl task and corresponding task configuration information are issued, comprising: receive terminal hair Rising for task pulls request；Request is pulled according to task, each content is grabbed into mission dispatching to terminal；When terminal is connected to under It is when the corresponding mobile terminal of the content crawl task schedule mobile terminal identification extremely of hair, the content issued crawl task is corresponding Task configuration information be issued to terminal；Wherein, the task configuration information issued is used to indicate terminal and is matching confidence according to task After breath generates control instruction, mobile terminal is controlled by control instruction, application content is grabbed according to control instruction.

Wherein, it is the computer instruction for request content crawl task that task, which pulls request,.Specifically, terminal can Task is initiated to server and pulls request, and server is after the task that receives initiates request, by scheduled to corresponding to idle shape The content of the mobile terminal identification of state grabs mission dispatching to the terminal.Task, which pulls request, periodically can initiate or detect spy Determine to initiate when event.Particular event can be new mobile terminal and be connected to the terminal.It is appreciated that new movement here Terminal can be the mobile terminal for not being connected to terminal, is also possible to once be connected to terminal and be again coupled to after disconnecting To the mobile terminal of terminal.

Further, terminal can inquire the mobile terminal identification for being connected to local mobile terminal, when the content issued is grabbed When taking the mobile terminal identification of task schedule extremely consistent with the mobile terminal identification inquired, then terminal can be further from service The content crawl corresponding task configuration information of task is obtained on device, and then control instruction is generated according to task configuration information, then Control instruction is transmitted to the corresponding mobile terminal of mobile terminal identification of the content crawl task schedule issued extremely, so that mobile Terminal grabs application content according to control instruction.

For example, terminal inquiry to the mobile terminal identification for being connected to local mobile terminal be a and b, issue Content crawl task B is scheduled to mobile terminal identification b, and content crawl task C is scheduled to mobile terminal identification c.So this When, due to terminal mobile terminal connection corresponding with mobile terminal identification b, then content is obtained from server and grab task B phase Control instruction is transmitted to mobile terminal mark after generating control instruction according to the task configuration information by the task configuration information answered The corresponding mobile terminal of b is known, so that the corresponding mobile terminal of mobile terminal identification b can grab application content according to control instruction. And the mobile terminal identification for inquiring is a, due to not having content crawl task schedule to the mobile terminal identification, is then refused Processing.Mobile terminal identification c is scheduled to for content crawl task C, since terminal is not currently connected to mobile terminal identification c Corresponding mobile terminal, is also disregarded.

Further, terminal issues the control instruction of generation according to task configuration information, is for controlling mobile terminal Instruction.In one embodiment, terminal can be according to the mapping relations of task configuration information and control instruction, query task configuration The control instruction that information is mapped to.Control instruction is such as: referring to using installation instruction, using instruction of restarting, jump instruction, click It enables, page scroll instruction, text input instruction, page data acquisition instruction and source code acquisition instruction etc..

In the present embodiment, terminal automatically determines each content crawl each self-regulated of task after getting content crawl task The mobile terminal identification of degree extremely, and then in the determining corresponding mobile terminal of mobile terminal identification and locality connection, that is, it obtains It is dispatched to the content crawl corresponding task configuration information of task of the mobile terminal identification, can be transmitted in this way to the mobile terminal According to the control instruction that task configuration information generates, so that the terminal grabs application content according to control instruction.Pass through end in this way The transfer at end issues task, reduces the workload of server.

In one embodiment, the dispatching method of content crawl task further include: receive the task execution knot that terminal reports Fruit；Task action result corresponds to content corresponding with the task configuration information issued and grabs task；When task action result table When showing task execution failure, content corresponding with task action result crawl task is rejoined to the tail of the queue of task queue.

Wherein, task action result is the result of mobile terminal execution content crawl task.Task action result includes appointing Business runs succeeded, task execution failure and task execution are overtime.Specifically, the task that terminal can receive mobile terminal feedback is held Row is as a result, and report to server for the task action result.When server received task implementing result indicates that task is held When going successfully, then determine that task action result is completed.When server received task implementing result indicates task execution failure When, then determine that task action result does not complete, the content crawl task of execution failure rejoined to the tail of the queue of task queue, It is executed again with waiting.

It is appreciated that the content crawl task for rejoining task queue can be original content crawl task, it can also To be that the content regenerated grabs task.But the two is the task of corresponding crawl same application content and is that failure retries Task.In the present embodiment, provide failed tasks retries approach, provides the success rate of task execution.

In one embodiment, the dispatching method of content crawl task further include: since when issuing task configuration information Timing；When timing duration reaches the first preset duration and does not receive task action result that terminal reports, then will with issue Task configuration information corresponding content crawl task rejoin the tail of the queue of task queue.

Wherein, the first preset duration is pre-set waiting-timeout duration.It is more than the first preset duration when waiting time When, it can determine that server connect time-out with terminal.Specifically, server can the timing since when issuing task configuration information, when When not receiving the task action result that terminal reports when timing duration reaches the first preset duration yet, then judge that task execution is super When, the content crawl task for executing time-out is rejoined to the tail of the queue of task queue.

It is appreciated that the content crawl task for rejoining task queue can be original content crawl task, it can also To be that the content regenerated grabs task.But the two is the task of corresponding crawl same application content and time-out retries Task.In the present embodiment, provide overtime task retries approach, provides the success rate of task execution.

Fig. 3 shows the schematic diagram of task queue in one embodiment.With reference to Fig. 3, task of task queue is added can be with The content crawl task updated including newly-increased content crawl task, timing, the content crawl task and time-out that unsuccessfully retry The content crawl task retried.Since application content has timeliness, it is therefore desirable to periodically grab, therefore be updated including timing again Content grab task.These tasks generate time sequencing according to task and task queue are added, in task schedule, it then follows first Into the principle first gone out, when there is the mobile terminal identification corresponding to idle state and when task queue is empty, successively taken from head of the queue Content crawl task is allocated out；Or it increases content crawl task newly in the queue and has had the movement corresponding to idle state Content crawl task is successively taken out from head of the queue when terminal iidentification to be allocated.

It is appreciated that the content crawl task of timing update, the content crawl task unsuccessfully retried and time-out retried Content grab task task generate the time it is not initial content crawl task the generation time, timing update when Between, time for retrying of time for unsuccessfully retrying and time-out.

In one embodiment, the dispatching method of content crawl task further include: extremely by each content crawl task schedule Mobile terminal identification is updated to correspond to busy condition by corresponding to idle state；The content crawl for corresponding to and issuing is received to appoint The task action result that business reports；When receiving task action result, the corresponding content of task action result is grabbed into task The mobile terminal identification being dispatched to is updated to correspond to idle state by corresponding to busy condition.

Wherein, idle state be it is schedulable have content crawl task and not yet scheduling have content crawl task state.Phase It answers, busy condition is the state that scheduling has content crawl task and non-scheduling to have content to grab task.Specifically, it services Device, by the mobile terminal identification of content crawl task schedule extremely, can be updated after carrying out task schedule by corresponding to idle state To grab task to same mobile terminal identification to avoid repetitive schedule content corresponding to busy condition.

Further, server also can receive the task action result that the content crawl task for corresponding to and issuing reports；? When receiving task action result, the Predicated execution content crawl task mobile terminal executed task (though success with It is no), it can receive content crawl task schedule, just the mobile end by the corresponding content crawl task schedule of task action result extremely End mark is updated to correspond to idle state, avoids the mobile terminal of practical free time that from can locating without task by corresponding to busy condition Reason.

In the present embodiment, it after the scheduled task of the mobile terminal of the task of execution or task execution are complete, timely updates The corresponding state of the mobile terminal identification of the mobile terminal avoids repetitive schedule content from grabbing task to same mobile terminal mark Know, or there are the mobile terminal free time can handle without task.

Fig. 4 is the flow diagram for the processing method that content grabs task in one embodiment.The present embodiment is mainly with this Method is applied to the terminal 120 in above-mentioned Fig. 1 to illustrate.Referring to Fig. 4, the processing method of content crawl task is specifically wrapped Include following steps:

S402 is obtained and is corresponded to the content crawl task that application content classification generates.

Specifically, terminal can determine the target application of pending application content crawl, determine that target application configured answers With content type, the corresponding content crawl task of each application content classification is generated respectively, to get corresponding to application content The content that classification generates grabs task.

In one embodiment, terminal can initiate task to server and pull request, and server is initiated in the task that receives After request, the scheduled content to the mobile terminal identification for corresponding to idle state is grabbed into mission dispatching to the terminal.Task Initiation when periodically can initiate or detect particular event that pulls request.Particular event can be new mobile terminal and be connected to The terminal.It is appreciated that new mobile terminal here can be the mobile terminal for not being connected to terminal, it is also possible to once connect It is connected to terminal and again connects to the mobile terminal of terminal after disconnecting.

In one embodiment, terminal can also belong to open access style in target application and correspond in application, obtaining The content that public identity generates grabs task.Wherein, the content crawl task generated corresponding to public identity can be terminal certainly What body generated, it is also possible to from server pull.

S404 determines the mobile terminal identification that each content crawl task is respectively dispatched to.

S406 is then obtained when the corresponding mobile terminal of mobile terminal identification and locality connection and is dispatched to mobile terminal mark The content of knowledge grabs the corresponding task configuration information of task.

Specifically, the task configuration information that terminal obtains can be when grabbing task to server pull content, together Pull and be stored in it is local, when needed, that is, when the corresponding mobile terminal of mobile terminal identification and when locality connection, then The content crawl corresponding task configuration information of task for being dispatched to mobile terminal identification is obtained from the task configuration information of storage.

In one embodiment, the task configuration information that terminal obtains is also possible to grab task tune in the content pulled When the degree corresponding mobile terminal of mobile terminal identification extremely and locality connection, then from server pull.

S408 transmits the control instruction generated according to task configuration information to mobile terminal.

S410 receives the application content that mobile terminal is grabbed and fed back according to control instruction；Application content belongs to and dispatches Content to mobile terminal identification grabs the corresponding application content classification of task.

Specifically, mobile terminal executes the control instruction receiving control instruction, and grabs after executing the instruction The application content of target application, such as the page data or source code data of the application page of current presentation, then answering crawl With content feed to terminal.

In one embodiment, control instruction can be once issuing and be instructed by the codeization of mobile terminal execution Sequence.Mobile terminal it is every execute code instruction sequence in a codeization instruction after, can will currently grab application in Appearance feeds back to server, can also be after having executed code instruction sequence, extremely by whole application content unified feedbacks of crawl Server.

In one embodiment, control instruction can be repeatedly issuing and be instructed by the codeization of mobile terminal execution Sequence.Wherein, codeization instruction next time is dependent on mobile terminal according to the application content of last code instruction feedback.

For example, Fig. 5 shows the interface schematic diagram of mobile terminal crawl application content in one embodiment.Assuming that mesh Mark application is Tencent's news application program, then grabbing task for corresponding to the content that " important news " column generates.Terminal can root Control instruction, which is generated, according to task configuration information (chooses instruction → application content using installation instruction → using enabled instruction → column Fetching instruction → page scroll instructs (page jump instruction) → application content fetching instruction →... → applying exit instruction → to answer With unloading command), then it is issued to mobile terminal, mobile terminal then executes control instruction analog subscriber operation application, and grabs Application content.With reference to Fig. 5 left figure, mobile terminal is after installation targets application, application icon of the analog user to target application 510 clicking operation starts target application, then chooses target column 520 such as in the application page shown after starting target application Shown in Fig. 5 right figure.Mobile terminal can grab the page presentation data or source code data of the page, then analog subscriber operation again Page presentation data or source code data that page jump perhaps grabs new page after page scroll again are carried out, until crawl The specified application content of complete control instruction, or crawl number reach the specified number of control instruction.It is appreciated that mobile whole When target application is installed on end, it is no longer necessary to reinstall and unload target application.

For another example, Fig. 6 shows the interface schematic diagram of mobile terminal crawl application content in another embodiment.Assuming that mesh Mark application is wechat application program, then grabbing task for corresponding to the content that public identity generates.Terminal can be according to task Configuration information generates control instruction (text input instruction) and is issued to mobile terminal, and mobile terminal then executes control instruction simulation User's operation is applied as shown in Fig. 6 left figure, into searched page, in the search box 610 of searched page input public identity into After row search, crawl application content feeds back to terminal.Terminal generates the (publication of new control instruction further according to the application content of feedback Content checks instruction) and issue.Mobile terminal then executes control instruction analog subscriber operation application as shown in Fig. 6 right figure, looks into It sees the publication content 620 issued by the public identity and grabs publication content feed to terminal, the control of terminal regeneration Cheng Xin Instruction, until mobile terminal has grabbed the specified application content of control instruction, or crawl number reaches what control instruction was specified Number.

Above content grabs the processing method of task, after getting content crawl task, automatically determines each content crawl The mobile terminal identification that task is respectively dispatched to, and then in the determining corresponding mobile terminal of mobile terminal identification and locality connection When, that is, the content crawl corresponding task configuration information of task for being dispatched to the mobile terminal identification is obtained, it in this way can be to the shifting The control instruction that dynamic terminal transmission is generated according to task configuration information, so that the terminal grabs application content according to control instruction. Since content crawl task corresponds to the generation of application content classification, the content of a certain application can be grabbed be separated into this way Multiple tasks are cooperated by multiple mobile terminals, and crawl belongs to the application content of different application content type respectively, greatly Improve the crawl efficiency of application content.

In one embodiment, it obtains and corresponds to the content crawl task that application content classification generates, comprising: generate task Pull request；Task is pulled into request and is sent to server；It receives server and the content crawl that request issues is pulled according to task Task；Content grabs task and generates corresponding to application content classification, and content crawl task is dispatched to correspondence by server respectively In the mobile terminal identification of idle state.

Specifically, terminal can initiate task to server and pull request, and server, will after the task that receives initiates request The scheduled content to the mobile terminal identification for corresponding to idle state grabs mission dispatching to the terminal.Task pulls request can Initiation when particular event is initiated or detected in timing.These tasks that terminal pulls, which pull request, is answered by server It is generated with content type, and content crawl task is dispatched to the mobile terminal identification corresponding to idle state by server respectively.

In the present embodiment, task is issued by the transfer of terminal, reduces the workload of server.

In one embodiment, the processing method of content crawl task further include: scanning is newly-increased by universal serial bus It is connected to local mobile terminal；When scanning to when being connected to local mobile terminal by the way that universal serial bus is newly-increased, then obtain Take mobile terminal device information corresponding to the mobile terminal of newly-increased connection；Eventually by the mobile terminal identification of mobile terminal and movement End equipment information correspondence reports to server；The mobile terminal device information reported is used to indicate server for mobile terminal identification Correspondence markings are idle state.

Wherein, universal serial bus (Universal Serial Bus, USB) is the serial ports for connecting external equipment.Terminal It can be connect by universal serial bus between mobile terminal, to carry out data transmission.Mobile terminal device information is that reflection moves The essential information of dynamic terminal configuration.Mobile terminal device information such as mobile terminal model, mobile terminal version model, movement are eventually Hold physical address and mobile terminal sequence number etc..

Specifically, terminal, which can be scanned, is connected to local mobile terminal by the way that universal serial bus is newly-increased, in scanning to newly When the mobile terminal of increasing, then to mobile terminal device information corresponding to newly-increased mobile terminal is obtained, to the mobile terminal It is registered.In the present embodiment, terminal specifically can by local monitoring thread periodic scanning mobile terminal identification list, It was found that when increasing mobile terminal identification in mobile terminal identification list newly, i.e., to the newly-increased corresponding mobile terminal of mobile terminal identification It is registered.

Further, terminal can be instructed by universal serial bus, the mobile terminal transmission device acquisition of information of Xiang Xinzeng, Mobile terminal feeds back mobile terminal device information to terminal after receiving apparatus information acquiring instruction.Terminal again will be new The mobile terminal identification of increasing mobile terminal is corresponding with the mobile terminal device information got to report to server, indicates server The mobile terminal device information is saved, and is idle state by the mobile terminal identification correspondence markings, to complete the mobile terminal Registration.In one embodiment, mobile terminal identification can be mobile terminal device information, also may include in mobile terminal Facility information.

In this way, the mobile terminal identification correspondence markings are being idle state by server, after the registration for completing mobile terminal, Can Scheduling content grab task to the mobile terminal.

Fig. 7 shows the flow diagram that mobile terminal is registered in one embodiment.With reference to Fig. 7, mobile terminal is by logical After being physically connected to terminal with universal serial bus, terminal can monitor newly-increased mobile terminal and be connected to local, then pass through equipment Acquisition of information instruction obtains mobile terminal device information, then the mobile terminal device information that will acquire reports to server, services Device then saves the mobile terminal device information, and is idle state by the mobile terminal identification correspondence markings of newly-increased mobile terminal, To complete the registration of the mobile terminal.

In above-described embodiment, after terminal scanning to newly-increased mobile terminal, that is, the mobile terminal of the mobile terminal is obtained Facility information simultaneously reports to server and is registered, and server can be dispatched and appoint when carrying out task schedule after the completion of registration It is engaged in the newly-increased mobile terminal.

In one embodiment, the processing method of content crawl task further include: periodically send heartbeat packet to mobile terminal； When not receiving response bag of the mobile terminal corresponding to heartbeat packet feedback in the second preset duration, then determine at mobile terminal In off-line state；The mobile terminal identification of mobile terminal and off-line state correspondence are reported into server；The off-line state reported It is off-line state that server, which is used to indicate, by mobile terminal identification correspondence markings.

Wherein, heartbeat packet is the self-defining data packet of terminal detection and the connection status of mobile terminal.Second preset duration Being that terminal is pre-set receives the waiting time for corresponding to the response bag of heartbeat packet.Response bag is mobile terminal to terminal notification The self-defining data packet of connection status.

Specifically, terminal periodically can send heartbeat packet to mobile terminal, and start timing after sending heartbeat packet every time.When When receiving response bag of the mobile terminal corresponding to heartbeat packet feedback in the second preset duration, then terminal and mobile terminal are determined Between connection be kept, then continue to send heartbeat packet on schedule, to detect connection status.If not received in the second preset duration When corresponding to the response bag of heartbeat packet feedback to mobile terminal, then determine that the connection between terminal and mobile terminal disconnects, it is mobile Terminal is in off-line state, the mobile terminal identification of mobile terminal and off-line state correspondence is reported to server, server is just It is off-line state by mobile terminal identification correspondence markings, leads to content to the mobile terminal to avoid Scheduling content crawl task Crawl task can not be performed in time.

Fig. 8 shows the schematic diagram that mobile terminal state changes in one embodiment.With reference to Fig. 8, pass through in mobile terminal When universal serial bus is physically connected to terminal, the state of the mobile terminal is off-line state by not-connected status variation.At this point, The mobile terminal device information of mobile terminal can be reported to server by terminal, complete the registration of mobile terminal, mobile whole in this way The state at end is then converted into idle state by off-line state, when mobile terminal is in idle condition, as waits scheduler task. For the schedulable task of server to the mobile terminal being in idle condition, it is busy shape that the state of the mobile terminal, which changes, in this way State.After mobile terminal completion task, it is idle state that the state of the mobile terminal, which changes,.Wherein, when mobile terminal is in Idle state or busy condition and time-out be corresponding terminal heartbeat packet when, then the state of the mobile terminal i.e. change for from Linear state.Mobile terminal in off-line state, which needs to register again, can just receive an assignment.

In the present embodiment, it by the connection status of heartbeat mechanism real-time detection terminal and mobile terminal, timely updates end Current state is held, to guarantee that task schedule is gone on smoothly with what is handled, avoids scheduler task to offline or busy shifting It moves terminal and leads to not the situation being performed in time.

Fig. 9 show in one embodiment task schedule with interacted between server, terminal and mobile terminal in treatment process Timing diagram.With reference to Fig. 9, firstly, server is not that empty or new content grabs task to empty task team in task queue When in column, inquiry corresponds to the mobile terminal identification of idle state.Content crawl task is successively taken out from the head of the queue of task queue, And it is dispatched to the mobile terminal identification inquired respectively, until each mobile terminal identification inquired is scheduled respectively in one The content crawl task held in crawl task or task queue has been dispatched.Server, will be each interior after completing task schedule Hold the mobile terminal identification of crawl task schedule extremely, is updated to correspond to busy condition by corresponding to idle state.

Further, multiple terminals grab task from server pull content.Corresponding each terminal, terminal judgement are drawn There is the content crawl task for being dispatched to locally-attached mobile terminal execution in the content crawl task taken right.If so, then will The corresponding task configuration information generation control instruction of task is grabbed according to the content and is issued to the mobile terminal being dispatched to, and instruction moves Dynamic terminal grabs application content according to control instruction and completes content crawl task.If so, then waiting next time from server pull Content grabs task.

Further, mobile terminal feeds back task action result after having executed content crawl task, and to mobile terminal. Task action result is reported to server again by terminal, and server is when receiving task action result, by task action result The mobile terminal identification of corresponding content crawl task schedule extremely, is updated to correspond to idle state by corresponding to busy condition.

Further, server is corresponding by the task action result when task action result indicates task execution failure Content crawl task rejoin the tail of the queue of task queue, dispatched again with waiting.Moreover, server pulls task in terminal Afterwards, and after issuing corresponding terminal profile, if do not receive the task action result that terminal reports, by with issue The corresponding content crawl task of task configuration information rejoins the tail of the queue of task queue, is dispatched again with waiting.

It should be understood that although each step in the flow chart of the various embodiments described above is successively shown according to the instruction of arrow Show, but these steps are not that the inevitable sequence according to arrow instruction successively executes.Unless expressly state otherwise herein, this There is no stringent sequences to limit for the execution of a little steps, these steps can execute in other order.Moreover, above-mentioned each implementation At least part step in example may include that perhaps these sub-steps of multiple stages or stage be not necessarily for multiple sub-steps It is to execute completion in synchronization, but can execute at different times, the execution sequence in these sub-steps or stage It is not necessarily and successively carries out, but can be at least part wheel of the sub-step or stage of other steps or other steps Stream alternately executes.

As shown in Figure 10, in one embodiment, a kind of dispatching device 1000 of content crawl task is provided.Referring to figure 10, the dispatching device 1000 of content crawl task includes: to obtain module 1001, generation module 1002, configuration module 1003, adjust It spends module 1004 and issues module 1005.

Obtain module 1001, the application content classification configured for obtaining target application.

Generation module 1002, for generating the corresponding content crawl task of each application content classification respectively.

Configuration module 1003 configures corresponding task configuration information for grabbing task for each content.

Scheduler module 1004, for each content crawl task to be dispatched to the mobile terminal mark corresponding to idle state respectively Know.

Module 1005 is issued, for issuing each content crawl task and corresponding task configuration information；The content issued is grabbed Task is taken, the corresponding mobile terminal of the mobile terminal identification being dispatched to is used to indicate and is answered according to the crawl of corresponding task configuration information With content, application content belongs to application content classification corresponding with the content crawl task issued.

In one embodiment, it is also used to obtain in application, obtaining module 1001 when target application belongs to open access style Take the public identity of access target application.Generation module 1002 is also used to generate the corresponding content crawl of each public identity respectively and appoints Business.Wherein, application content, belong to the corresponding application content classification of content crawl task that issues, alternatively, with the content that issues The corresponding public identity of crawl task is corresponding.

In one embodiment, each content that generation module 1002 is also used to generate grabs task, by corresponding task The generation time sequentially adds task queue.Scheduler module 1004 is also used to inquire the mobile terminal mark for currently corresponding to idle state Know；Content crawl task is successively taken out from the head of the queue of task queue；The content successively taken out crawl task is dispatched to respectively and is looked into The mobile terminal identification ask.

In one embodiment, issue module 1005 be also used to receive terminal initiation task pull request；According to task Request is pulled, each content is grabbed into mission dispatching to terminal；The content crawl task schedule that is connected to and issues when terminal is extremely When the corresponding mobile terminal of mobile terminal identification, the content issued the crawl corresponding task configuration information of task is issued to end End；Wherein, the task configuration information issued is used to indicate terminal after generating control instruction according to task configuration information, passes through Control instruction controls mobile terminal and grabs application content according to control instruction.

As shown in figure 11, in one embodiment, the dispatching device 1000 of content crawl task further include: processing module 1006, the task action result reported for receiving terminal；Task action result corresponds to and the task configuration information phase that issues The content crawl task answered；It, will be corresponding to the task configuration information issued when task action result indicates task execution failure Content crawl task rejoin the tail of the queue of task queue.

In one embodiment, processing module 1006 is also used to the timing since when issuing task configuration information；Work as timing When duration reaches the first preset duration and do not receive the task action result that terminal reports, then it will be configured with issuing for task The corresponding content crawl task of information rejoins the tail of the queue of task queue.

In one embodiment, scheduler module 1004 is also used to grabbing each content into the mobile terminal mark of task schedule extremely Know, is updated to correspond to busy condition by corresponding to idle state.The dispatching device 1000 of content crawl task further include: processing Module 1006 is used to receive the task action result that the content crawl task for corresponding to and issuing reports；Receiving task execution knot When fruit, by the mobile terminal identification of the corresponding content crawl task schedule of task action result extremely, by corresponding to busy condition more New is corresponding to idle state.

As shown in figure 12, in one embodiment, a kind of processing unit 1200 of content crawl task is provided.Referring to figure 12, the processing unit 1200 of content crawl task includes: to obtain module 1201, determining module 1202,1203 and of transmission module Receiving module 1204.

Module 1201 is obtained, for obtaining the content crawl task for corresponding to application content classification and generating.

Determining module 1202, the mobile terminal identification being respectively dispatched to for determining each content crawl task.

Module 1201 is obtained to be also used to then obtain scheduling in the corresponding mobile terminal of mobile terminal identification and locality connection Content to mobile terminal identification grabs the corresponding task configuration information of task.

Transmission module 1203, for transmitting the control instruction generated according to task configuration information to mobile terminal.

Receiving module 1204, the application content for grabbing and feeding back according to control instruction for receiving mobile terminal；Using interior Hold and belongs to application content classification corresponding with the content crawl task of mobile terminal identification is dispatched to.

In one embodiment, acquisition module 1201 is also used to generate task and pulls request；Task is pulled request to send To server；It receives server and the content crawl task that request issues is pulled according to task；Content grabs task and corresponds to application Content type generates, and content crawl task is dispatched to the mobile terminal identification corresponding to idle state by server respectively.

In one embodiment, the processing unit 1200 of content crawl task further include: registration module 1205, for scanning Local mobile terminal is connected to by the way that universal serial bus is newly-increased；When scanning is connected to this to by the way that universal serial bus is newly-increased When the mobile terminal on ground, then mobile terminal device information corresponding to the mobile terminal of newly-increased connection is obtained；By mobile terminal Mobile terminal identification and mobile terminal device information correspondence report to server；The mobile terminal device information reported is used to indicate Mobile terminal identification correspondence markings are idle state by server.

As shown in figure 13, in one embodiment, the processing unit 1200 of content crawl task further include: registration module 1205 and reporting module 1206.

Reporting module 1206, for periodically sending heartbeat packet to mobile terminal；When not received in the second preset duration When mobile terminal corresponds to the response bag of heartbeat packet feedback, then determine that mobile terminal is in off-line state；By the shifting of mobile terminal Dynamic terminal iidentification and off-line state correspondence report to server；The off-line state reported is used to indicate server for mobile terminal mark Knowledge correspondence markings are off-line state.

Figure 14 shows the internal structure chart of computer equipment in one embodiment.The computer equipment specifically can be figure Server 110 or terminal 120 in 1.As shown in figure 14, which includes the processing connected by system bus Device, memory and network interface.Wherein, memory includes non-volatile memory medium and built-in storage.The computer equipment Non-volatile memory medium is stored with operating system, can also be stored with computer program, which is executed by processor When, it may make the dispatching method of processor content crawl task or the processing method of content crawl task.In the built-in storage Computer program can also be stored, when which is executed by processor, processor may make to execute content crawl task Dispatching method or content crawl task processing method.It will be understood by those skilled in the art that structure shown in Figure 14, The only block diagram of part-structure relevant to application scheme, does not constitute the calculating being applied thereon to application scheme The restriction of machine equipment, specific computer equipment may include more certain than more or fewer components as shown in the figure, or combination Component, or with different component layouts.

In one embodiment, when computer equipment is terminal 120, which can also include passing through system The Universal Serial Bus Interface of bus connection.The Universal Serial Bus Interface is used to establish physical connection with mobile terminal.

In one embodiment, the processing of the dispatching device of content crawl task provided by the present application and content crawl task Device can be implemented as a kind of form of computer program, and computer program can be transported in computer equipment as shown in figure 14 Row, the non-volatile memory medium of computer equipment can store the dispatching device of component content crawl task or content crawl is appointed Each program module of the processing unit of business, for example, acquisition module 1001 shown in Fig. 10, generation module 1002, configuration module 1003, scheduler module 1004 and issue module 1005 etc. or Figure 12 shown in acquisition module 1201, determining module 1202, pass Defeated module 1203 and receiving module 1204 etc..The computer program of each program module composition makes processor execute this specification Described in each embodiment of the application content crawl task dispatching method or content crawl task processing method in The step of.

For example, computer equipment shown in Figure 14 can grab the dispatching device of task by content as shown in Figure 10 Acquisition module 1001 in 1000 obtains the application content classification that target application is configured.It is generated respectively by generation module 1002 The corresponding content of each application content classification grabs task.It is that each content grabs task configuration corresponding by configuration module 1003 Business configuration information.Each content crawl task is dispatched to the mobile terminal corresponding to idle state respectively by scheduler module 1004 Mark.Each content crawl task and corresponding task configuration information are issued by issuing module 1005；The content crawl issued is appointed Business is used to indicate the corresponding mobile terminal of the mobile terminal identification being dispatched to according in the crawl application of corresponding task configuration information Hold, application content belongs to application content classification corresponding with the content crawl task issued.

For another example computer equipment shown in Figure 14 can grab the processing unit of task by content as shown in figure 12 Acquisition module 1201 in 1200, which obtains, corresponds to the content crawl task that application content classification generates.Pass through determining module 1202 Determine the mobile terminal identification that each content crawl task is respectively dispatched to.Again by obtaining module 1201 in mobile terminal identification phase When the mobile terminal and locality connection answered, the content crawl corresponding task of task for being dispatched to mobile terminal identification is obtained with confidence Breath.The control instruction generated according to task configuration information is transmitted to mobile terminal by transmission module 1203.Pass through receiving module The application content that 1204 reception mobile terminals are grabbed and fed back according to control instruction；Application content belongs to and is dispatched to mobile terminal The content of mark grabs the corresponding application content classification of task.

In one embodiment, a kind of computer readable storage medium is provided, is deposited on the computer readable storage medium Computer program is contained, when which is executed by processor, so that processor executes following steps: obtaining target application The application content classification configured；The corresponding content crawl task of each application content classification is generated respectively；Appoint for the crawl of each content Business configures corresponding task configuration information；Each content crawl task is dispatched to the mobile terminal mark corresponding to idle state respectively Know；Issue each content crawl task and corresponding task configuration information；The content crawl task issued, is used to indicate and is dispatched to The corresponding mobile terminal of mobile terminal identification grabs application content according to corresponding task configuration information, and application content belongs to under The corresponding application content classification of content crawl task of hair.

In one embodiment, when target application belongs to open access style in application, computer program also to handle Device executes following steps: obtaining the public identity of access target application；The corresponding content crawl of each public identity is generated respectively to appoint Business；Wherein, application content, belong to the corresponding application content classification of content crawl task that issues, alternatively, with the content that issues The corresponding public identity of crawl task is corresponding.

In one embodiment, computer program also makes processor execute following steps: each content of generation is grabbed Task sequentially adds task queue by the corresponding task generation time.Each content crawl task is dispatched to respectively and corresponds to sky The mobile terminal identification of not busy state, comprising: inquiry currently corresponds to the mobile terminal identification of idle state；From the team of task queue First content of successively taking out grabs task；The content successively taken out crawl task is dispatched to the mobile terminal mark inquired respectively Know.

In one embodiment, computer program also makes processor execute following steps: receiving the task that terminal reports Implementing result；Task action result corresponds to content corresponding with the task configuration information issued and grabs task；Work as task execution When as a result indicating task execution failure, content corresponding with task action result crawl task is rejoined to the team of task queue Tail.

In one embodiment, computer program also makes processor execute following steps: from issuing task configuration information When start timing；When timing duration reaches the first preset duration and does not receive task action result that terminal reports, then will Content crawl task corresponding with the task configuration information issued rejoins the tail of the queue of task queue.

In one embodiment, computer program also makes processor execute following steps: each content is grabbed task tune The mobile terminal identification of degree extremely is updated to correspond to busy condition by corresponding to idle state；Receive the content for corresponding to and issuing The task action result that crawl task reports；When receiving task action result, the corresponding content of task action result is grabbed The mobile terminal identification of task schedule extremely is taken, is updated to correspond to idle state by corresponding to busy condition.

In one embodiment, a kind of computer readable storage medium is provided, is deposited on the computer readable storage medium Computer program is contained, when which is executed by processor, so that processor executes following steps: acquisition is answered Task is grabbed with the content that content type generates；Determine the mobile terminal identification that each content crawl task is respectively dispatched to；Work as shifting When the dynamic corresponding mobile terminal of terminal iidentification and locality connection, then the content crawl task phase for being dispatched to mobile terminal identification is obtained The task configuration information answered；The control instruction generated according to task configuration information is transmitted to mobile terminal；Receive mobile terminal root The application content for grabbing and feeding back according to control instruction；Application content belongs to and is dispatched to the content crawl task of mobile terminal identification Corresponding application content classification.

In one embodiment, computer program also makes processor execute following steps: scanning is total by general serial Line is newly-increased to be connected to local mobile terminal；When scanning is connected to local mobile terminal to by the way that universal serial bus is newly-increased When, then obtain mobile terminal device information corresponding to the mobile terminal of newly-increased connection；By the mobile terminal identification of mobile terminal It is corresponding with mobile terminal device information to report to server；The mobile terminal device information reported is used to indicate server and will move Terminal iidentification correspondence markings are idle state.

In one embodiment, computer program also makes processor execute following steps: periodically sending to mobile terminal Heartbeat packet；When not receiving response bag of the mobile terminal corresponding to heartbeat packet feedback in the second preset duration, then determine to move Dynamic terminal is in off-line state；The mobile terminal identification of mobile terminal and off-line state correspondence are reported into server；It reports It is off-line state that off-line state, which is used to indicate server for mobile terminal identification correspondence markings,.

In one embodiment, a kind of computer equipment, including memory and processor are provided, is stored in memory Computer program, when computer program is executed by processor, so that processor executes following steps: obtaining target application and configured Application content classification；The corresponding content crawl task of each application content classification is generated respectively；Task configuration is grabbed for each content Corresponding task configuration information；Each content crawl task is dispatched to the mobile terminal identification corresponding to idle state respectively；Under Send out content each crawl task and corresponding task configuration information；The content crawl task issued, is used to indicate the movement being dispatched to Terminal iidentification corresponding mobile terminal grabs application content according to corresponding task configuration information, and application content belongs to and issues Content grabs the corresponding application content classification of task.

In one embodiment, a kind of computer equipment, including memory and processor are provided, is stored in memory Computer program, when computer program is executed by processor, so that processor executes following steps: obtaining and correspond to application content The content that classification generates grabs task；Determine the mobile terminal identification that each content crawl task is respectively dispatched to；Work as mobile terminal When identifying corresponding mobile terminal and locality connection, then the content crawl task that acquisition is dispatched to mobile terminal identification is appointed accordingly Business configuration information；The control instruction generated according to task configuration information is transmitted to mobile terminal；Mobile terminal is received according to control The application content that instruction is grabbed and fed back；Application content belongs to corresponding with the content crawl task of mobile terminal identification is dispatched to Application content classification.

Those of ordinary skill in the art will appreciate that realizing all or part of the process in above-described embodiment method, being can be with Relevant hardware is instructed to complete by computer program, the program can be stored in a non-volatile computer and can be read In storage medium, the program is when being executed, it may include such as the process of the embodiment of above-mentioned each method.Wherein, provided herein Each embodiment used in any reference to memory, storage, database or other media, may each comprise non-volatile And/or volatile memory.Nonvolatile memory may include that read-only memory (ROM), programming ROM (PROM), electricity can be compiled Journey ROM (EPROM), electrically erasable ROM (EEPROM) or flash memory.Volatile memory may include random access memory (RAM) or external cache.By way of illustration and not limitation, RAM is available in many forms, such as static state RAM (SRAM), dynamic ram (DRAM), synchronous dram (SDRAM), double data rate sdram (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronization link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) directly RAM (RDRAM), straight Connect memory bus dynamic ram (DRDRAM) and memory bus dynamic ram (RDRAM) etc..

Each technical characteristic of above embodiments can be combined arbitrarily, for simplicity of description, not to above-described embodiment In each technical characteristic it is all possible combination be all described, as long as however, the combination of these technical characteristics be not present lance Shield all should be considered as described in this specification.

Only several embodiments of the present invention are expressed for above embodiments, and the description thereof is more specific and detailed, but can not Therefore limitations on the scope of the patent of the present invention are interpreted as.It should be pointed out that for those of ordinary skill in the art, Without departing from the inventive concept of the premise, various modifications and improvements can be made, these belong to protection model of the invention It encloses.Therefore, the scope of protection of the patent of the invention shall be subject to the appended claims.

Claims

1. a kind of dispatching method of content crawl task, comprising:

Issue each content crawl task and corresponding task configuration information；The content crawl task issued, is used to indicate tune The degree corresponding mobile terminal of mobile terminal identification extremely grabs application content according to corresponding task configuration information, in the application Hold and belongs to application content classification corresponding with the content crawl task issued.

2. the method according to claim 1, wherein when the target application belongs to open access style application When, the method also includes:

Obtain the public identity for accessing the target application；

The corresponding content crawl task of each public identity is generated respectively；

Wherein, the application content belongs to application content classification corresponding with the content crawl task issued, alternatively, with issuing The corresponding public identity of content crawl task it is corresponding.

3. the method according to claim 1, wherein the method also includes:

Each content of generation is grabbed into task, sequentially adds task queue by the corresponding task generation time；

It is described that each content crawl task is dispatched to the mobile terminal identification corresponding to idle state respectively, comprising:

Inquiry currently corresponds to the mobile terminal identification of idle state；

Content crawl task is successively taken out from the head of the queue of the task queue；

The content successively taken out crawl task is dispatched to mobile terminal identifications inquire and different respectively.

4. according to the method described in claim 3, it is characterized in that, described issue each content crawl task and corresponding Business configuration information, comprising:

It receives the task that terminal is initiated and pulls request；

Request is pulled according to the task, each content is grabbed into mission dispatching to the terminal；

When the terminal is connected to mobile terminal corresponding with the content crawl task schedule mobile terminal identification extremely issued, The content issued the crawl corresponding task configuration information of task is issued to the terminal；

Wherein, the task configuration information issued is used to indicate the terminal after generating control instruction according to task configuration information, Mobile terminal is controlled by the control instruction, and application content is grabbed according to the control instruction.

5. according to the method described in claim 4, it is characterized in that, the method also includes:

Receive the task action result that the terminal reports；The task action result corresponds to and the task configuration information that issues Corresponding content grabs task；

When the task action result indicates task execution failure, content crawl corresponding with the task action result is appointed Business rejoins the tail of the queue of the task queue.

6. according to the method described in claim 4, it is characterized in that, the method also includes:

The timing since when issuing task configuration information；

It, then will be with when timing duration reaches the first preset duration and does not receive task action result that the terminal reports The corresponding content crawl task of the task configuration information issued rejoins the tail of the queue of task queue.

7. method according to any of claims 1-4, which is characterized in that the method also includes:

By the mobile terminal identification of each content crawl task schedule extremely, it is updated to by corresponding to idle state corresponding to busy State；

Receive the task action result that the content crawl task for corresponding to and issuing reports；

When receiving task action result, by the movement of content corresponding with task action result crawl task schedule extremely Terminal iidentification is updated to correspond to idle state by corresponding to busy condition.

8. a kind of processing method of content crawl task, comprising:

When the corresponding mobile terminal of the mobile terminal identification and locality connection, then obtains and be dispatched to the mobile terminal identification Content grab the corresponding task configuration information of task；

Receive the application content that the mobile terminal is grabbed and fed back according to the control instruction；The application content belongs to and adjusts It spends to the content of the mobile terminal identification and grabs the corresponding application content classification of task.

9. according to the method described in claim 8, it is characterized in that, described obtain corresponds to the content that application content classification generates Crawl task, comprising:

Generation task pulls request；

The task is pulled into request and is sent to server；

It receives the server and the content crawl task that request issues is pulled according to the task；The content crawl task is corresponding It is generated in application content classification, and content crawl task is dispatched to the shifting corresponding to idle state by the server respectively Dynamic terminal iidentification.

10. according to the method described in claim 9, it is characterized in that, the method also includes:

Scanning is connected to local mobile terminal by the way that universal serial bus is newly-increased；

When scanning is to when being connected to local mobile terminal by the way that universal serial bus is newly-increased, then acquisition increases the shifting of connection newly Mobile terminal device information corresponding to dynamic terminal；

The mobile terminal identification of the mobile terminal and mobile terminal device information correspondence are reported into server；It reports Mobile terminal device information, being used to indicate the server for the mobile terminal identification correspondence markings is idle state.

11. according to the method described in claim 9, it is characterized in that, the method also includes:

Periodically heartbeat packet is sent to the mobile terminal；

When not receiving response bag of the mobile terminal corresponding to heartbeat packet feedback in the second preset duration, then sentence The fixed mobile terminal is in off-line state；

The mobile terminal identification of the mobile terminal and off-line state correspondence are reported into server；The off-line state reported is used for Indicate the server by the mobile terminal identification correspondence markings be off-line state.

12. a kind of dispatching device of content crawl task, comprising:

Scheduler module, for each content crawl task to be dispatched to the mobile terminal identification corresponding to idle state respectively；

Module is issued, for issuing each content crawl task and corresponding task configuration information；The content crawl issued is appointed Business is used to indicate the corresponding mobile terminal of the mobile terminal identification being dispatched to according in the crawl application of corresponding task configuration information Hold, the application content belongs to application content classification corresponding with the content crawl task issued.

13. a kind of processing unit of content crawl task, comprising:

The acquisition module is also used to, and when the corresponding mobile terminal of the mobile terminal identification and locality connection, then obtains tune It spends to the content of the mobile terminal identification and grabs the corresponding task configuration information of task；

Receiving module, the application content for grabbing and feeding back according to the control instruction for receiving the mobile terminal；It is described to answer Belong to application content classification corresponding with the content crawl task of the mobile terminal identification is dispatched to content.

14. a kind of computer readable storage medium, computer program, the meter are stored on the computer readable storage medium When calculation machine program is executed by processor, so that the processor executes the method as described in any one of claims 1 to 11 Step.

15. a kind of computer equipment, including memory and processor, computer program, the meter are stored in the memory When calculation machine program is executed by the processor, so that the processor executes the side as described in any one of claims 1 to 11 The step of method.