CN111064997B - Resource pre-distribution method and device - Google Patents
Resource pre-distribution method and device Download PDFInfo
- Publication number
- CN111064997B CN111064997B CN201811206374.3A CN201811206374A CN111064997B CN 111064997 B CN111064997 B CN 111064997B CN 201811206374 A CN201811206374 A CN 201811206374A CN 111064997 B CN111064997 B CN 111064997B
- Authority
- CN
- China
- Prior art keywords
- target files
- time point
- copies
- target
- tasks
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 21
- 230000000737 periodic effect Effects 0.000 claims 6
- 230000009286 beneficial effect Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 2
- 238000012163 sequencing technique Methods 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000006855 networking Effects 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/632—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing using a connection between clients on a wide area network, e.g. setting up a peer-to-peer communication via Internet for retrieving video segments from the hard-disk of other client devices
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/25—Management operations performed by the server for facilitating the content distribution or administrating data related to end-users or client devices, e.g. end-user or client device authentication, learning user preferences for recommending movies
- H04N21/266—Channel or content management, e.g. generation and management of keys and entitlement messages in a conditional access system, merging a VOD unicast channel into a multicast channel
- H04N21/2662—Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/647—Control signaling between network components and server or clients; Network processes for video distribution between server and clients, e.g. controlling the quality of the video stream, by dropping packets, protecting content from unauthorised alteration within the network, monitoring of network load, bridging between two different networks, e.g. between IP and wireless
- H04N21/64723—Monitoring of network processes or resources, e.g. monitoring of network load
- H04N21/64738—Monitoring network characteristics, e.g. bandwidth, congestion level
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Computer Networks & Wireless Communication (AREA)
- Computer Security & Cryptography (AREA)
- Information Transfer Between Computers (AREA)
Abstract
本发明公开了一种资源预分发方法及装置，该方法包括：分别计算多个目标文件的预测带宽量；依据每个所述目标文件的码率及其预测带宽量计算每个所述目标文件的计划下发任务份数；依据当前所有可用的服务设备已经持有的每个所述目标文件的份数修正所述计划下发任务份数以获取每个所述目标文件的实际下发任务份数；依据所述实际下发任务份数分发至可用的服务设备。实施本发明的有益效果：精确计算不同文件的需求量，充分考虑了节点服务能力，做到合理利用资源；在真正需要之前做到预分发，提升了文件命中率。
The invention discloses a resource pre-distribution method and device. The method includes: calculating the predicted bandwidth of a plurality of target files respectively; According to the number of copies of each of the target files already held by all currently available service equipment, amend the number of copies of the planned tasks to obtain the actual task of each of the target files. The number of copies; distributed to the available service equipment according to the actual number of task assignments. The beneficial effects of implementing the present invention: accurately calculate the demand of different files, fully consider the node service capability, and make reasonable use of resources; pre-distribute before the real need, and improve the file hit rate.
Description
Technical Field
The invention relates to the technical field of network content distribution, in particular to a resource pre-distribution method and device.
Background
The pre-distribution technology is commonly used in CDN (content delivery network) and P2P (peer-to-peer network) products, and is usually to send files to nodes in an HTTP (Hyper Text Transfer Protocol) manner or a TCP (Transmission Control Protocol) private Protocol. The prior art mainly has the following problems:
1. the demand of different files is not accurately calculated, which causes the situation that the bandwidth resource is wasted due to excessive issued quantity or the situation that the file issued quantity is insufficient for short supply and short demand.
2. Failure to pre-distribute before it is really needed misses the opportunity for file hits.
Disclosure of Invention
The invention aims to provide a resource pre-distribution method and a device, which solve the problems that bandwidth resources are wasted due to excessive issued copies or insufficient supply and demand are caused by inaccurate calculation of the demand of different files in the prior art; and the problem that the file hit opportunity is missed because the pre-distribution cannot be carried out before the actual need.
The technical scheme of the invention is realized as follows:
in one aspect, the present invention provides a method for pre-distributing resources, including:
respectively calculating the predicted bandwidth amounts of a plurality of target files;
calculating the number of tasks planned to be issued of each target file according to the code rate and the predicted bandwidth amount of each target file;
correcting the planned task issuing number according to the number of the target files held by all the available service equipment at present to acquire the actual task issuing number of each target file;
and distributing to available service equipment according to the actually issued task number.
In the method for pre-distributing resources according to the present invention, the calculating the predicted bandwidth amounts of the plurality of target files respectively includes:
acquiring a first time point according to the current moment, and acquiring a second time point after a first period interval of the first time point;
acquiring a third time point before a second period interval of the second time point, and acquiring the total bandwidth amount of the third time point;
acquiring a fourth time point before a first cycle interval of the first time point, determining a time period from the fourth time point to the first time point as a measured time period, and acquiring an access frequency ranking table of a plurality of target files in the measured time period; the access frequency ranking table comprises a plurality of access frequencies of the target files;
dividing the number of accesses of each of the target files by the total number of accesses of the plurality of target files to obtain a weight coefficient of each of the target files;
multiplying the weight coefficient of each of the target files by the total amount of bandwidth at the third point in time to obtain a predicted amount of bandwidth at the second point in time for each of the target files. In the method for pre-distributing resources of the present invention, the calculating the number of tasks to be scheduled to be issued for each target file according to the code rate and the predicted bandwidth amount of each target file includes:
acquiring the code rate of each target file from the access times ranking table; the access times ranking table also comprises the code rate of each target file;
and dividing the predicted bandwidth amount of each target file by the corresponding code rate to obtain the planned issuing task number of each target file.
In the method for pre-distributing resources of the present invention, the correcting the number of task copies scheduled to be issued according to the number of copies of each target file already held by all currently available service devices to obtain the actual number of task copies issued for each target file includes:
acquiring the number of copies of each target file which are already held by all currently available service equipment;
and subtracting the corresponding number of copies from the planned issued task number of each target file to obtain the actual issued task number of copies.
In the method for pre-distributing resources according to the present invention, the distributing to available service devices according to the number of actually issued tasks includes:
sequencing a plurality of target files according to the access times of the target files;
and selecting available service equipment according to the sorting result to distribute the target files respectively.
In another aspect, there is provided a resource pre-distribution apparatus having stored therein a plurality of instructions adapted to be loaded and executed by a processor, comprising:
respectively calculating the predicted bandwidth amounts of a plurality of target files;
calculating the number of tasks planned to be issued of each target file according to the code rate and the predicted bandwidth amount of each target file;
correcting the planned task issuing number according to the number of the target files held by all the available service equipment at present to acquire the actual task issuing number of each target file;
and distributing to available service equipment according to the actually issued task number.
In the resource pre-allocation apparatus according to the present invention, the calculating the predicted bandwidth amounts of the plurality of target files, respectively, includes:
acquiring a first time point according to the current moment, and acquiring a second time point after a first period interval of the first time point;
acquiring a third time point before a second period interval of the second time point, and acquiring the total bandwidth amount of the third time point;
acquiring a fourth time point before a first cycle interval of the first time point, determining a time period from the fourth time point to the first time point as a measured time period, and acquiring an access frequency ranking table of a plurality of target files in the measured time period; the access frequency ranking table comprises a plurality of access frequencies of the target files;
dividing the number of accesses of each of the target files by the total number of accesses of the plurality of target files to obtain a weight coefficient of each of the target files;
multiplying the weight coefficient of each of the target files by the total amount of bandwidth at the third point in time to obtain a predicted amount of bandwidth at the second point in time for each of the target files. In the resource pre-distribution device of the present invention, the calculating the number of tasks scheduled to be issued for each target file according to the code rate and the predicted bandwidth amount of each target file includes:
acquiring the code rate of each target file from the access times ranking table; the access times ranking table also comprises the code rate of each target file;
and dividing the predicted bandwidth amount of each target file by the corresponding code rate to obtain the planned issuing task number of each target file.
In the resource pre-distribution apparatus of the present invention, the correcting the number of task copies scheduled to be issued according to the number of copies of each target file already held by all currently available service devices to obtain the actual number of task copies issued for each target file includes:
acquiring the number of copies of each target file which are already held by all currently available service equipment;
and subtracting the corresponding number of copies from the planned issued task number of each target file to obtain the actual issued task number of copies.
In the resource pre-distribution apparatus of the present invention, the distributing to the available service devices according to the number of actually issued tasks includes:
sequencing a plurality of target files according to the access times of the target files;
and selecting available service equipment according to the sorting result to distribute the target files respectively.
Therefore, the method has the advantages that the demand of different files is accurately calculated, the node capacity is fully considered, and resources are reasonably utilized; the file is pre-distributed before the real need, so that the hit rate of the file is improved.
Drawings
The invention will be further described with reference to the accompanying drawings and examples, in which:
fig. 1 is a flowchart of a resource pre-distribution method according to an embodiment of the present invention;
fig. 2 is a schematic diagram of time point distribution according to an embodiment of the present invention.
Detailed Description
In order to more clearly understand the technical features, objects, and effects of the present invention, embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It is to be understood that the following description is only a specific illustration of the embodiments of the present invention and should not be taken as limiting the scope of the invention.
Referring to fig. 1, fig. 1 is a flowchart of a resource pre-distribution method according to an embodiment of the present invention, where the resource pre-distribution method includes steps S1-S4:
s1, respectively calculating the predicted bandwidth amounts of the target files; step S1 includes steps S11-S15:
s11, acquiring a first time point according to the current moment, and acquiring a second time point after a first period interval of the first time point; referring to fig. 2, fig. 2 is a schematic diagram of time point distribution provided by an embodiment of the present invention, for example: the pre-distribution system calculates the demand of the target file at the next interval time point through a preset algorithm A, wherein the interval during the first week is a preset threshold value, such as 30 minutes. The current time is 12:10, the first time point is 12:00 of the current date, the first week interval is 30 minutes, and the second time point is 12:30 of the current date.
S12, acquiring a third time point before the second period interval of the second time point, and acquiring the total bandwidth amount of the third time point; for example: the required amount of the next interval time point of the target file is calculated by the preset algorithm a of step S11 (hereinafter, this time point is referred to as time point C). And then, acquiring the total bandwidth amount of the same time point C of yesterday through big data statistics. I.e. the second week interval is one day. Referring to fig. 2, time point C (12:30) of the current date is the second time point, yesterday the same time point C (12:30) is the third time point, and the second week interval is 1 day.
S13, acquiring a fourth time point before the first cycle interval of the first time point, determining a time period from the fourth time point to the first time point as a measured time period, and acquiring an access frequency ranking table of a plurality of target files in the measured time period; the access frequency ranking table comprises a plurality of access frequencies of the target files; for example: acquiring an access time ranking table of a target file in an interval time period (hereinafter referred to as time period D) at a time point C, wherein the access time ranking table of the target file comprises { file identification, access times and file code rate }; referring to FIG. 2, the fourth time point is 11:30 of the current date; from 11:30 to 12:00 of the current date is time period D, i.e., the measured time period.
S14, dividing the access times of each target file by the total access times of the plurality of target files to obtain a weight coefficient of each target file; for example: and dividing the access times of each target file in the target file access times sorting table of the time period D by the sum of the access times of all files to be used as a weight coefficient.
And S15, multiplying the weight coefficient of each target file by the total bandwidth amount at the third time point to obtain the predicted bandwidth amount of each target file at the second time point. For example: the total bandwidth amount of the same time point C (third time point) of yesterday is multiplied by the weight coefficient of each target file, and the predicted bandwidth amount of each target file at the time point C (second time point) is calculated. The method comprises the steps of calculating the accurate number of the target to be issued according to a preset algorithm B, and issuing the accurate number to available service equipment, wherein the service equipment provides an uploading bandwidth service for the networking hardware equipment implanted with preset software.
S2, calculating the number of tasks planned to be issued of each target file according to the code rate and the predicted bandwidth of each target file; step S2 includes steps S21-S22:
s21, obtaining the code rate of each target file from the access times sorting table; the access times ranking table also comprises the code rate of each target file; for example: and obtaining the code rate of each target file and the predicted bandwidth amount of the time point C from the target file access frequency sorting table of the time period D.
And S22, dividing the predicted bandwidth amount of each target file by the corresponding code rate to obtain the planned issuing task number of each target file. For example: and dividing the predicted bandwidth amount of each target file by the code rate of the target file to obtain the number of tasks (hereinafter referred to as task number E) required by each target file to be issued in a plan manner.
S3, correcting the planned task issuing number according to the number of the target files held by all the available service equipment at present to obtain the actual task issuing number of each target file; step S3 includes steps S31-S32:
and S31, acquiring the number of copies of each target file which are already held by all the currently available service equipment.
And S32, subtracting the corresponding number of copies from the number of the planned issued tasks of each target file to obtain the number of the actually issued tasks. For example: subtracting the number of the tasks E of each target file from the number of the tasks F of each target file, which is recorded by the scheduling server currently and is already held by all available service equipment, and calculating the result to be the number of the issued tasks F of each target file.
And S4, distributing the number of the actually issued task copies to available service equipment. Step S4 includes steps S41-S42:
s41, sorting the target files according to the access times of the target files; for example: and the scheduling server sorts the target files in the file access times sorting table according to the access times.
And S42, selecting available service equipment according to the sorting result to distribute the target files respectively. For example: and selecting available service equipment for distributing the files according to the target file copy number F from high to low one by one.
In another aspect, a resource pre-distribution apparatus is provided having stored therein a plurality of instructions adapted to be loaded and executed by a processor. Therefore, the resource pre-distribution device realizes the resource pre-distribution method.
In summary, although the present invention has been described with reference to the preferred embodiments, the above-described preferred embodiments are not intended to limit the present invention, and those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention, therefore, the scope of the present invention shall be determined by the appended claims.
Claims (8)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811206374.3A CN111064997B (en) | 2018-10-16 | 2018-10-16 | Resource pre-distribution method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811206374.3A CN111064997B (en) | 2018-10-16 | 2018-10-16 | Resource pre-distribution method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111064997A CN111064997A (en) | 2020-04-24 |
| CN111064997B true CN111064997B (en) | 2021-11-16 |
Family
ID=70296690
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811206374.3A Active CN111064997B (en) | 2018-10-16 | 2018-10-16 | Resource pre-distribution method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111064997B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113315836B (en) * | 2021-05-27 | 2023-03-14 | 北京达佳互联信息技术有限公司 | File access request scheduling method and device, electronic equipment and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101800885A (en) * | 2010-02-26 | 2010-08-11 | 北京新岸线网络技术有限公司 | Video data distribution method and system method and system for distributing video data |
| WO2011008984A1 (en) * | 2009-07-16 | 2011-01-20 | Netflix, Inc. | A digital content distribution system and method |
| KR101462557B1 (en) * | 2013-12-20 | 2014-11-27 | 연세대학교 산학협력단 | Method of reorganizing representation in DASH |
| CN104767796A (en) * | 2015-03-18 | 2015-07-08 | 广东工业大学 | A mobile P2P-based hotspot resource management method for Internet of Vehicles |
| CN105282203A (en) * | 2014-07-21 | 2016-01-27 | 腾讯科技（深圳）有限公司 | Method and device for establishing concentrated-directory P2P network |
| CN105791427A (en) * | 2016-04-22 | 2016-07-20 | 百度在线网络技术（北京）有限公司 | Method, device and system for supplementing resource heat |
| CN105872856A (en) * | 2016-03-21 | 2016-08-17 | 乐视云计算有限公司 | Method and system for distributing stream media files |
| CN106161284A (en) * | 2015-04-10 | 2016-11-23 | 深圳市云帆加速科技有限公司 | A kind of resource regulating method and system |
| CN108156257A (en) * | 2018-01-17 | 2018-06-12 | 北京奇艺世纪科技有限公司 | A kind of information-pushing method and device |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2169914B1 (en) * | 2008-09-19 | 2014-04-02 | Limelight Networks, Inc. | Content delivery network and related method |
| US8650301B2 (en) * | 2008-10-02 | 2014-02-11 | Ray-V Technologies, Ltd. | Adaptive data rate streaming in a peer-to-peer network delivering video content |
| US8626910B1 (en) * | 2012-06-19 | 2014-01-07 | Edgecast Networks, Inc. | Systems and methods for performing localized server-side monitoring in a content delivery network |
-
2018
- 2018-10-16 CN CN201811206374.3A patent/CN111064997B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011008984A1 (en) * | 2009-07-16 | 2011-01-20 | Netflix, Inc. | A digital content distribution system and method |
| CN101800885A (en) * | 2010-02-26 | 2010-08-11 | 北京新岸线网络技术有限公司 | Video data distribution method and system method and system for distributing video data |
| KR101462557B1 (en) * | 2013-12-20 | 2014-11-27 | 연세대학교 산학협력단 | Method of reorganizing representation in DASH |
| CN105282203A (en) * | 2014-07-21 | 2016-01-27 | 腾讯科技（深圳）有限公司 | Method and device for establishing concentrated-directory P2P network |
| CN104767796A (en) * | 2015-03-18 | 2015-07-08 | 广东工业大学 | A mobile P2P-based hotspot resource management method for Internet of Vehicles |
| CN106161284A (en) * | 2015-04-10 | 2016-11-23 | 深圳市云帆加速科技有限公司 | A kind of resource regulating method and system |
| CN105872856A (en) * | 2016-03-21 | 2016-08-17 | 乐视云计算有限公司 | Method and system for distributing stream media files |
| CN105791427A (en) * | 2016-04-22 | 2016-07-20 | 百度在线网络技术（北京）有限公司 | Method, device and system for supplementing resource heat |
| CN108156257A (en) * | 2018-01-17 | 2018-06-12 | 北京奇艺世纪科技有限公司 | A kind of information-pushing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111064997A (en) | 2020-04-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109246229A (en) | A kind of method and apparatus of distribution resource acquisition request | |
| US20110040876A1 (en) | Capacity planning for data center services | |
| US20180191819A1 (en) | Determining load state of remote systems using delay and packet loss rate | |
| Napoli et al. | Improving files availability for bittorrent using a diffusion model | |
| CN106161485B (en) | A kind of resource regulating method of infrastructure service cluster, device and system | |
| CN108769145A (en) | A kind of resources contribution amount method of proof, apparatus and system | |
| US20210400121A1 (en) | Generalized resource accountant | |
| CN111064997B (en) | Resource pre-distribution method and device | |
| CN110109800A (en) | A kind of management method and device of server cluster system | |
| Zhang et al. | Identifying, analyzing, and modeling flashcrowds in bittorrent | |
| Meulpolder et al. | Modeling and analysis of bandwidth-inhomogeneous swarms in BitTorrent | |
| CN111064776A (en) | Method, device and storage medium for generating blocks in blockchain | |
| JP6263443B2 (en) | Video quality estimation apparatus, method and program | |
| Van De Ven et al. | Distributed backup scheduling: Modeling and optimization | |
| CN112187839B (en) | Distribution method, server and storage medium | |
| CN103685369B (en) | For the method for multifile distribution, multifile cache control server and system | |
| CN113301445A (en) | Address allocation method and device for content distribution network | |
| CN110798691B (en) | Live content uploading and acquiring method and device | |
| KR20110035665A (en) | Ranking data system, ranking query system and ranking estimation method for estimating large capacity ranking in real time | |
| Braun et al. | Mobile peer-to-peer assisted coded streaming | |
| TW201308939A (en) | Device and method for distributed control of file downloading network | |
| US11277661B2 (en) | Quality information collection system, quality information collection method and program | |
| US10721180B2 (en) | Resource path monitoring | |
| Hwang et al. | Proof of violation for response time auditing in cloud systems | |
| CN115134351B (en) | Resource deployment method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP01 | Change in the name or title of a patent holder |
Address after: Room 201, Building A, No. 1 Qianwan Road, Qianhai Shenzhen-Hong Kong Cooperation Zone, Nanshan District, Shenzhen City, Guangdong Province Patentee after: Kingsoft Cloud (Shenzhen) Edge Computing Technology Co.,Ltd. Address before: Room 201, Building A, No. 1 Qianwan Road, Qianhai Shenzhen-Hong Kong Cooperation Zone, Nanshan District, Shenzhen City, Guangdong Province Patentee before: SHENZHEN YUNFAN ACCELERATION TECHNOLOGY Co.,Ltd. |
|
| CP01 | Change in the name or title of a patent holder |