[go: up one dir, main page]

CN104679594B - A kind of middleware distributed computing method - Google Patents

A kind of middleware distributed computing method Download PDF

Info

Publication number
CN104679594B
CN104679594B CN201510122457.4A CN201510122457A CN104679594B CN 104679594 B CN104679594 B CN 104679594B CN 201510122457 A CN201510122457 A CN 201510122457A CN 104679594 B CN104679594 B CN 104679594B
Authority
CN
China
Prior art keywords
server
cluster
resource
instance
servers
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related
Application number
CN201510122457.4A
Other languages
Chinese (zh)
Other versions
CN104679594A (en
Inventor
江国健
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
FUZHOU HUANYA ZHONGZHI COMPUTER CO., LTD.
Original Assignee
Fuzhou Huanya Zhongzhi Computer Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fuzhou Huanya Zhongzhi Computer Co Ltd filed Critical Fuzhou Huanya Zhongzhi Computer Co Ltd
Priority to CN201510122457.4A priority Critical patent/CN104679594B/en
Publication of CN104679594A publication Critical patent/CN104679594A/en
Application granted granted Critical
Publication of CN104679594B publication Critical patent/CN104679594B/en
Expired - Fee Related legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Landscapes

  • Computer And Data Communications (AREA)
  • Multi Processors (AREA)

Abstract

The invention provides a kind of middleware distributed computing method, this method includes:It is determined that the server in Idle state and in overload state, according to the resource and load distribution situation of the overall situation, is provided manager and is performed resource adjustment behavior by distributing to resource reclaim manager and resource;The load distribution server provided using each cluster, load is redistributed.The present invention be used for cluster distributed environment under charter resource-sharing, prevent server resource waste and overload, increase operation rate.

Description

A kind of middleware distributed computing method
Technical field
The present invention relates to Distributed Calculation, more particularly to a kind of middleware distributed computing method.
Background technology
Current mainstream server all supports clustering functionality, and traditional server cluster runs one on every physical server Individual example.Due to this scheme be in order to tackle peak load, it is not high in the utilization rate of many times server.PaaS is with flat The form of platform service provides computing resource for application, shields the complexity and isomerism of bottom cloud facility, more upper layer application There is provided flexible resource to provide, charter the support of the cloud computing such as resource-sharing key characteristic more, final realize ensures application performance, efficiently Utilize the purpose of platform resource.The IT infrastructure input of tenant is reduced, improves the resource utilization of data center.Cloud platform needs Ensure the performance isolation between tenant's application, and appropriate Server Consolidation is carried out using state according to Current resource.But The isolation method of current cluster has a great influence to cluster instance migration, and the degree of resource-sharing is not high, easily produces server Overload, influences application performance.
The content of the invention
To solve the problems of above-mentioned prior art, the present invention proposes a kind of middleware distributed computing method, Including:
It is determined that in Idle state and in overload state server, according to the overall situation resource and load distribution situation, by Distribute to resource reclaim manager and resource provides manager and performs resource adjustment behavior;The load point provided using each cluster With server, load is redistributed, the adjustment behavior includes integrating idle server and division Overloaded Servers.
Preferably, the integration idle server further comprises:First server set idle in lookup system, is every Individual cluster instance selects a suitable destination server, on the premise of system operation constraint is ensured, by these cluster instances Move in destination server, originally idle server closing or into holding state, if the server environment of system makes Built, then directly withdrawn virtual machine with virtualization technology;
If i-th of server SiCurrent resource usage amount KSiThe Current resource total amount TKS of/i-th serveri< 0.3, the i.e. low utilization of resources 30%, then server free, possesses the condition of Server Consolidation, is integrated so as to reduce work Make the quantity of server, the cluster instance in n platform servers is re-assigned in m platform servers, wherein n is physical server Number, and m is less than n;During Server Consolidation, by cluster instance from a server migration to another server Before, first start identical cluster instance in destination server, start successfully and then by the cluster instance in current server Close, the load distribution server of cluster is modified to its forwarding list, forwarded requests in new clustered node, realizes collection The migration of group node;
The result of integration is predicted during Server Consolidation, prevents the destination server after integrating from occurring Carry, the server set of idle condition is in lookup system, and carry out descending arrangement, by the cluster instance on each server Move in server forward after sorting, minimize cluster instance migration number;It is currently used to count all cluster instances Total resources, and minimum number of servers serversize is searched in server list, exceed in the total amount of every kind of resource In the case that all cluster instances take 1.5 times of total resources, all cluster instances are all moved to above-mentioned In the server of serversize quantity, S is migratediIn all cluster instances into preceding serversize platforms server, wherein serversize<I≤n, the desired stock number after destination server migration is calculated before migrating cluster instance, and selective value is most Small server, cluster instance is migrated.
Preferably, the division Overloaded Servers further comprise:Selection needs to migrate from the server of overload The part cluster instance gone out, and these cluster instances are re-deployed in other servers of system, eliminate server Overload;
If if i-th of server SiCurrent resource usage amount KSiThe Current resource total amount TKS of/i-th serveri >0.8, i.e. resource utilization is more than 80%, then server resource overload occurs, it is necessary to carry out server division;Partition process can To be divided into following steps:
(1) search division in need server set full_servers;(2) for any Si∈full_ Servers, search the cluster instance set cluster_nodes [S for needing to migrate outi] so that these cluster instances migrate Server is at busy and non-overloaded level after going out;(3) to each in SiJ-th of cluster instance S of upper operationij ∈cluster_nodes[Si], by cluster instance SijIt is placed into other servers of platform;Reduce branch server node Si The cluster instance set cluster_nodes [S divided awayi]。
The present invention compared with prior art, has advantages below:
The present invention be used for cluster distributed environment under charter resource-sharing, prevent server resource waste and overload, Increase operation rate.
Brief description of the drawings
Fig. 1 is the flow chart of middleware distributed computing method according to embodiments of the present invention.
Embodiment
Retouching in detail to one or more embodiment of the invention is hereafter provided together with the accompanying drawing for illustrating the principle of the invention State.The present invention is described with reference to such embodiment, but the invention is not restricted to any embodiment.The scope of the present invention is only by right Claim limits, and the present invention covers many replacements, modification and equivalent.Illustrate in the following description many details with Thorough understanding of the present invention is just provided.These details are provided for exemplary purposes, and without in these details Some or all details can also realize the present invention according to claims.
The present invention realizes the server cluster system towards PaaS.And from two layers of logical construction and physical arrangement to cluster Architecture is introduced.An aspect of of the present present invention provides a kind of middleware distributed computing method.Fig. 1 is according to the present invention The middleware distributed computing method flow chart of embodiment.
In logical layer, system includes multiple server clusters, the corresponding application of each cluster, each server cluster Including a load distribution server and one or more stateless server instance, and apply correlation behavior generally use Distributed caching is safeguarded that server instance failure does not interfere with application availability.When server instance needs migration, only Need to start new process in destination server, original process is closed in current server, while change cluster load distribution service The forwarding list of device.Compared to virtual machine (vm) migration, such moving method is very small to application performance impact.
In physical layer, system includes multiple servers, and as the running environment of cluster, each server instance is with operation The mode of system process is operated in these servers;Can run on the same server one or more come from it is multiple The server instance of cluster, these examples can share service and the resource that same operating system provides, and in global scope, All application clusters share identical server environment.
Within the physical layer, including two class server nodes:Primary server joint and branch server node.Master server section The periodic status that point is used to receive branch server node is reported, and planning of implemening resource integration;Multiple branch server sections Point is used to monitor and report the various performance datas of the server instance of operation, while receives and perform the control of primary server joint System order.Therefore, the resource adjustment operation of cluster is calculated by primary server joint, and control command is sent into branch Server node performs, and reaches the effect of adjustment finally by the forwarding list change of load distribution server.In order to disperse wind Nearly and the wasting of resources is avoided, can only be deposited in same server using the server instance of the deployment same cluster of constraint requirements At one.
The resource consolidation of system is periodic behavior, realizes the adaptive offer of resource.Branch server monitoring nodes The resource service condition of server and cluster instance, and obtained information is shipped regularly to primary server joint.Master server Determination methods below node use, the server in Idle state and in overload state is searched, and consider the overall situation Resource and load distribution situation, formulate a series of resource comprising adjustment behaviors and strategy is provided;Resource management component will be adjusted and gone Manager is provided and performed to distribute to resource reclaim manager and resource;Finally rely on the load distribution service that each cluster provides Device, load is redistributed.
What server resource was integrated aim at ensure in group system each application can normal operation while, reduce The usage quantity of server in host environment, energy resource consumption is reduced, reduce the operation cost of data center.It mainly passes through the free time The integration of server and the division of Overloaded Servers are realized.
Server Consolidation needs server set idle in first lookup system, for each cluster instance selection one above Individual suitable destination server, on the premise of system operation constraint is ensured, these cluster instances are moved into destination server In, originally idle server can close or into the power save mode such as standby.If the server environment of system uses virtual Change technique construction, then directly withdraw these virtual machines.
Server is divided for handling the server in overload, and therefrom selection needs the part cluster migrated out Example, and these cluster instances are re-deployed in other servers of system, so as to eliminate the overload shape of these servers State, the resource contention between cluster instance in the server is reduced, performance isolation is realized with this.
Hereafter with S1、S2、…SnRepresent n physical server in group system, ssize (Si) representative server SiUpper fortune Capable server instance quantity, SijRepresent in SiJ-th of cluster instance of upper operation.KSiRepresent the current money of i-th of server Source usage amount, KSijRepresent SiJ-th of cluster instance Current resource usage amount, TKSiRepresent the current money of i-th of server Source total amount.Resource herein can refer to the power of one or several kinds of data of the indexs such as CPU, internal memory, network read/write, disk read/write Sum again.
Server and cluster instance state are simply judged, suitable resource consolidation is selected further according to judged result Method.Judged result is described using symbol full and idle herein:
If KSi/TKSi>0.8, i.e. resource utilization is more than 80%, then full (SiThere is resource in)=true, server Overload is, it is necessary to carry out server division.If KSi/TKSi<0.3, the i.e. low utilization of resources 30%, then idle (Si)= True, server free, possesses the condition of Server Consolidation.
The object of Server Consolidation is all servers in idle condition, and they are integrated into less server, So as to reduce the quantity of workspace server, the cluster instance in n platform servers is re-assigned in m platform servers, subtracted as far as possible The size of few m values, and ensure that this m platform server is not in overload.
During Server Consolidation, some of which cluster instance is serviced from a server migration to another Device.Because clustered node is stateless, it is possible to first start identical cluster instance in destination server, after success, The cluster instance in current server is closed again.The load distribution server of cluster can modify to its forwarding list, will Request is forwarded in new clustered node, realizes the migration of clustered node.
When carrying out Server Consolidation, consider the cost brought due to cluster instance migration, reduce the movement of cluster instance. Avoid same cluster instance migration multiple, while need to be predicted the result of integration, prevent the purpose clothes after integrating Business device overloads.Performance parameter, such as CPU, network, response time.
Integration process first looks for being in the server set of idle condition in system, and carries out descending arrangement.Will rearward Cluster instance on the server of position is moved in server on the front, is minimized cluster instance migration number, is reduced system System change, keep the stabilization of each application cluster and system.The currently used total resources of all cluster instances is counted, is being serviced The number of servers serversize of minimum is searched in device list, ensures that the total amount of every kind of resource takes more than all cluster instances 1.5 times of total resources, then it can try one's best and all move to all cluster instances in this serversize server.Migration Si(serversize<I≤n) in all cluster instances into preceding serversize platforms server.Migration cluster instance it Desired stock number after preceding calculating destination server migration, and the server that selective value is minimum, cluster instance is migrated, and Overloaded after preventing migration.
The object of server division is the server for having occurred overload, it is therefore an objective to which part cluster instance is divided into others Run in server, eliminate the overload of the server, improve overall operational efficiency.
Partition process can be divided into following steps:
(1) search division in need server set full_servers.
(2) for any Si∈ full_servers, search the cluster instance set cluster_ for needing to migrate out nodes[Si] so that these cluster instances migrate out rear server and are at busy and non-overloaded level.
(3) to each clustered node Sij∈cluster_nodes[Si], by cluster instance SijIt is placed into other clothes of platform It is engaged in device.
Branch server node S is reduced as far as possibleiThe cluster instance set cluster_nodes [S divided awayi], to reduce The cost of cluster instance migration, therefore it is required that cluster_nodes [Si] in each cluster instance there is higher resource Usage amount, undertake more load.But for loading higher clustered node, born due to needing to shift substantial amounts of user Carry so that migration cost is higher, in transition process, can produce certain influence to the service quality of application, therefore require again Migration loads relatively low clustered node.Integrate, pay the utmost attention to migrate the quantity of clustered node, secondly consider migration low-load Node.Assuming that N=ssize (Si), then total time complexity is O (N2)。
In summary, the present invention be used under cluster distributed environment charter resource-sharing, prevent that server resource is unrestrained Take and overload, increase operation rate.
Obviously, can be with general it should be appreciated by those skilled in the art, above-mentioned each module of the invention or each step Computing system realize that they can be concentrated in single computing system, or be distributed in multiple computing systems and formed Network on, alternatively, they can be realized with the program code that computing system can perform, it is thus possible to they are stored Performed within the storage system by computing system.So, the present invention is not restricted to any specific hardware and software combination.
It should be appreciated that the above-mentioned embodiment of the present invention is used only for exemplary illustration or explains the present invention's Principle, without being construed as limiting the invention.Therefore, that is done without departing from the spirit and scope of the present invention is any Modification, equivalent substitution, improvement etc., should be included in the scope of the protection.In addition, appended claims purport of the present invention Covering the whole changes fallen into scope and border or this scope and the equivalents on border and repairing Change example.

Claims (2)

1. a kind of middleware distributed computing method, for being implement resource integration in the server cluster system towards PaaS, It is characterised in that it includes:
It is determined that the server in Idle state and in overload state, according to the resource and load distribution situation of the overall situation, by distributing Manager, which is provided, to resource reclaim manager and resource performs resource adjustment behavior;The load provided using each cluster distributes clothes Business device, load is redistributed, and the adjustment behavior includes integrating idle server and division Overloaded Servers;
The integration idle server further comprises:First server set idle in lookup system, it is each cluster instance A suitable destination server is selected, on the premise of system operation constraint is ensured, these cluster instances are moved into purpose In server, originally idle server closing or into holding state, if the server environment of system uses virtualization skill Art is built, then is directly withdrawn virtual machine;
If i-th of server SiCurrent resource usage amount KSiThe Current resource total amount TKS of/i-th serveri<0.3, i.e., The low utilization of resources 30%, then server free, possesses the condition of Server Consolidation, is integrated so as to reduce work service The quantity of device, the cluster instance in n platform servers being re-assigned in m platform servers, wherein n is physical server number, And m is less than n;During Server Consolidation, by cluster instance from a server migration to another server before, First start identical cluster instance in destination server, start successfully and then close the cluster instance in current server, The load distribution server of cluster is modified to its forwarding list, is forwarded requests in new clustered node, realizes cluster section The migration of point;
The result of integration is predicted during Server Consolidation, prevents the destination server after integrating from overloading, looks into Look in system and be in the server set of idle condition, and carry out descending arrangement, the cluster instance on each server is migrated After to sequence in forward server, cluster instance migration number is minimized;Count the currently used resource of all cluster instances Total amount, and minimum number of servers serversize is searched in server list, exceed in the total amount of every kind of resource all In the case that cluster instance takes 1.5 times of total resources, all cluster instances are all moved into above-mentioned serversize numbers In the server of amount, S is migratediIn all cluster instances into preceding serversize platforms server, wherein serversize<i ≤ n, the desired stock number after destination server migration is calculated before migrating cluster instance, and the server that selective value is minimum, Cluster instance is migrated.
2. according to the method for claim 1, it is characterised in that the division Overloaded Servers further comprise:From overload Selection needs the part cluster instance migrated out in the server of state, and these cluster instances are re-deployed into system In other servers, the overload of server is eliminated;
If i-th of server SiCurrent resource usage amount KSiThe Current resource total amount TKS of/i-th serveri>0.8, i.e., Resource utilization is more than 80%, then server resource overload occurs, it is necessary to carry out server division;Partition process can be divided into Lower step:
(1) search division in need server set full_servers;(2) for any Si∈ full_servers, are looked into Look for the cluster instance set cluster_nodes [S that needs migrate outi] so that these cluster instances service after migrating out Device is at busy and non-overloaded level;(3) to each in SiJ-th of cluster instance S of upper operationij∈cluster_ nodes[Si], by cluster instance SijIt is placed into other servers of platform;Reduce branch server node SiDivide away Cluster instance set cluster_nodes [Si]。
CN201510122457.4A 2015-03-19 2015-03-19 A kind of middleware distributed computing method Expired - Fee Related CN104679594B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510122457.4A CN104679594B (en) 2015-03-19 2015-03-19 A kind of middleware distributed computing method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510122457.4A CN104679594B (en) 2015-03-19 2015-03-19 A kind of middleware distributed computing method

Publications (2)

Publication Number Publication Date
CN104679594A CN104679594A (en) 2015-06-03
CN104679594B true CN104679594B (en) 2017-11-14

Family

ID=53314685

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510122457.4A Expired - Fee Related CN104679594B (en) 2015-03-19 2015-03-19 A kind of middleware distributed computing method

Country Status (1)

Country Link
CN (1) CN104679594B (en)

Families Citing this family (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104917836A (en) * 2015-06-10 2015-09-16 北京奇虎科技有限公司 Method and device for monitoring and analyzing availability of computing equipment based on cluster
CN104917639B (en) * 2015-06-10 2018-07-03 北京奇虎科技有限公司 Data service method and device is distributed based on cluster monitoring
CN105302630B (en) * 2015-10-26 2018-07-13 深圳大学 A kind of dynamic adjusting method and its system of virtual machine
CN107203255A (en) * 2016-03-20 2017-09-26 田文洪 Power-economizing method and device are migrated in a kind of network function virtualized environment
CN105868011A (en) * 2016-03-29 2016-08-17 上海斐讯数据通信技术有限公司 Method and device for improving running efficiency of software system
CN108304236B (en) * 2017-12-28 2021-03-16 麒麟软件有限公司 User interface refreshing method based on message subscription under cloud platform
CN109740178B (en) * 2018-11-27 2021-05-07 中国科学院计算技术研究所 Multi-tenant data center energy efficiency optimization method and system and combined modeling method
CN109451056A (en) * 2018-12-20 2019-03-08 中国软件与技术服务股份有限公司 Server dynamic allocation method and system between more clusters
CN111432159B (en) * 2020-03-19 2022-05-17 深圳市鹏创软件有限公司 Computing task processing method, device and system and computer readable storage medium

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102096461A (en) * 2011-01-13 2011-06-15 浙江大学 Energy-saving method of cloud data center based on virtual machine migration and load perception integration
CN103077082A (en) * 2013-01-08 2013-05-01 中国科学院深圳先进技术研究院 Method and system for distributing data center load and saving energy during virtual machine migration
CN103607459A (en) * 2013-11-21 2014-02-26 东北大学 Dynamic resource monitoring and scheduling method of cloud computing platform IaaS layer

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9058198B2 (en) * 2012-02-29 2015-06-16 Red Hat Inc. System resource sharing in a multi-tenant platform-as-a-service environment in a cloud computing system

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102096461A (en) * 2011-01-13 2011-06-15 浙江大学 Energy-saving method of cloud data center based on virtual machine migration and load perception integration
CN103077082A (en) * 2013-01-08 2013-05-01 中国科学院深圳先进技术研究院 Method and system for distributing data center load and saving energy during virtual machine migration
CN103607459A (en) * 2013-11-21 2014-02-26 东北大学 Dynamic resource monitoring and scheduling method of cloud computing platform IaaS layer

Also Published As

Publication number Publication date
CN104679594A (en) 2015-06-03

Similar Documents

Publication Publication Date Title
CN104679594B (en) A kind of middleware distributed computing method
CN107548549B (en) Resource balancing in a distributed computing environment
Krishnamurthy et al. Pratyaastha: an efficient elastic distributed sdn control plane
CN102479099B (en) Virtual machine management system and use method thereof
US8510745B2 (en) Dynamic application placement under service and memory constraints
US8914513B2 (en) Hierarchical defragmentation of resources in data centers
CN110221920B (en) Deployment method, device, storage medium and system
CN105159736B (en) A kind of construction method for the SaaS software deployment schemes for supporting performance evaluation
CN106685724B (en) Node server management method based on election, apparatus and system
Xu et al. Migration cost and energy-aware virtual machine consolidation under cloud environments considering remaining runtime
CN110837418A (en) High-concurrency web system based on container and implementation method
CN105245617A (en) A container-based server resource provisioning method
CN114356543A (en) A Kubernetes-based Multi-tenant Machine Learning Task Resource Scheduling Method
CN102158513A (en) Service cluster and energy-saving method and device thereof
CN110661842B (en) Resource scheduling management method, electronic equipment and storage medium
CN104050042A (en) Resource allocation method and resource allocation device for ETL (Extraction-Transformation-Loading) jobs
US12386680B2 (en) Resource scheduling based on optimal path of a resource label forest
CN103095788A (en) Cloud resource scheduling policy based on network topology
CN114598706B (en) Storage system elastic expansion method based on Serverless function
KR20130073449A (en) Distribution and management method of components having reliance
Bourhim et al. Inter-container communication aware container placement in fog computing
US20210373972A1 (en) Vgpu scheduling policy-aware migration
CN111767139A (en) Cross-region multi-data-center resource cloud service modeling method and system
CN105516267B (en) Cloud platform efficient operation method
Imdoukh et al. Optimizing scheduling decisions of container management tool using many‐objective genetic algorithm

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
CB03 Change of inventor or designer information

Inventor after: Jiang Guojian

Inventor before: Wang Meiting

CB03 Change of inventor or designer information
TA01 Transfer of patent application right

Effective date of registration: 20171016

Address after: 350000 2-57D room, 1 floor, No. 27 Lake Road, Mawei District, Fujian, Fuzhou province (FTA test area)

Applicant after: FUZHOU HUANYA ZHONGZHI COMPUTER CO., LTD.

Address before: West high tech Zone Fucheng Road in Chengdu city of Sichuan province 610000 399 No. 6 Building 1 unit 6 floor No. 6

Applicant before: CHENGDU YICHEN DEXUN TECHNOLOGY CO., LTD.

TA01 Transfer of patent application right
GR01 Patent grant
GR01 Patent grant
CF01 Termination of patent right due to non-payment of annual fee

Granted publication date: 20171114

Termination date: 20210319

CF01 Termination of patent right due to non-payment of annual fee