Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following embodiments of the present invention are described in further detail with reference to the accompanying drawings.
It should be noted that all expressions using "first" and "second" in the embodiments of the present invention are used for distinguishing two entities with the same name but different names or different parameters, and it should be noted that "first" and "second" are merely for convenience of description and should not be construed as limitations of the embodiments of the present invention, and they are not described in any more detail in the following embodiments.
In view of the foregoing, a first aspect of the embodiments of the present invention provides an embodiment of an active push method for a distributed system. Fig. 1 is a schematic diagram illustrating an embodiment of a method for active push of a distributed system provided by the present invention. As shown in fig. 1, the embodiment of the present invention includes the following steps:
s1, acquiring monitoring data to judge whether a message event exists;
s2, responding to the existence message event, notifying the client and receiving the request of the client;
s3, establishing corresponding threads according to the number of the requests of the client, and taking out a corresponding client message queue from the global service queue by each thread;
s4, resolving a corresponding message event from the client message queue according to the request of the client, and sending an event response request to the cluster; and
and S5, responding to the response returned by the cluster, and sending the message event to the client.
When the client and the server interact for the first time, authentication (TOKEN authentication or user management system) needs to be carried out on the client, authentication verification is carried out after the identity authentication information of the client is received, and if the authentication is successful, the request data receiving and sending thread is waited for further processing; the request data receiving and sending thread polls all client requests which are authenticated by the request processing thread, packs the received client requests into a message structure (such as a common Json object) and then sends the message data to a client message queue (ismClientQueue) corresponding to the client, and then adds the message queue to the tail of a global service queue (globalserviceQueue) and waits for further processing; if the message data is received, the message data is directly decoded and then forwarded to the client. The connection request mode and the identity authentication mode can be adapted to the current mainstream mode, and the invention does not make specific requirements.
Specifically, the following process may be adopted:
and judging whether the connection of the client is received or not, if so, going to the next step, and if not, continuing to wait for receiving the message connection request from the client. A client request processing thread is newly built and started and marked as A; and newly building and starting a request data transceiving thread, and marking as B. The temporary process data tmpArrA of a is initialized. The thread A monitors the client request, and the thread B polls the processing result array tmpArr of the thread A. The client requests the processing thread to authenticate and request the identity of the client, and if the authentication is successful, the client is put into tmpArr. The thread B acquires tmpArr data, if len (tmpArr) is larger than 0, the tmpArr data is packed into a message structure body, and then the tmp data is sent to an ismClientQueue queue; if tmpArr is empty, polling continues. And the thread B acquires the returned response data and sends the data to the corresponding client according to the client identifier.
The cluster includes two nested layers of message queues: a global service queue (globalServiceQueue) and a client message queue (ismClientQueue) to which the client has established a connection. The global service queue is a common global message circular queue (which can be implemented by using a common array), and is mainly responsible for maintaining and scheduling the client message queue (that is, different client message queues exist in the global service queue); the client message queue maintains different requests of the client, collects and monitors data in the storage cluster, and encapsulates data events into an ismDataItem form and puts the data events into the client message queue (namely, the client message queue has different requested ismDataItem message events) if the cluster data change is monitored and the changes need to be informed to the client (such as hardware equipment part failure in the cluster, storage pool/logical volume data update and other important operations and other monitorable items). The global service queue manages the client request queues as if there are multiple concurrent producers and concurrent consumers, so that locking is required for access operations such as message enqueuing or dequeuing. The global service queue directly allocates a default queue space (for example, 1000) to the queue during initialization, and automatically expands the capacity of the queue when the queue is full, generally expanding the capacity of the allocated space by twice.
Each element in the global service queue is a reference of a client message queue, a space is allocated by default and a plurality of empty client service queues are mounted during initialization, and when different clients send different requests, corresponding request message events are filled into the ismClientQueue. When the module is started, a plurality of working threads (6 working threads are configured by default, the number of the threads can be modified, and generally is not more than 128), when the working threads start working, a global service queue is locked, then a client message queue is continuously taken out from the global service queue, then a message event (ismDataItem) is taken out from the client message queue, and a corresponding cluster service is called (generally, a callback operation is performed, for example, a disk state alarm or a storage pool creation operation event is obtained from an alarm service of a cluster); after the message event is acquired, the message is decoded and packaged into message data with a corresponding format, the message data is directly sent to the client request monitoring module, and the request data is processed by a request data receiving and sending thread in the client request monitoring module and returned to the client. Namely: the request data response operation does not pass through a message queue, and the data result is directly forwarded to the client request monitoring module for processing.
Monitoring data is acquired to determine whether a message event exists. Data changes or device state changes belong to message events, so if there are data changes or device state changes in the cluster, it can be regarded as a message event.
In response to the presence message event, a request is sent to the client to notify the client and receive the client's request. If the cluster has the message event, the server side sends a request to the client side to inform the client side that the request can be sent, and at the moment, the client side sends a corresponding request according to the requirement.
In some embodiments, establishing the corresponding threads according to the number of requests of the client, wherein each thread fetching the corresponding client message queue from the global service queue further comprises: and locking a corresponding client message queue, and performing format conversion on the message event to write the message event into the client message queue.
In some embodiments, the method further comprises: and responding to a new message event written in the client message queue, and writing the client message queue into the tail of the global service queue.
And establishing corresponding threads according to the number of the requests of the client, and taking out a corresponding client message queue from the global service queue by each thread. After receiving the requests sent by the client, the server establishes a corresponding number of threads according to the number of the requests sent by the client, for example, one thread is established by one request, or two threads are established by one request, or one thread is established by two requests. After the thread starts, the global service queue is locked. Each thread determines a corresponding client according to the source of the request, determines a client message queue based on the client, and then takes the client message queue out of the global service queue.
And analyzing a corresponding message event from the client message queue according to the request of the client, and sending an event response request to the cluster. The thread analyzes the request of the client, then determines the message event needed by the request, and then analyzes the corresponding message event from the message queue of the client, if the message event needed by the client exists in the message queue of the client, the thread calls the request service of the corresponding ismDataItem, and sends an event response request to the cluster.
And sending the message event to the client in response to receiving the response returned by the cluster. In some embodiments, said sending said message event to said client comprises: decoding the message event and format converting the decoded message event for sending to the client. And releases the global service queue.
The invention provides an active pushing method in a distributed storage system.A storage cluster actively pushes request data to a client in a client queue by collecting monitoring data of the storage cluster when data updating exists in a data queue. The invention can not only reduce the development task amount of the client, but also reduce the pressure on the server caused by polling the cluster data of the server by the client, and improve the real-time performance and the accuracy of cluster monitoring; meanwhile, for the development framework, the request and the service are flattened, and the cluster monitoring logic sinks to the service end, so that the integrity and the usability of the whole development framework are facilitated.
It should be particularly noted that, the steps in the embodiments of the method for unsolicited push request described above can be mutually intersected, replaced, added, or deleted, so that these methods for unsolicited push request transformed by reasonable permutation and combination also belong to the scope of protection of the present invention, and the scope of protection of the present invention should not be limited to the embodiments.
In view of the above, a second aspect of the embodiments of the present invention provides an active push system of a distributed system, including: the acquisition module is configured to acquire monitoring data to judge whether a message event exists; a receiving module configured to notify a client and receive a request of the client in response to a presence message event; the thread module is configured to establish corresponding threads according to the number of the requests of the client, and each thread takes out a corresponding client message queue from the global service queue; the execution module is configured to analyze a corresponding message event from the client message queue according to the request of the client and send an event response request to the cluster; and the sending module is configured to send the message event to the client in response to receiving a response returned by the cluster.
In some embodiments, the thread module is further configured to lock a corresponding client message queue and format convert the message event for writing to the client message queue.
In some embodiments, further comprising: and the second queue module is configured to respond to a new message event written in the client message queue and write the client message queue into the tail of the global service queue.
In some embodiments, the sending module is further configured to: decoding the message event and format converting the decoded message event for sending to the client.
In view of the above object, a third aspect of the embodiments of the present invention provides a computer device, including: at least one processor; and a memory storing computer instructions executable on the processor, the instructions being executable by the processor to perform the steps of: s1, acquiring monitoring data to judge whether a message event exists; s2, responding to the existence message event, notifying the client and receiving the request of the client; s3, establishing corresponding threads according to the number of the requests of the client, and taking out a corresponding client message queue from the global service queue by each thread; s4, resolving a corresponding message event from the client message queue according to the request of the client, and sending an event response request to the cluster; and S5, responding to the response returned by the cluster, and sending the message event to the client.
In some embodiments, establishing the corresponding threads according to the number of requests of the client, wherein each thread fetching the corresponding client message queue from the global service queue further comprises: and locking a corresponding client message queue, and performing format conversion on the message event to write the message event into the client message queue.
In some embodiments, further comprising: and responding to a new message event written in the client message queue, and writing the client message queue into the tail of the global service queue.
In some embodiments, said sending said message event to said client comprises: decoding the message event and format converting the decoded message event for sending to the client.
Fig. 2 is a schematic hardware structural diagram of an embodiment of the computer device for active push request according to the present invention.
Taking the apparatus shown in fig. 2 as an example, the apparatus includes a processor 301 and a memory 302, and may further include: an input device 303 and an output device 304.
The processor 301, the memory 302, the input device 303 and the output device 304 may be connected by a bus or other means, and fig. 2 illustrates the connection by a bus as an example.
The memory 302 is a non-volatile computer-readable storage medium and can be used for storing non-volatile software programs, non-volatile computer-executable programs, and modules, such as program instructions/modules corresponding to the method of unsolicited push request in the embodiments of the present application. The processor 301 executes various functional applications of the server and data processing by running nonvolatile software programs, instructions and modules stored in the memory 302, namely, the method of implementing the active push request of the above-described method embodiment.
The memory 302 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the method of the active push request, and the like. Further, the memory 302 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some embodiments, memory 302 optionally includes memory located remotely from processor 301, which may be connected to a local module via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input device 303 may receive information such as a user name and a password that are input. The output means 304 may comprise a display device such as a display screen.
Program instructions/modules corresponding to the one or more methods of proactive push request are stored in the memory 302 and when executed by the processor 301, perform the methods of proactive push request in any of the above-described method embodiments.
Any embodiment of the computer device executing the method for actively pushing a request may achieve the same or similar effects as any corresponding embodiment of the method described above.
The invention also provides a computer readable storage medium storing a computer program which, when executed by a processor, performs the method as above.
Finally, it should be noted that, as one of ordinary skill in the art can appreciate that all or part of the processes in the methods of the above embodiments can be implemented by a computer program to instruct related hardware, and the program of the method for actively pushing a request can be stored in a computer readable storage medium, and when executed, the program can include the processes of the embodiments of the methods as described above. The storage medium of the program may be a magnetic disk, an optical disk, a Read Only Memory (ROM), a Random Access Memory (RAM), or the like. The embodiments of the computer program may achieve the same or similar effects as any of the above-described method embodiments.
Furthermore, the methods disclosed according to embodiments of the present invention may also be implemented as a computer program executed by a processor, which may be stored in a computer-readable storage medium. Which when executed by a processor performs the above-described functions defined in the methods disclosed in embodiments of the invention.
Further, the above method steps and system elements may also be implemented using a controller and a computer readable storage medium for storing a computer program for causing the controller to implement the functions of the above steps or elements.
Further, it should be appreciated that the computer-readable storage media (e.g., memory) herein can be either volatile memory or nonvolatile memory, or can include both volatile and nonvolatile memory. By way of example, and not limitation, nonvolatile memory can include Read Only Memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM), which can act as external cache memory. By way of example and not limitation, RAM is available in a variety of forms such as synchronous RAM (DRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The storage devices of the disclosed aspects are intended to comprise, without being limited to, these and other suitable types of memory.
Those of skill would further appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the disclosure herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as software or hardware depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the disclosed embodiments of the present invention.
The various illustrative logical blocks, modules, and circuits described in connection with the disclosure herein may be implemented or performed with the following components designed to perform the functions herein: a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP, and/or any other such configuration.
The steps of a method or algorithm described in connection with the disclosure herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In the alternative, the processor and the storage medium may reside as discrete components in a user terminal.
In one or more exemplary designs, the functions may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. A storage media may be any available media that can be accessed by a general purpose or special purpose computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. Disk and disc, as used herein, includes Compact Disc (CD), laser disc, optical disc, Digital Versatile Disc (DVD), floppy disk, blu-ray disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.
The foregoing is an exemplary embodiment of the present disclosure, but it should be noted that various changes and modifications could be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions, steps and/or actions of the method claims in accordance with the disclosed embodiments described herein need not be performed in any particular order. Furthermore, although elements of the disclosed embodiments of the invention may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.
It should be understood that, as used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly supports the exception. It should also be understood that "and/or" as used herein is meant to include any and all possible combinations of one or more of the associated listed items.
The numbers of the embodiments disclosed in the embodiments of the present invention are merely for description, and do not represent the merits of the embodiments.
It will be understood by those skilled in the art that all or part of the steps for implementing the above embodiments may be implemented by hardware, or may be implemented by a program instructing relevant hardware, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.
Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, of embodiments of the invention is limited to these examples; within the idea of an embodiment of the invention, also technical features in the above embodiment or in different embodiments may be combined and there are many other variations of the different aspects of the embodiments of the invention as described above, which are not provided in detail for the sake of brevity. Therefore, any omissions, modifications, substitutions, improvements, and the like that may be made without departing from the spirit and principles of the embodiments of the present invention are intended to be included within the scope of the embodiments of the present invention.