CN111090691B - A data processing method, device, electronic device and storage medium - Google Patents

Abstract

The present application discloses a data processing method, apparatus, electronic device and storage medium, and relates to the field of data processing. The specific implementation scheme is as follows: the unified management platform, in response to the data source access request of the target content provider, obtains the access type and access parameters matching the data source access request, and the unified management platform connects multiple content providers in advance; The management platform obtains the target data source access module matching the access type in multiple types of data source access modules; the unified management platform triggers the target data source access module according to the access parameters, and the access and data source access A matching target data source is requested; the unified management platform obtains the target data from the target data source, and stores the target data in the storage space matched with the target content provider. The technical solution of the present application can realize that the data sources of different content providers are managed and maintained by a unified management platform.

Description

A data processing method, device, electronic device and storage medium

technical field

The embodiments of the present application relate to computer technology, and in particular, to a data processing method, apparatus, electronic device, and storage medium.

Background technique

With the development of computer technology, community Internet products have been widely used. In community-based Internet products, there are a large number of high-quality time-sensitive content, and it is necessary to uniformly manage the high-quality and time-sensitive content of each content provider of community-based Internet products.

In the prior art, the data sources of each content provider of community-based Internet products are relatively independent from each other, and it is impossible to uniformly manage the high-quality and time-sensitive content of each content provider, and it is impossible to achieve high-quality content among various content providers. , Flexible sharing of time-sensitive content, the maintenance and sharing of high-quality and time-sensitive content is costly, and the user experience is poor.

SUMMARY OF THE INVENTION

The embodiments of the present application disclose a data processing method, apparatus, electronic device, and storage medium, which can implement the management and maintenance of data sources of different content providers by a unified management platform.

In a first aspect, an embodiment of the present application discloses a data processing method, including:

The unified management platform acquires the access type and access parameter matching the data source access request in response to the data source access request of the target content provider, and the unified management platform connects multiple content providers in advance;

The unified management platform obtains, among multiple types of data source access modules, a target data source access module matching the access type;

The unified management platform triggers the target data source access module according to the access parameter, and accesses the target data source matching the data source access request;

An embodiment in the above application has the following advantages or beneficial effects:

By connecting the data sources of different content providers to the unified management platform, the data sources of different content providers can be managed and maintained by a unified management platform.

In addition, the data processing method according to the above-mentioned embodiment of the present application may also have the following additional technical features:

Further, after the unified management platform triggers the target data source access module according to the access parameter to access the target data source matching the data source access request, the method further includes:

The unified management platform acquires target data from the target data source, and stores the target data in a storage space matched with the target content provider.

An embodiment in the above application has the following advantages or beneficial effects:

The target data sources required by different content providers are connected to the unified management platform for storage, which can realize unified management and flexible sharing of target data sources.

Further, after the unified management platform accesses the target data source matching the data source access request, the method further includes:

The unified management platform, in response to the permission setting instruction of the target content provider, sets the viewing permission of the target data source as a private permission or a public permission;

Wherein, a data source set as private permission is only allowed to be accessed by content providers who have access to the data source, and a data source set as public permission is also allowed to be accessed by other content providers with viewing permission.

An embodiment in the above application has the following advantages or beneficial effects:

By setting the permissions of the data source, different data sources can access the data source flexibly according to the set permissions.

Further, after the unified management platform sets the viewing authority of the target data source as a public authority in response to the authority setting instruction of the target content provider, the method further includes:

The unified management platform responds to data access requests from other content providers;

If the unified management platform determines that the other content provider is another content provider with viewing rights, it provides the stored target data to the other content provider for access.

An embodiment in the above application has the following advantages or beneficial effects:

When the viewing permission of the target data source is set to public permission, the data sharing of the target data source to other content providers with open permissions can be realized.

Further, the method also includes:

The unified management platform counts the read and write status of each connected data source;

The unified management platform performs offline and/or online processing on each of the data sources according to the read and write status;

The data source in the offline state cannot be accessed by the content provider, and the data source in the online state can be accessed by the content provider.

An embodiment in the above application has the following advantages or beneficial effects:

It realizes the management of the data source online and offline, realizes the intelligent switching of the data source status, and reduces the labor cost of maintaining the data source.

Further, the unified management platform performs offline and/or online processing on each of the data sources according to the read and write status, including:

If the unified management platform determines that the number of reads and writes of the first data source currently in the online state within the first time interval is less than the first threshold, the storage state of the first data corresponding to the first data source is modified by the exposed storage. For hidden storage, to perform offline processing on the first data source;

If the unified management platform determines that the number of reads and writes of the second data source currently in the offline state in the second time interval is greater than or equal to the second threshold, then the storage state of the second data corresponding to the second data source will be hidden. The storage is modified to expose storage for online processing of the first data source.

An embodiment in the above application has the following advantages or beneficial effects:

The intelligent switching of the online and offline status of the data source is realized, the management efficiency of the data source is improved, and the cost of manual maintenance of the data source is reduced.

Further, after the unified management platform counts the read and write status of each connected data source, it also includes:

If the unified management platform determines that the number of times of reading and writing in the third time interval of the third data source currently in the offline state is less than the third threshold, then disconnecting the access to the third data source;

The unified management platform synchronously deletes all data corresponding to the third data source in the storage space.

An embodiment in the above application has the following advantages or beneficial effects:

The timely recovery of invalid data sources is realized, and the waste of resources caused by the long-term occupation of storage space by invalid data sources is reduced.

Further, the access type includes at least one of the following:

Application program interface push-pull access, one-key mirroring access, file import access, simple information aggregation access and fifth-generation hypertext markup language page access.

An embodiment in the above application has the following advantages or beneficial effects:

The access time of the data source is reduced, and the fast access of the data source is realized.

Further, the content provider is a provider of vertical services in the network site.

Other effects of the above-mentioned optional manners will be described below with reference to specific embodiments.

In a second aspect, an embodiment of the present application discloses a data processing device, including:

an access type and access parameter acquisition module, configured to make the unified management platform respond to a data source access request from a target content provider to acquire an access type and access parameters matching the data source access request, the The unified management platform connects multiple content providers in advance;

a target data source access module acquiring module, configured to enable the unified management platform to acquire a target data source access module matching the access type from among multiple types of data source access modules;

a target data source access module, configured to enable the unified management platform to trigger the target data source access module according to the access parameter to access a target data source matching the data source access request;

In a third aspect, an embodiment of the present application discloses an electronic device, including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute any one of the embodiments of the present application. method.

In a fourth aspect, an embodiment of the present application discloses a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the description in any one of the embodiments of the present application. Methods.

In the technical solutions of the embodiments of the present application, the data sources of multiple content providers are connected to the unified management platform through the access module. It solves the problem of high management and maintenance cost caused by independent data sources among various content providers in the prior art. It realizes the unified maintenance and management of different data sources by a unified management platform.

Description of drawings

The accompanying drawings are used to better understand the present solution, and do not constitute a limitation to the present application. in:

1 is a flowchart of a data processing method provided in Embodiment 1 of the present application;

2 is a flowchart of a data processing method provided in Embodiment 2 of the present application;

3 is a flowchart of a data processing method provided in Embodiment 3 of the present application;

4 is a schematic structural diagram of a data processing apparatus provided in Embodiment 4 of the present application;

FIG. 5 is a schematic structural diagram of an electronic device provided in Embodiment 5 of the present application.

Detailed ways

Exemplary embodiments of the present application are described below with reference to the accompanying drawings, which include various details of the embodiments of the present application to facilitate understanding, and should be considered as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present application. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.

Example 1

FIG. 1 is a flowchart of a data processing method provided in Embodiment 1 of the present application. This embodiment is applicable to the situation where unified management and maintenance of data sources of different content providers is required, and the method can be performed by a data processing device. To implement, the apparatus can be implemented by software and/or hardware, and is generally integrated in a server.

As shown in Figure 1, the technical solution of the embodiment of the present application specifically includes the following steps:

S110. The unified management platform acquires the access type and access parameters matching the data source access request in response to the data source access request of the target content provider, and the unified management platform connects multiple content providers in advance.

Among them, the role of the unified management platform is to enable unified access to the data sources and to manage the data in the data sources in a unified manner. The content provider may be a subject that provides certain services, and the content provider may establish a connection with the unified management platform in advance. The unified management platform may include multiple content providers, and the embodiment of the present application does not limit the specific number of content providers. The data source can be a device or original media that provides some required data, and all the information for establishing a database connection can be stored in the data source. The access type can be used to indicate how the data source is connected to the unified management platform, and the access parameter can be a parameter that needs to be configured when the data source is connected to the unified management module.

In the embodiment of the present application, the unified management platform connects multiple content providers in advance, where the target content provider refers to a content provider that currently needs to access a new data source, and the target content provider needs to access the data source in a unified manner When managing the platform, a data source access request is sent to the unified management platform, and the access type and access parameters can be obtained according to the data source access request.

In an optional embodiment of the present application, the access type may include at least one of the following: application program interface push-pull access, one-key mirroring access, file import access, and simple information aggregation access access method and the fifth generation hypertext markup language page access method.

The push-pull access method of the application programming interface (Application Programming Interface, API) may be to directly access the database through the application programming interface. The push-pull access method of the application program interface can include two methods: push and pull. The push method can be that after establishing a connection with the database, the database actively pushes data; the pull method can send a data acquisition request to the database after establishing a connection with the database. The content provider can flexibly choose the access method of the application program interface according to its own business characteristics and risk control. The one-key mirroring access method requires pre-configuration by the user, which can realize fast synchronization of data. The file import access method can generate files according to the data, and import the data by selecting the corresponding file and analyzing the file. A Simple Syndication (Really Simple Syndication, RSS) access method may refer to a simple method for sharing content between a site and other sites. The fifth-generation hypertext markup language (Hyper Text Markup Language 5, H5) access method may be to establish a connection with the database through the H5 interface, and obtain data by parsing the interface.

In the embodiment of the present application, multiple access types of data sources are set. The advantage of this setting is that the access modes can be flexibly configured according to the actual needs of the data sources, the access time of the data sources is reduced, and the data source is realized. quick access.

In an optional embodiment of the present application, the content provider may be a provider of vertical services in a network site.

The unified management platform may include multiple vertical categories, and different vertical categories correspond to different content providers. The vertical category can be a section included in the unified management platform. In a specific example, a community-based Internet product can include a unified management platform, and the unified management platform can include cute pet game sections, entertainment sections, and chat sections, etc. Games, entertainment, and chat belong to different vertical categories, corresponding to different content providers. Correspondingly, different content providers are providers of setting services for different vertical categories.

S120: The unified management platform obtains, from among multiple types of data source access modules, a target data source access module matching the access type.

The function of the data source access module is to access the data source, the data source access module may correspond to the access type of the data source, and different data source access types may correspond to different data source access modules. In a specific example, the application program interface pushes and pulls the data source access module, and the application program interface push and pull method can be used to connect the data source to the unified management platform. In the unified management platform, access types of multiple data sources can be set, and correspondingly, multiple types of data source access modules can be set. Optionally, the data source access module may be pre-configured in the unified management platform.

In the embodiment of the present application, among the various types of data source access modules set on the unified management platform, a data source access module corresponding to the access type is obtained, and used as a target data source access module to connect to the target data source enter.

S130. The unified management platform triggers the target data source access module according to the access parameter, and accesses the target data source matching the data source access request.

In the embodiment of the present application, the access parameter in the target data source access request may trigger the target data source access module, and after the target data source access module is triggered, the target data source is connected to the unified management platform.

In the technical solutions of the embodiments of the present application, the data sources of multiple content providers are connected to the unified management platform through the access module. It solves the problem of high management and maintenance cost caused by independent data sources among various content providers in the prior art. It realizes the unified maintenance and management of different data sources by a unified management platform.

Embodiment 2

FIG. 2 is a flowchart of a data processing method provided in Embodiment 2 of the present application. On the basis of the above-mentioned embodiment, in this embodiment of the present application, after the step of accessing the target data source, setting the viewing authority of the target data source is added, and The step of storing the target data in the target data source in the storage space, and the step of calling the target data according to the viewing authority of the target data source is further specified.

Correspondingly, as shown in FIG. 2 , the technical solution of the embodiment of the present application specifically includes the following steps:

S210. The unified management platform acquires the access type and access parameters matching the data source access request in response to the data source access request of the target content provider, and the unified management platform connects multiple content providers in advance.

S220: The unified management platform obtains, from among multiple types of data source access modules, a target data source access module matching the access type.

S230. The unified management platform triggers the target data source access module according to the access parameter, and accesses the target data source matching the data source access request.

S240. The unified management platform, in response to the permission setting instruction of the target content provider, sets the viewing permission of the target data source as a private permission or a public permission.

The permission setting instruction may be an instruction issued by the content provider to set the viewing permission of the data source. View permissions can be used to indicate whether data in a data source can be accessed by other data sources.

In this embodiment of the present application, a data source set to private authority may only allow access to content providers accessing the data source, and a data source set to public authority may also allow access to all content providers accessing the data source. Content provider access to which permissions are issued.

One server provider may issue authority to all server providers included in the unified management platform, or may issue authority to only one or specified several content providers included in the unified management platform, which is not performed in this embodiment. limit.

S250. The unified management platform acquires target data from the target data source, and stores the target data in a storage space matching the target content provider.

The target data may be data that needs to be shared. In a specific example, the target data may be high-quality data in the target data source. When a new content provider appears, the new content provider needs to learn from the high-quality content in the data source of other content providers. The high-quality content in the data source of the content provider needs to be shared with the new content provider, and the high-quality content in the data source of other content providers can become the target data.

In the embodiment of the present application, after the target data source is connected to the unified management platform, the unified management platform stores the target data in the target data source in the storage space corresponding to the target content provider, and the storage space corresponding to each content provider The data in are relatively independent.

S260. Determine whether the viewing authority of the target data source is a public authority, and if so, execute S270, otherwise, execute S2100.

S270. The unified management platform responds to data access requests from other content providers.

The data access request may be sent by other content providers than the target content provider to request access to the target data in the target data source of the target content provider.

In a specific example, the content provider of the entertainment category may send a data access request to the data source corresponding to the content provider of the cute pet game category.

S280: Determine whether the other content providers are other content providers with viewing rights, and if so, go to S290, otherwise, go to S2100.

In this embodiment of the present application, if other content providers are other content providers with viewing rights, the other content providers can access the target data source and call the target data.

S290. Provide the stored target data to the other content providers for access.

S2100. End.

In a specific example, when content provider A logs in to the unified management platform for data management, by sending a data access request to the unified management platform, the unified management platform can first, according to the identification information of the content provider A, retrieve the data from the storage space. , obtain the data stored corresponding to the identification information and provide it to the content provider A, that is, provide all the data obtained from the data source that the content provider A selects to access to the content provider A; in addition, the unified The management platform will also filter out the data stored corresponding to the identification information of other content providers, and filter out the shared data whose viewing rights are public rights and content provider A has the viewing rights, and provide the shared data to the unified management platform. Furthermore, the content provider A may perform data processing, processing or publishing based on the above two types of data provided by the unified management platform.

In the technical solution of this embodiment, by connecting the data sources of multiple content providers to the unified management platform through the access module, the data in the data sources are stored in the storage space corresponding to the content providers, and the data sources are stored in the storage space corresponding to the content providers. Set different viewing permissions, and determine whether the data corresponding to the data source can be accessed by other content providers according to the viewing permissions. It solves the problem that in the prior art, the data sources among the content providers are independent of each other, so that data cannot be shared among the data sources across the content providers. Data sharing between data sources across content providers is realized, the cost for content providers to acquire data in data sources corresponding to other content providers is reduced, and the work efficiency of content providers is improved.

Embodiment 3

FIG. 3 is a flowchart of a data processing method provided in Embodiment 3 of the present application. On the basis of the above-mentioned embodiment, this embodiment of the present application adds statistics on the read/write status of each data source accessed, and calculates the read/write state according to the read/write status. Status, go online and offline for each data source, or disconnect the data source.

Correspondingly, as shown in FIG. 3 , the technical solution of the embodiment of the present application specifically includes the following steps:

S310. The unified management platform acquires the access type and access parameters matching the data source access request in response to the data source access request of the target content provider, and the unified management platform connects multiple content providers in advance.

S320: The unified management platform obtains, from among multiple types of data source access modules, a target data source access module matching the access type.

S330: The unified management platform triggers the target data source access module according to the access parameter, and accesses the target data source matching the data source access request.

S340. The unified management platform acquires target data from the target data source, and stores the target data in a storage space matching the target content provider.

S350. The unified management platform counts the read/write status of each data source accessed.

The read/write status may be the number of times the data source has read data or written data within a certain period of time. The read and write status can reflect the usage status of the data source. In a period of time, the higher the number of reads and writes of the data source, the higher the activity or usage rate of the data source, and vice versa. Low.

S360. The unified management platform determines whether the current data source is in an online state, and if so, executes S370, otherwise, executes S390.

The online status may be that the storage status of the data in the data source is exposed storage, and the exposed storage may refer to the data in the data source being displayed to the user in the unified management platform. The offline status can be that the data storage status in the data source is hidden storage. Hidden storage can mean that the data in the data source is hidden in the unified management platform and cannot be obtained by users.

In this embodiment of the present application, the data source in the offline state cannot be accessed by the content provider, and the data source in the online state can be accessed by the content provider.

S370. The unified management platform judges whether the number of reads and writes of the first data source currently in the online state within the first time interval is less than the first threshold, and if so, execute S380, otherwise, execute S3140.

In the embodiment of the present application, if the first data source is currently in the online state, but the number of reads and writes in the first time interval is less than the first threshold, it can indicate that the activity or usage rate of the first data source is low, and the The state of the first data source is modified to an offline state to improve resource utilization.

The number of times of reading and writing may be the sum of the number of times of reading and writing, and the average or the maximum value of the two may be obtained, which is not limited in this embodiment. The number of writes of the data source reflects the number of new data appearing in the data source, and the number of reads of the data source reflects the number of times the data in the data source is used by other downstream modules. When the number of reads and writes is relatively low, it indicates that the activity and usage rate of a data source is low.

Typically, the unified management platform can monitor the read/write status of each data source in the platform in a set time unit (eg, month, week, day, or hour, etc.) through a preset offline task.

S380. Modify the storage state of the first data corresponding to the first data source from exposed storage to hidden storage, so as to perform offline processing on the first data source.

In this embodiment of the present application, if the activity or usage rate of the first data source is low, the first data storage state in the first data source may be modified from exposed storage to hidden storage, thereby changing the storage status of the first data source. The status is changed to offline status.

The advantage of this setting is that it realizes intelligent switching of the states of data sources with low usage rates, reduces waste of resources, and reduces maintenance costs of data sources.

S390. The unified management platform judges whether the number of times of reading and writing in the second time interval of the second data source that is currently offline is greater than or equal to the second threshold, and if so, execute S3100, otherwise, execute S3110.

In this embodiment of the present application, if the second data source is currently in the offline state, but the number of reads and writes in the second time interval is greater than or equal to the second threshold, it can indicate that the second data source has a high activity or usage rate, Its status can be modified to go online to improve the utilization of the data source.

S3100. The unified management platform modifies the storage state of the second data corresponding to the second data source from hidden storage to exposed storage, so as to perform online processing on the first data source.

In this embodiment of the present application, if the second data source is offline, but the activity or usage rate is high, the storage state of the second data in the second data source can be changed from hidden storage to exposed storage, so that The state of the second data source is modified to the online state.

The advantage of this setting is that it realizes intelligent switching of the status of the data source with low usage rate, and improves the utilization efficiency of the data source.

S3110: Determine whether the number of times of reading and writing in the third time interval of the third data source currently in the offline state is less than the third threshold, and if so, execute S3120, otherwise, execute S3140.

In this embodiment of the present application, if the third data source is currently in the offline state, but the number of reads and writes in the third time interval is less than the third threshold, it can indicate that the data accessed by the third data source in the third time interval Less, or accessed less frequently, the usage rate is low, and the storage space is occupied for a long time, which will cause waste of resources.

In this embodiment, the values of the first threshold, the second threshold and the third threshold may be the same or different, which is not limited in this embodiment. Optionally, the second threshold may be set to be greater than the first threshold , the first threshold is greater than the third threshold.

It should be noted that the first time interval, the second time interval, and the third time interval may be set to be the same or different, and the specific sizes of the first time interval, the second time interval, and the third time interval are different in this embodiment of the present application. limit.

S3120. Disconnect the access to the third data source.

In this embodiment of the present application, if the third data source is offline and the activity or usage rate is low, the third data source may be disconnected from the unified management platform, and the third data source may be corresponding to All data is deleted from the unified management platform.

The advantage of this setting is that the data sources that are not active, have not been used for a long time, or have low usage rates can be deactivated in time, which can avoid wasting resources and improve the utilization efficiency of the storage space of the unified management platform.

S3130: The unified management platform synchronously deletes all data corresponding to the third data source in the storage space.

S3140. End.

In the technical solution of this embodiment, by connecting the data sources of multiple content providers to the unified management platform through the access module, the data in the data sources is stored in the storage space corresponding to the content provider, and the access The read and write status of each data source is counted, and according to the read and write status, the status of the data source is modified, or the access to the data source is disconnected. It solves the problem of resource waste caused by data sources that have not been referenced for a long time or even empty data sources occupying storage space for a long time, and the problems of high cost and low efficiency of manual management of data sources in the prior art. It realizes the automatic management of the online and offline status of data sources, as well as the timely recovery of invalid data sources, which improves the management efficiency of data sources and saves the management cost of data sources.

It should be noted that, in addition to the unified management platform that can determine the read and write status of all data sources connected to the platform, and perform online or offline processing of data sources, content providers can independently choose to One or more specified data sources accessed are processed online or offline, and the processing strategy for online or offline processing is set accordingly, for example, the above-mentioned first time interval, second time interval, first threshold, and second threshold value, etc. The unified management platform conducts on-line or off-line processing of data sources that meet the requirements according to the data sources selected by the content provider to monitor and the corresponding processing strategies.

Embodiment 4

FIG. 4 is a schematic structural diagram of a data processing apparatus provided in Embodiment 4 of the present application. The apparatus includes: an access type and access parameter acquisition module 410, a target data source access module acquisition module 420, and a target data source access module 430. in:

The access type and access parameter acquisition module 410 is configured to make the unified management platform respond to the data source access request of the target content provider to acquire the access type and access parameters matching the data source access request, so The unified management platform is pre-connected to multiple content providers;

a target data source access module obtaining module 420, configured to enable the unified management platform to obtain a target data source access module matching the access type from among multiple types of data source access modules;

a target data source access module 430, configured to enable the unified management platform to trigger the target data source access module according to the access parameter to access the target data source matching the data source access request;

In the technical solutions of the embodiments of the present application, the data sources of multiple content providers are connected to the unified management platform through the access module. It solves the problem of high management and maintenance cost caused by independent data sources among various content providers in the prior art. It realizes the unified maintenance and management of different data sources by a unified management platform.

On the basis of the foregoing embodiment, the device further includes:

A target data storage module, configured to enable the unified management platform to acquire target data from the target data source, and store the target data in a storage space matched with the target content provider.

On the basis of the foregoing embodiment, the device further includes:

The viewing authority setting module is configured to enable the unified management platform to set the viewing authority of the target data source as a private authority or a public authority in response to the authority setting instruction of the target content provider;

Wherein, a data source set as private permission is only allowed to be accessed by content providers who have access to the data source, and a data source set as public permission is also allowed to be accessed by other content providers with viewing permission.

On the basis of the foregoing embodiment, the device further includes:

a data access request response module, configured to make the unified management platform respond to data access requests from other content providers;

A target data providing access module, configured to enable the unified management platform to provide the stored target data to the other content providers for access if it is determined that the other content providers are other content providers with viewing rights .

On the basis of the foregoing embodiment, the device further includes:

A read-write state statistics module, used to make the unified management platform count the read-write states of each connected data source;

a data source processing module, configured to enable the unified management platform to perform offline and/or online processing on each of the data sources according to the read and write status;

The data source in the offline state cannot be accessed by the content provider, and the data source in the online state can be accessed by the content provider.

On the basis of the above embodiment, the data source processing module includes:

The offline processing unit is used for the unified management platform, if it is determined that the number of reads and writes of the first data source currently in the online state in the first time interval is less than the first threshold, the first data source corresponding to the first data source will be processed. The storage state of the first data source is modified from exposed storage to hidden storage, so as to perform offline processing on the first data source;

The online processing unit is used for the unified management platform, if it is determined that the number of reads and writes of the second data source currently in the offline state in the second time interval is greater than or equal to the second threshold, the second data source corresponding to the second data source will be processed. The storage state of the data is changed from hidden storage to exposed storage, so as to perform online processing on the first data source.

On the basis of the foregoing embodiment, the device further includes:

The data source access disconnection module is used for the unified management platform to disconnect the third data source if it is determined that the number of times of reading and writing in the third time interval of the third data source that is currently offline is less than a third threshold. access to data sources;

A data deletion module is used for the unified management platform to synchronously delete all data corresponding to the third data source in the storage space.

On the basis of the foregoing embodiment, the access type includes at least one of the following:

Application program interface push-pull access, one-key mirroring access, file import access, simple information aggregation access and fifth-generation hypertext markup language page access.

On the basis of the above embodiment, the content provider is a provider of vertical services in the network site.

The data processing apparatus provided by the embodiment of the present application can execute the data processing method provided by any embodiment of the present application, and has functional modules and beneficial effects corresponding to the execution method.

Embodiment 5

FIG. 5 is a schematic structural diagram of an electronic device provided in Embodiment 5 of the present application. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are by way of example only, and are not intended to limit implementations of the application described and/or claimed herein.

As shown in FIG. 5, the electronic device includes: one or more processors 501, a memory 502, and interfaces for connecting various components, including a high-speed interface and a low-speed interface. The various components are interconnected using different buses and may be mounted on a common motherboard or otherwise as desired. The processor may process instructions executed within the electronic device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In other embodiments, multiple processors and/or multiple buses may be used with multiple memories and multiple memories, if desired. Likewise, multiple electronic devices may be connected, each providing some of the necessary operations (eg, as a server array, a group of blade servers, or a multiprocessor system). A processor 501 is taken as an example in FIG. 5 .

The memory 502 is the non-transitory computer-readable storage medium provided by the present application. Wherein, the memory stores instructions executable by at least one processor, so that the at least one processor executes the data processing method provided by the present application. The non-transitory computer-readable storage medium of the present application stores computer instructions for causing a computer to perform the data processing method provided by the present application.

As a non-transitory computer-readable storage medium, the memory 502 can be used to store non-transitory software programs, non-transitory computer-executable programs and modules, such as modules corresponding to the data processing methods in the embodiments of the present application (for example, as shown in FIG. 4 ). The shown access type and access parameter acquisition module 410, target data source access module acquisition module 420, and target data source access module 430). The processor 501 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions and modules stored in the memory 502, ie, implements the data processing methods in the above method embodiments.

The memory 502 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the electronic device for data processing, and the like. Additionally, memory 502 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 502 may optionally include memory located remotely from processor 501 that may be connected to the data processing electronics via a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The electronic device of the data processing method may further include: an input device 503 and an output device 504 . The processor 501 , the memory 502 , the input device 503 and the output device 504 may be connected by a bus or in other ways, and the connection by a bus is taken as an example in FIG. 5 .

Input device 503 may receive input numerical or character information, and generate key signal input related to user settings and functional control of data processing electronics, such as a touch screen, keypad, mouse, trackpad, touchpad, pointing stick, an or Multiple input devices such as mouse buttons, trackballs, joysticks, etc. Output devices 504 may include display devices, auxiliary lighting devices (eg, LEDs), haptic feedback devices (eg, vibration motors), and the like. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some implementations, the display device may be a touch screen.

Various implementations of the systems and techniques described herein can be implemented in digital electronic circuitry, integrated circuit systems, application specific ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpretable on a programmable system including at least one programmable processor that The processor, which may be a special purpose or general-purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device an output device.

These computational programs (also referred to as programs, software, software applications, or codes) include machine instructions for programmable processors, and may be implemented using high-level procedural and/or object-oriented programming languages, and/or assembly/machine languages calculation program. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and/or apparatus for providing machine instructions and/or data to a programmable processor ( For example, magnetic disks, optical disks, memories, programmable logic devices (PLDs), including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal used to provide machine instructions and/or data to a programmable processor.

To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (eg, visual feedback, auditory feedback, or tactile feedback); and can be in any form (including acoustic input, voice input, or tactile input) to receive input from the user.

The systems and techniques described herein may be implemented on a computing system that includes back-end components (eg, as a data server), or a computing system that includes middleware components (eg, an application server), or a computing system that includes front-end components (eg, a user's computer having a graphical user interface or web browser through which a user may interact with implementations of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include: Local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

A computer system can include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other.

It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, the steps described in the present application can be performed in parallel, sequentially or in different orders, and as long as the desired results of the technical solutions disclosed in the present application can be achieved, no limitation is imposed herein.

The above-mentioned specific embodiments do not constitute a limitation on the protection scope of the present application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of this application shall be included within the protection scope of this application.

Claims (12)

1. A method of data processing, comprising:

the method comprises the steps that a unified management platform responds to a data source access request of a target content provider, and obtains an access type and an access parameter which are matched with the data source access request, wherein the unified management platform is connected with a plurality of content providers in advance;

the unified management platform acquires a target data source access module matched with the access type from a plurality of types of data source access modules;

and the unified management platform triggers the target data source access module according to the access parameters to access the target data source matched with the data source access request.

2. The method according to claim 1, wherein after the unified management platform triggers the target data source access module to access the target data source matching the data source access request according to the access parameter, the method further comprises:

and the unified management platform acquires target data from the target data source and stores the target data in a storage space matched with the target content provider.

3. The method of claim 2, further comprising, after the unified management platform accesses the target data source matching the data source access request:

the unified management platform responds to the authority setting instruction of the target content provider and sets the viewing authority of the target data source as a private authority or a public authority;

the data source set to the private authority only allows the content provider accessing the data source to access, and the data source set to the public authority also allows other content providers with viewing authority to access.

4. The method as claimed in claim 3, wherein after the unified management platform responds to the permission setting instruction of the target content provider to set the viewing permission of the target data source to the public permission, the method further comprises:

the unified management platform responds to data access requests of other content providers;

and if the unified management platform determines that the other content providers are other content providers with viewing rights, the stored target data is provided for the other content providers to access.

5. The method of claim 1, further comprising:

the unified management platform counts the read-write state of each accessed data source;

the unified management platform carries out offline and/or online processing on each data source according to the read-write state;

the data source in the offline state cannot be accessed by the content provider, and the data source in the online state can be accessed by the content provider.

6. The method according to claim 5, wherein the unified management platform performs offline and/or online processing on each data source according to the read-write status, including:

if the unified management platform determines that the number of reading and writing times of a first data source which is currently in an online state in a first time interval is smaller than a first threshold, the unified management platform changes the storage state of first data corresponding to the first data source from exposed storage to hidden storage so as to perform offline processing on the first data source; and/or

And if the unified management platform determines that the number of reading and writing times of the second data source in the offline state in the second time interval is greater than or equal to a second threshold, the unified management platform changes the storage state of the second data corresponding to the second data source from hidden storage to exposed storage so as to perform online processing on the first data source.

7. The method according to claim 5, wherein after the unified management platform counts the read-write status of each accessed data source, the method further comprises:

if the unified management platform determines that the reading and writing times of a third data source currently in the offline state in a third time interval are smaller than a third threshold, the unified management platform disconnects the access to the third data source;

and the unified management platform synchronously deletes all data corresponding to the third data source in the storage space.

8. The method according to any of claims 1-7, wherein the access type comprises at least one of:

the system comprises an application program interface push-pull access mode, a one-key mirror access mode, a file import access mode, a simple information aggregation access mode and a fifth generation hypertext markup language page access mode.

9. The method according to any of claims 1-7, wherein the content provider is a provider of a set vertical service in a web site.

10. An apparatus for data processing, comprising:

the system comprises an access type and access parameter acquisition module, a data source access request acquisition module and a content source access request acquisition module, wherein the access type and access parameter acquisition module is used for enabling a unified management platform to respond to a data source access request of a target content provider and acquire an access type and an access parameter matched with the data source access request, and the unified management platform is connected with a plurality of content providers in advance;

the target data source access module acquisition module is used for enabling the unified management platform to acquire a target data source access module matched with the access type in a plurality of types of data source access modules;

and the target data source access module is used for triggering the target data source access module by the unified management platform according to the access parameters and accessing a target data source content provider matched with the data source access request.

11. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9.

12. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-9.

Images