TWI861875B - News public opinion analysis and collection system and method thereof - Google Patents

Abstract

A public opinion analysis and collection system includes: an extraction-conversion-loading server coupled to multiple data sources for collecting and standardizing multiple news and public opinion data; a database server coupled to the extraction-transformation-loading server a server for storing the standardized news and public opinion data; and a query device coupled to the database server for obtaining the required news and public opinion data from the standardized news and public opinion data by means of a criterion, wherein The criterion includes the condition option (A) as the denominator, and the setting condition (B) as the numerator, and the standardized news and public opinion data are analyzed by the criterion and the score (B)/(A) is the closest to 1, which is defined as “accurate information”.

Description

Public opinion analysis collection system and method

The present invention relates to a public opinion analysis and collection system and method thereof.

The analysis and management of public opinion has become an important part of corporate risk management. However, the previous way to collect news and public opinion was to arrange for a dedicated person to conduct an Internet search for a specific company and manually collect relevant news and public opinion for interpretation. This often takes a lot of manpower and cannot standardize the data collection process. In addition, the search results of different people are often different, which often leads to divergent results in subsequent judgments.

Therefore, developing a standardized news and public opinion collection and query system and method is what people in this field are considering.

An embodiment of the present invention provides a public opinion analysis collection system, which includes: an extraction-conversion-loading server, coupled to a plurality of data sources, for collecting and standardizing a plurality of news public opinion data; a database server, coupled to the extraction-conversion-loading server, for storing the standardized news public opinion data; and a query device, coupled to the database server, for querying the news public opinion data by A criterion condition obtains the required news and public opinion data from the standardized news and public opinion data, wherein the criterion condition includes the number of condition options (A) as the denominator and the number of conditions established (B) as the numerator. The standardized news and public opinion data is analyzed by the criterion condition and the score (B)/(A) is closest to 1 or equal to 1, which is defined as "accurate information".

In some embodiments, the standardized news and public opinion data are analyzed by the criteria to obtain a score (B)/(A) result sequence, which is defined as "substandard information".

In some embodiments, the condition options include keywords, data sources, information publishing time, or information collection time.

In some embodiments, if the content contained in one of the news and public opinion data appears more than 0 times after being filtered by the set condition of the criterion condition, the set condition is met, and the number of times the set condition is met is set to 1.

In some embodiments, the data sources include external websites, news homepage websites, regulatory agency announcement websites, or important information announcement websites.

In some embodiments, the database server periodically transmits corresponding news and public opinion data to the query device according to the criteria.

In some embodiments, the extract-transform-load server is used to collect and standardize the news and public opinion data from the data sources according to a tracking list using a crawler.

In some embodiments, the public opinion analysis and collection system further includes an upload device coupled to the extract-convert-load server for transmitting the tracking list to the extract-convert-load server.

In some embodiments, the extract-transform-load server further uses the crawler program to periodically collect and standardize the news and public opinion data from the data sources based on the tracking list and store them in the database server.

Another embodiment of the present invention provides a method for collecting public opinion analysis, comprising: using an extraction-conversion-loading server coupled to a plurality of data sources to collect and standardize a plurality of news public opinion data; using a database server coupled to the extraction-conversion-loading server to store the standardized news public opinion data; and using a query device coupled to the database server to query the data. The required news and public opinion data are obtained from the standardized news and public opinion data by using a criterion, wherein the criterion includes the number of conditional options (A) as the denominator and the number of conditions established (B) as the numerator. The standardized news and public opinion data are analyzed by the criterion and the score (B)/(A) is closest to 1 or equal to 1, which is defined as "accurate information".

According to the technical content of this case, the public opinion analysis collection system improves the current method of manually collecting news, press releases from competent authorities, and major information, and presents the data set in the form of subject tags for users to browse data, share data, or specify subscriptions. In order to provide users with more accurate information, the news and public opinion data of the database server can be filtered by criteria, and then the data can be divided into "accurate information" and "less accurate information" and provided to users according to their needs, saving users time in reviewing data.

The following will clearly illustrate the spirit of the present invention with diagrams and detailed descriptions. After understanding the embodiments of the present invention, any person with ordinary knowledge in the relevant technical field can make changes and modifications based on the techniques taught by the present invention without departing from the spirit and scope of the present invention.

The terms used herein are only for describing specific embodiments and are not intended to be limiting of the present invention. Singular forms such as "a", "this", "here", "this" and "the" as used herein also include plural forms.

As used herein, “coupled” or “connected” may refer to two or more elements or devices being in direct physical contact with each other, or being in indirect physical contact with each other, or may refer to two or more elements or devices operating or moving with each other.

The words "include", "including", "have", "contain", etc. used in this article are open terms, meaning including but not limited to.

As used herein, "and/or" includes any or all combinations of the items described.

The terms used in this document generally have the ordinary meanings of each term used in this field, in the context of this case and in the specific context, unless otherwise specified. Certain terms used to describe this case will be discussed below or elsewhere in this specification to provide additional guidance to those skilled in the art in describing this case.

In the traditional way of collecting news and public opinion, a dedicated person must be assigned to conduct an Internet search for a specific enterprise. Manually collecting relevant news and public opinion for interpretation often requires a lot of manpower and cannot standardize the data collection process. Therefore, this case provides a system mechanism to handle public opinion collection, replacing the manual search process and establishing a standardized query process.

Please refer to FIG. 1, which shows a functional flow chart 50 of a public opinion analysis and collection system according to some embodiments of the present invention. This system includes a data collection function 52, which searches for network information by system batches or by user-specified time by connecting to a data source 56. The data source 56 includes external websites, news homepage websites, competent authority announcement websites, major information announcement websites, and other designated websites. The data collection 52 still needs to include a public opinion analysis function 54, such as analyzing public opinion according to search conditions 55, or analyzing public opinion according to topic classification 57. After the data collection function 52, the system also obtains "accurate information" from the collected data by means of a data judgment function 60. The purpose of information criterion is to save users from searching and sorting out information scattered on various websites. Information that has been criterioned is the "accurate information" that users need. Data criterion 60 can, for example, utilize the function of "keyword verification of information relevance 62" and/or the function of "website source determination of data source 64". After data collection 52 and data criterion 60, the public opinion of this system is stored in the database 66 for users to "query or subscribe 68".

Please refer to FIG. 2, which shows a functional block diagram of a public opinion analysis collection system according to some embodiments of the present invention. The public opinion analysis collection system 100 includes an extract-transform-load (ETL) server 110 (hereinafter referred to as ETL server 110), a database server 120, a query device 130, and an upload device 140. In some embodiments, the ETL server 110 is coupled to the database server 120. The ETL server 110 is a server for data integration. In the process of data integration, data is collected from different data sources or news networks 200, 201...20n, standardized, and then loaded into the database server 120 for analysis and storage. In some embodiments, the ETL server 110 is coupled to a plurality of different data sources or news websites 200, 201...20n, and is used to collect corresponding news and public opinion data from different data sources or news websites 200, 201...20n and store them in the database server 120 according to the tracking list transmitted by the user 150 through the upload device 140. In addition, the ETL server 110 also regularly updates the news and public opinion data stored in the database server 120 according to the latest data in the news websites 200, 201...20n and the tracking list. Thus, the database server 120 can retrieve corresponding news and public opinion data from the database server 120 according to the query list sent by the user 150 through the query device 130 and return it to the query device 130 for display, and present the data set in the form of subject tags for users to browse data, share data or specify subscription.

In some embodiments, the ETL server 110 can convert the data sources or news and public opinion data in the news websites 200, 201 ... 20n distributed in multiple systems and using different languages into a unified format and type, so as to make it easier to analyze them. Among them, the different news websites 200, 201 ... 20n can be external websites, news homepage websites, competent authority announcement websites, major information announcement websites and other designated websites, etc. The external website can include a company financial report website, a joint recruitment information website, a public information website and a news website, so as to provide the company's financial report, joint recruitment information, public information and news and public opinion, etc. In some embodiments, the ETL server 110 extracts data from the news websites 200, 201 ... 20n according to the tracking list transmitted by the upload device 140. In some embodiments, the ETL server 110 crawls the website data in the news network 200, 201...20n by a web crawler. Since the data from different news networks 200, 201...20n may have different structures and characteristics, the ETL server 110 will transform the extracted data to standardize these different data, convert these data into a common format, and then clean the data to eliminate noise in the data, repair missing values and inconsistencies, and delete duplicate data. Then, the data is loaded, wherein the transformed data will be loaded into the database server 120 for analysis and storage. In this way, the user 150 can input the criteria through the query device 130 to extract the required data from the database server 120. In some preferred embodiments, the ETL server 110 and the database server 120 have necessary components such as a processor unit, a communication unit, and a storage unit for running the ETL server 110 and the database server 120.

In some embodiments, the query device 130 communicates with the database server 120 via an Internet. The query device 130 may be a terminal device of the user 150, and the terminal device may be a personal computer, a laptop, a smart phone, and a tablet computer, but is not limited to the above. In addition, the present case does not limit the user 150 to have only a single query device 130, and the user 150 may have multiple query devices 130 at the same time. In some embodiments, a query program 131 is pre-installed in the query device 130. The query program 131 is issued by the public opinion analysis and collection system 100 and is used to provide a user interface for the user 150 to perform a query service of the database server 120 and display the query results. The query program 131 can be run by the public opinion analysis and collection system 100. Thus, the user 150 can input a specific customer to be queried or a customer list to be queried through the user interface of the query program 131 of the query device 130, and the database server 120 can search the database server 120 according to the specific customer or customer list, and display the search results on the user interface of the query program 131. In some embodiments, the database server 120 has a search program 121 for searching in the database server 120 according to a specific customer or customer list input by the query program 131, and returning the search results to the user interface of the query program 131 for display. In some embodiments, because the ETL server 110 can regularly update the news and public opinion data stored in the database server 120 according to the latest data in the news network 200, 201...20n. Therefore, when the news and public opinion data corresponding to the specific customer or customer list stored in the database server 120 is updated, the search program 121 synchronously transmits the latest data to the query device 130 for display on the user interface of the query program 131. This ensures that the news and public opinion data queried by the user 150 are the latest data. In some embodiments, the query device 130 has necessary components for running the query device 130, such as a processing unit, a communication unit, and a storage unit.

In some embodiments, the upload device 140 is coupled to the ETL server 110 to transmit the tracking list imported by the user 150 to the ETL server 110. Thus, the ETL server 110 can crawl the website data related to the tracking list in the news websites 200, 201...20n using a web crawler according to the tracking list, such as news and public opinion data, and store them in the database server 120. In some embodiments, the user 150 can import the tracking list of the customers to be tracked into the upload device 140, and transmit it to the server 110 through the upload device 140. In some embodiments, the upload device 140 is a signal converter for converting the tracking list imported by the customer into a signal readable by the ETL server 110 .

In some embodiments, the query device 130 is coupled to the database server 120 to obtain the required news and public opinion data from the standardized news and public opinion data by a criterion. The criterion includes the number of condition options (A) as the denominator, and the number of set conditions satisfied (B) as the numerator. If the number of occurrences of the content included in one of the news and public opinion data after filtering by the set condition of the criterion is greater than 0, the set condition is satisfied, and the number of set conditions satisfied is set to 1. The standardized news and public opinion data analyzed by the criterion and the score (B)/(A) result is closest to 1 or equal to 1 is defined as "accurate information" and can be provided to users as "priority disclosure information". The standardized news and public opinion data are analyzed by the criteria and the scores (B)/(A) are in the order (i.e., the (B)/(A) results are less close to 1), which is defined as "sub-quasi information". The user can choose whether to disclose "sub-quasi information" through system inquiry or setting to achieve the purpose of information criterion and analysis function. In some embodiments, the condition options can be, for example, keywords (multiple keywords are regarded as individual conditions, which can be entered into the fields separately or defined by symbols +, -, &, (), data source, information publishing time (the time when the information appears on each website, which can be defined as a period or before...) or information collection time (the time when the system uploads to the database). In some embodiments, the user knows that Bank A will be fined by the competent authority in the near future. In order to prevent Bank A from delaying the release of important information and being fined by the competent authority, the user searches for "Bank A" (condition 1) + "fine" (condition 2) through the query device 130 query program 131 in the system at 16:54 on February 7, 2013, and selects "Information source: Financial Supervisory Commission website" (condition 3) and "Information publication time: February 7, 2013 15:00 (before)" (condition 4). The query device 130 searches from the database server 120, analyzes and aggregates 23 data from the Financial Supervisory Commission website, and a total of 45 data that match "bank" + "fine", and 13 of which match the information publication time. There are 2 items that meet the above-mentioned conditions 1 to 4 at the same time (because the score result (the number of conditions established (B)/the number of condition options (A)) = 4/4 is 1) defined as "accurate information" and actively disclosed to users. The remaining scores less than 1 are "sub-accurate information" and are not actively disclosed to users. Through the above system analysis, users learned about the penalty information of the Financial Supervisory Commission on Bank A, and found that the penalty information of Bank A appeared on the "Major Penalties" page and "Press Release" page of the Financial Supervisory Commission's website. Users quickly reported to the head office of Bank A and disclosed this important information to the company's shareholders immediately. The user is willing to check the "sub-accurate information" of the query, and finds that the information about the penalty of Bank A appeared on the Y website and the G website at 16:00 on February 7, 2013, and other news and netizens' discussions were extended. In order to avoid unnecessary trouble, the user can provide the above information to the public opinion control team of the head office, and then issue a statement to reduce the impact on the reputation of Bank A. In some embodiments, the user is concerned about this matter and subscribes to the above conditions, but enters the information publishing time as after 16:00 on February 7, 2013, and uses the daily condition to batch search and aggregate E-mails from the database server 120 to the user's company mailbox or query device 130. The user found out from the daily report that the public opinion died down after 5 days and informed the supervisor so that the case could be closed.

Please refer to FIG. 3, which shows a method flow chart of a public opinion analysis collection system according to an embodiment of the present invention. Please refer to FIG. 2 and FIG. 3 at the same time. The public opinion analysis collection method 300, in step 310, crawls content through a crawler. In some embodiments, the ETL server 110 is coupled to a plurality of different news websites 200, 201 ... 20n, and is used to collect corresponding news public opinion data from different data sources or news websites 200, 201 ... 20n according to the tracking list sent by the user 150 through the upload device 140, and store the website data related to the tracking list in the database server 120, such as news web page content. In step 320, the converted crawled content is stored in the database. In some embodiments, since the data from different news networks 200, 201...20n have different structures and characteristics, the ETL server 110 will convert the crawled content to standardize these different crawled contents, convert these crawled contents into a common format, and then perform data cleaning to eliminate noise in the data, repair missing values and inconsistencies, and delete duplicate data. Then, the sorted data is stored, wherein the converted data will be loaded into the database server 120 for analysis and storage. In step 330, the judgment criteria are imported. In some embodiments, a query device 130 communicates with the database server 120 via an Internet. A query program 131 is pre-installed in the query device 130. The query program 131 is issued by the public opinion analysis and collection system 100 and is used to provide a user interface for the user 150 to perform a query service of the database server 120 and display the query results. The user 150 can enter the criteria through the user interface of the query program 131 of the query device 130, such as a specific customer to be queried or a customer list to be queried or the quantity (A) of the above condition options as the denominator and the quantity (B) of the condition being satisfied as the numerator. In some embodiments, the query is performed using a specific customer's uniform number or a customer list uniform number to be queried. In step 340, the query result is displayed. In some embodiments, the database server 120 can search the database server 120 according to the specific customer or customer list, and display the search result on the user interface of the query program 131. In some embodiments, the database server 120 can aggregate the search result into the user-required format and then display the search result on the user interface of the query program 131. In some embodiments, the search results display "accurate information (results with scores (B)/(A) closest to 1 or equal to 1)" and "less accurate information (results with scores (B)/(A) at the lowest level)" that meet the multiple conditions set above.

Accordingly, the public opinion analysis and collection system of this case improves the current method of manually collecting news, press releases of competent authorities, and major information, and presents the data set in the form of subject tags for users to browse data, share data, or specify subscriptions. In addition, in order to provide users with more accurate information, the news and public opinion data of the database server can be filtered by criteria, and then the data can be divided into "accurate information" and "less accurate information" and provided to users according to their needs, saving users time in reviewing data.

Although the present invention is disclosed as above by way of embodiments, it is not intended to limit the present invention. Anyone skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of the patent application attached hereto.

50: Functional flow chart 52: Data collection 54: Analysis function 55: Based on search conditions 56: Data source 57: Classification by topic 60: Data judgment 62: Keyword verification of information relevance 64: Website source to determine the source of data 66: Database 68: Query or subscription 100: Public opinion analysis collection system 110: Extract-Transform-Load (ETL) server 120: Database server 121: Search program 130: Query device 131: Query program 140: Upload device 150: User 200, 201…20n: News network 300: Public Opinion Analysis Collection Method 310: Steps 320: Steps 330: Steps 340: Steps

The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present invention and are used together with the specification to illustrate the technical solutions of the embodiments of the present invention. Figure 1 shows a functional flow chart of a public opinion analysis and collection system according to some embodiments of the present invention. Figure 2 shows a functional block diagram of a public opinion analysis and collection system according to some embodiments of the present invention. Figure 3 shows a method flow chart of a public opinion analysis and collection system according to an embodiment of the present invention.

100: Public Opinion Analysis Collection System

110: Extract-Transform-Load (ETL) Server

120: Database server

121:Search program

130: Query device

131: Query program

140: Upload device

150:User

200, 201...20n: News Network

Claims (9)

A public opinion analysis and collection system includes: an extraction-conversion-loading server coupled to a plurality of data sources for collecting and standardizing a plurality of news public opinion data; a database server coupled to the extraction-conversion-loading server for storing the standardized news public opinion data; and a query device coupled to the database server for obtaining the required news public opinion data from the standardized news public opinion data by a criterion, wherein The criteria include the number of conditional options (A) as the denominator, and the number of conditions that are satisfied (B) as the numerator. The standardized news and public opinion data analyzed by the criteria and the score (B)/(A) closest to 1 or equal to 1 are defined as "accurate information", and the standardized news and public opinion data analyzed by the criteria and the score (B)/(A) are defined as "inaccurate information". The public opinion analysis collection system as described in claim 1, wherein the condition options include keywords, data sources, information publishing time or information collection time. As described in claim 1, in the public opinion analysis collection system, if the number of times the content contained in one of the news public opinion data is filtered by the set condition of the criterion condition is greater than 0, the set condition is satisfied, and the number of times the set condition is satisfied is set to 1. The public opinion analysis collection system as described in Request Item 1, wherein the data sources include external websites, news homepage websites, competent authority announcement websites or major information announcement websites. The public opinion analysis and collection system as described in claim 1, wherein the database server periodically transmits corresponding news public opinion data to the query device according to the criteria. The public opinion analysis collection system as described in claim 1, wherein the extract-transform-load server is used to collect and standardize the news public opinion data from the data sources using a crawler according to a tracking list. The public opinion analysis and collection system as described in claim 6 further includes an upload device coupled to the extraction-conversion-loading server for transmitting the tracking list to the extraction-conversion-loading server. The public opinion analysis and collection system as described in claim 6, wherein the extract-transform-load server further uses the crawler program to periodically collect and standardize the news public opinion data from the data sources based on the tracking list and store them in the database server. A method for collecting public opinion analysis includes: coupling a plurality of data sources by an extraction-conversion-loading server to collect and standardize a plurality of news public opinion data; coupling a database server to the extraction-conversion-loading server to store the standardized news public opinion data; and coupling a query device to the database server to obtain the required news public opinion data from the standardized news public opinion data by a criterion. The criterion includes the number of conditional options (A) as the denominator, and the number of conditions that are satisfied (B) as the numerator. The standardized news and public opinion data analyzed by the criterion and the score (B)/(A) closest to 1 or equal to 1 is defined as "accurate information", and the standardized news and public opinion data analyzed by the criterion and the score (B)/(A) is the lowest, which is defined as "sub-accurate information".

Images