CN107818415B - A general recognition method based on subway card swiping data - Google Patents

Abstract

The invention discloses a general recognition method based on subway card swiping data, comprising the following steps: 1) based on subway card swiping information, collecting data including IC card number, station and card swiping time and preprocessing; 2) searching for each card number The station with the highest frequency of use and the corresponding station with the highest frequency of use are selected as the candidate stations for the home station and the school station; 3) According to the schedule of work and rest of primary and secondary schools in the city, the distance between the two candidate stations of each card number is calculated. The travel records are classified according to the entry time, and the home station and school station are determined according to certain time rules; 4) Find and delete non-school travel records and difficult-to-determine card numbers and their records. From the perspective of time and space, the present invention integrates and processes a large amount of subway card swiping data, and proposes for the first time a method for identifying general students based on subway card swiping data, solves the basic problem of using big data to study general behavior, and makes up for the deficiencies of traditional survey methods. .

Description

A general recognition method based on subway card swiping data

technical field

The invention relates to a method for collecting and analyzing travel behavior data in traffic planning, in particular to a general recognition method based on subway card swiping data.

Background technique

In recent years, the choice of students' travel mode has attracted more and more scholars' attention. In China, students in primary and junior high schools should be admitted according to the nearest admission policy. However, due to the uneven distribution of educational resources, more and more parents will obtain high-quality educational resources for their children through school choice, which usually leads to a long distance between schools. As the main medium and long-distance travel mode in the city, the subway will be chosen by them. However, these students who use the subway are still driven to school by their parents on a daily basis. In order to guide these students to use the subway more, it is necessary to better understand their daily commuting patterns.

The commuting behavior is the process of going from home to school corresponding to the commuting behavior. At present, the traditional questionnaire survey method is used to obtain the information of commuting to school for analysis. The traditional questionnaire survey method consumes a lot of manpower and time, and the survey sample size is small and the sample The coverage is small and incomplete, resulting in biased analysis results or limited local area analysis. Moreover, learning behavior is a long-term process, and it is difficult to obtain complete data through short-term traditional surveys. Fortunately, subway smart card systems designed for toll collection can provide detailed card swipe records, including card types that can be used for many purposes, card swipe dates, and subway stations, which can better replace the data obtained by many traditional survey methods. However, although the research methods of commuting recognition based on smart card data are becoming more and more mature, there are very few studies on the recognition of school commuting behavior. Conventional commuting identification methods focus on temporal and spatial patterns, and identify the commuting trips of people based on the frequency of commuting within a week, fixed alighting stations, and the time interval between two commuting trips. In the identification method of going to school behavior, because it is very common for parents to drive students to and from school, the number of primary and secondary school students who use the subway for a long time is not very large. If only relying on the frequency of travel will mistakenly exclude a large number of real school-going people. ; At the same time, due to the school's lunch break system, the time spent in the same place is not a good identification method. Therefore, strict and reasonable time and space constraints must be used to accurately judge whether a trip within a working day is a school-bound behavior.

SUMMARY OF THE INVENTION

Purpose of the invention: Based on the above deficiencies, the present invention provides a method for identifying people who use the subway to go to school and travel records based on subway card swiping data, which can accurately judge the behavior of going to school.

Technical solution: A general school identification method based on subway card swiping data according to the present invention includes the following steps:

(1) Data collection and preprocessing: IC data and subway station coordinate data of all subway stations in a city for three consecutive weeks or more are required. IC data includes card number, entry date, entry time, exit date, exit time, entry station number, exit station number, and card type. After collecting subway IC card data, all travel records of consecutive working days are merged in chronological order by the cardholder, and only all card swiping records whose card type is student card are screened and retained, and abnormal data is deleted.

(2) Calculate the frequency of occurrence of the station used by the card number for each card number, find out the station with the highest frequency of occurrence, and count the number of stations with the highest frequency of occurrence.

(3) If the number of stations with the highest frequency is 1, the station will be regarded as the candidate station Si1 of the home station or school station of the card number; if the number of stations with the highest frequency is 2, the two stations will be regarded as the The candidate stations Si1 and Si2 of the home station or school station of the card number; if the number of stations with the highest frequency is greater than 2, the adjacent stations are merged, and then the station with the highest frequency is used as the candidate station Si1 of the home station or school station. If there are 2 stations with the highest frequency, the combined two stations will be used as the candidate stations Si1 and Si2 of the home station and school station of the card number. If the number of parallels after the combination is greater than 2, this part of the card number and its trip will be deleted. Record.

(4) For the number of stations with the highest frequency of occurrence in step 3, 1 and greater than 2, calculate and judge the number of stations with the highest frequency of occurrence among the stations corresponding to the selected candidate stations: if the station with the highest frequency of occurrence If the number is 1, the station will be used as the home station of the card number or another candidate station Si2 of the school station; if the number of stations with the highest frequency is greater than or equal to 2, the adjacent stations will be merged and then the station with the highest frequency will be used as the home station. Another candidate station Si2 of the station or school station, if there is still a tie after the merger, this part of the card number and its travel record will be deleted.

(5) According to the schedule of work and rest in primary and secondary schools, the travel records between the candidate stations of each card number are divided into four categories according to the entry time: (I) am: the entry time is before the last school time in the morning; (II) pm: The pit stop time is after the earliest school time in the afternoon; (III) noon1: The pit stop time is within the range from the earliest school time in the morning to the earliest school time in the afternoon; (IV) noon2: The pit stop time is between the earliest school time in the afternoon and the earliest school time in the afternoon. within the evening school hours.

(6), sort the four categories according to the card number and the entry time respectively; for the card number with the trip record belonging to category (I), the entry station number of the first trip record is determined as the home station, and the corresponding exit station The station number is the school station; for the card number with a trip record belonging to category (II), the outbound station number of the last trip record is determined as the home station, and the corresponding inbound station number is the school station; for the card number with a trip record belonging to (III) ) class card number, the inbound station number of the first trip record is determined as the school station, and the corresponding outbound station number is the home station; for the card number with the trip record belonging to category (IV), the outbound station number of the last trip record is determined as the home station. The station station number is determined as the school station, and the corresponding entry station number is the home station.

(7) Delete the records of departures from the school station before the last school time in the morning and the records of departures from the home station after the latest school time in the afternoon from all travel records identified with the card numbers of the home station and the school station.

(8) Count the number of school days for each card number, and delete the card numbers and their records whose school days are less than the specified threshold days.

Beneficial effect: Compared with the prior art, the present invention has the following advantages:

The subway card swiping data adopted by the present invention is easy to obtain, the data is comprehensive and objective, and it is easier to reveal its own laws by using big data. Although the identification of subway commuting behaviors is relatively mature, students' commuting behaviors cannot be identified by traditional commuting identification methods because students' travel patterns are affected by their parents' travel patterns, resulting in diverse modes of daily subway use. Therefore, the present invention proposes a method for recognizing passing behavior based on subway card swiping data in consideration of the possible pick-up and drop-off behavior of students, and in combination with the school's school-to-school and school-discharge time regulations. Compared with the existing commuting identification method, the time and space constraints of the present invention are more strict and reasonable, and the accuracy of the commuting identification result is improved.

Description of drawings

FIG. 1 is a flow chart of the method of the present invention.

Detailed ways

The technical solutions of the present invention will be further described below with reference to the accompanying drawings.

As shown in Figure 1, the method for identifying general school behavior based on subway card swiping data includes three stages: 1. Data collection and preprocessing, corresponding to step 1 in Figure 1; 2. Identifying home and school stations for each card number, corresponding to Figure 1 Step 2-Step 6 in 1; 3. Delete abnormal travel records and card numbers with too few school days and their travel records, corresponding to steps 7-8 in Figure 1, and the specific process is described in detail below.

1. Data collection and preprocessing

According to the present invention, IC card data and subway station coordinate data of all subway stations in all working days for three consecutive weeks or more in a city are required. The original data in this embodiment is the credit card data of all subway stations in Nanjing from October 10 to October 28, 2016. In step 1, first save the original data in the database in CSV format for reading by R software, extract the names of "card number", "inbound date", "inbound time", "outbound date", "outbound date" Station Time", "Inbound Station Number", "Outbound Station Number", "Card Type", "Inbound Station Longitude", "Inbound Station Latitude", "Outbound Station Longitude", "Outbound Station Latitude" , and then preprocess the original data to keep only the records whose card type is student card and whose entry date belongs to working days. In the present invention, the card type is 54, which means the student card, delete the records with different "inbound date" and "outbound date" and the same record with "inbound station number" and "outbound station number", and obtain the original travel record data X . The specific data format is shown in Table 1.

Table 1 Example of subway card swiping data

Note: The actual application of coordinate data is to retain 9 decimal places

2. Identify home and school stations for each card number

First, find the candidate stations Si1 and Si2 of the home station and school station for each card number i, and then classify the travel records between the two candidate stations of each card number according to the entry time according to the schedule of primary and secondary schools in the city. , and then determine the home station and school station according to certain time rules. The home station represents the location of the student's home, and the school station represents the location of the student's school.

In step 2, the usage frequency of all stations of the card number is calculated for each card number, the station with the highest frequency of occurrence is found, and the number of stations with the highest frequency of occurrence of each card number is determined. Specifically, sort the original travel record data X according to the card number, extract the three columns of card number, inbound station number, and outbound station number in the original data X and store it as database O; copy the data in database O to a new database N, and rename it The columns in the new database N are card number, outbound station number, and inbound station number; databases O and N are merged by row, that is, the inbound station number in database O and the outbound station number in database N are merged into one column, which is called is station number A, the outbound station number in database O and the inbound and outbound station number in database N are combined into one column, called station number B, to form database M, whose data form is shown in Table 2; The data are sorted according to the card number branch, use the circular algorithm to count the frequency of station number A or station number B of each card number in the database M, and define new columns Si1 and Si2 for all card numbers in the database M, and the value is recorded as 0 ( Nanjing subway station does not include the station numbered 0), recorded in the database P, and its data form is shown in Table 3.

Table 2 Examples of data in database M

Table 3 Examples of data in database P

card number station number frequency Si1 Si2 9726xxxxxx78 2 1 0 0 9726xxxxxx78 18 7 0 0 9726xxxxxx78 twenty one 2 0 0 9726xxxxxx78 19 4 0 0

In step 3, the candidate stations of the home station and the school station are searched for the first time. Find the station number with the highest frequency of each card number in the database P, and use the loop statement to calculate the number of station numbers with the highest frequency of each card number and store it in the new column n. For example, in Table 3, among the stations used by the card number 9726xxxxxx78, the station with the highest frequency is only station 18, and the number of station numbers with the highest frequency is 1. Next, find candidate stations for home or school stations based on the number of stations with the highest frequency. The following judgment is made: if the number of stations is 1, the station is the candidate station Si1 of the card number; if the number of stations is 2, the two stations are the candidate stations Si1 and Si2 of the card number; if the number of stations is greater than 2, then merge After adjacent stations, let the station with the highest frequency be Si1. If the number of stations with the highest frequency is still greater than 2 after merging, delete the card number and the corresponding data.

Specifically, if n is 1, make Si1 of this part of the card number equal to the station number with the highest frequency of occurrence, store all the data of this part of the card number including the column "Si1" as nrep and merge it with X according to the card number by column (the final nrep Should contain all the columns in X and the original nrep of all card numbers in the original nrep (ie, the nrep before merging). As shown in Table 4, the station number with the highest frequency is 1, the station number is 18, and Si1=18.

Table 4 Examples of data in database nrep

Note: For the convenience of description, the latitude and longitude data of the same site as X in Table 4 are not shown

If n is 2, set Si1 and Si2 of this part of the card number to be equal to the two station numbers respectively (no sequential correspondence), and store all the data of this part of the card number including the columns "Si1" and "Si2" in rep. As shown in Table 5, there are two stations with the highest frequency, and the two station numbers are randomly assigned to Si1 and Si2 corresponding to the card numbers. In this embodiment, Si1=9 and Si2=73.

Table 5 Examples of data in database rep

card number station number frequency Si1 Si2 9726xxxxxx52 9 8 9 73 9726xxxxxx52 73 8 9 73 9726xxxxxx52 twenty three 1 9 73 9726xxxxxx52 twenty four 1 9 73

Note: For the convenience of description, the latitude and longitude data of the same site as X in Table 5 are not shown

并存储于新列d12。If n is greater than 2, as shown in Table 6, the station numbers 14, 15, and 23 all appear three times, then use the station latitude and longitude information to calculate the Euclidean distance between these stations by the following formula.

and stored in a new column d12.

Table 6 Data example of n greater than 2

card number station number frequency site longitude site latitude 9961xxxxxx61 14 3 118.79168700 32.08972168 9961xxxxxx61 15 3 118.79687500 32.09790039 9961xxxxxx61 twenty three 3 118.75372310 32.03948975 9961xxxxxx61 9 1 118.77893070 32.04327393

in,

Among them, Long1 is the inbound coordinate longitude, Lat1 is the inbound coordinate latitude, Long2 is the outbound coordinate longitude, and Lat2 is the outbound coordinate latitude.

If _d12 is less than or equal to the specified threshold, the two stations are merged. In the present invention, the threshold is set to 1km: take one of the stations, replace the frequency of the station with the sum of the frequencies of the two stations, and then set Si1 to be equal to the highest frequency. Station numbers, as in the above example, station 14 and station 15 can be combined, and the combined result is shown in Table 7, Si1=14.

Table 7 Example of data after n is greater than 2

card number station number frequency Si1 9961xxxxxx61 14 6 14 9961xxxxxx61 twenty three 3 14 9961xxxxxx61 9 1 14

If there is still a tie after the merger, let Si1 and Si2 be equal to the station number with the highest tie frequency after the merger; if the tie number after the merger is greater than 2, it can be considered that the travel OD points are too scattered to judge, delete this part of the card number and his travel records. Store all data of the processed card number in the database P including the column "Si1" in nrep1 and merge it with X according to the card number by column, and then combine nrep1 by row and store it in nrep.

After the above processing, for some card numbers, both the candidate station and the candidate school station (ie Si1 and Si2) have been obtained, but for some card numbers, only one of the candidate station and the candidate school station (ie Si1) has been obtained, Another candidate station needs to be found. That is, step 4 described below, to find candidate stations for the second time. For the case where only one candidate station Si1 is found in step 3, the frequency of occurrence of the station corresponding to the candidate station Si1 is calculated. The corresponding station in the present invention refers to: in the travel records of each card number, some have the station shown in Si1 ( Indicates that the trip involves the station indicated by Si1), some do not (indicating that the trip does not involve the station indicated by Si1), here only the travel records of Si1 are selected, and the remaining stations except Si1 are extracted. The station is the station corresponding to Si1. Find another candidate station Si2 for home or school according to the number of corresponding stations with the highest occurrence frequency, and perform the following judgment: if the number of stations is 1, the station is another candidate station Si2 for the card number; if the number of stations is greater than or equal to 2. After the adjacent stations are merged, the station with the highest frequency will be Si2. If there is still a tie after the merger, the card number and the corresponding record will be deleted.

并存储于新列d₁₂'，若d₁₂'小于或等于1km则合并这两个站点，删除合并后仍有并列频次最高的车站的卡号及其出行记录。将M2中这部分处理过的卡号包含列“Si2”的所有数据存储于nrep'并与X按照卡号按列合并，然后将nrep'按行合并存储于nrep。Specifically, in the data nrep, the inbound station number or the outbound station number containing the candidate station Si1 is screened out for each card number in the data nrep and stored as O2. Similar to step 2, the inbound station of each card number in O2 is stored as O2. Number and exit station number are combined and stored as M2, sort M2 according to the card number, delete all the card numbers whose entry station number is Si1, and count the frequency of the remaining inbound station numbers for each card number in M2. Find the station number with the highest frequency of each card number in the data M2, use a loop statement to calculate the number of station numbers with the highest frequency of each card number and store it in a new column n', and then use a conditional statement to judge. If n' is 1, set Si2 of this part of the card numbers to be equal to the station number with the highest frequency. If n' is greater than 2 or equal to 2, similar to step 3, calculate the Euclidean distance between these card numbers by using the formula shown in step 3.

And stored in a new column d ₁₂ ', if d ₁₂ ' is less than or equal to 1km, merge the two stations, delete the card number and travel record of the station with the highest parallel frequency after the merger. Store all data of the processed card number in M2 including the column "Si2" in nrep' and merge it with X according to the card number by column, and then combine nrep' by row and store it in nrep.

The rep and nrep are combined and stored in the database Q by row. Delete the trip records in the database Q that contain station numbers other than Si1 and Si2 in the inbound station number or outbound station number, and obtain a database R of all school trip records that only travel between candidate stations.

The next thing to do is to judge whether the candidate stations Si1 and Si2 of each card number i in the database R are home stations or school stations.

In step 5, the travel records between the two candidate stations of all card numbers are selected and sorted according to the entry time, and the category to which each travel record of each card number belongs is determined in order according to the entry time. According to the schedule of work and rest time of primary and secondary schools in Nanjing (see Table 8), the travel records between the two candidate stations Si1 and Si2 of each card number i are divided into 4 categories according to the entry time: (I)am: The entry time is at Before the last school time in the morning; (II) pm: The entry time is after the earliest school dismissal time in the afternoon; (III) noon1: The entry time is within the morning dismissal time range, that is, within the range from the earliest morning dismissal time to the earliest afternoon school entry time ; (IV)noon2: The entry time is within the time range of the school in the afternoon, that is, within the range from the earliest start time in the afternoon to the earliest release time in the afternoon. In order to allow the error, the range can be appropriately relaxed. In this embodiment, it is set as (I)am: the time of entering the station is before 9:00; (II) pm: the time of entering the station is after 14:00; (III) noon1: the time of entering the station Time between 11:30 and 13:00; (IV)noon2: Pit time between 13:00 and 14:00.

Table 8 Nanjing primary and secondary school start and end time

In step 6, the home station and the school station are determined according to the time condition. The specific operation process is as follows: (1) assign 0 to the new column "home station" and column "school station" in database R; (2) filter out the trip records from database R whose entry time is earlier than or equal to 9:00. For am, sort am according to "card number" and "entry time" in turn, select the travel record with the earliest entry time for each card number i, define its entry station number as home station and store it in the column "home station", then The exit station number in this record is the school station and is stored in the column "School Station"; (3) Filter out the travel records whose entry time is later than or equal to 14:00 from the database R and store them in pm, delete the records with For the card numbers and travel records with the same card number in am, sort pm according to "card number" and "entry time" in turn, select the travel record with the latest entry time for each card number i, and define its exit station number as the home station Stored in the column "Home Station", the entry station number in this record is the school station and stored in the column "School Station"; (4) Filter out the entry time from the database R between 11:30 and 13:00 The travel records in between are stored in noon1, delete the card number and its travel record in noon1 with the same card number as the card number in am and pm, sort noon1 according to "card number" and "entry time" in turn, and select the earliest entry time for each card number i The travel record of , defines its inbound station number as the school station and stored in the column "school station", then the outbound station number in this record is the home station and stored in the column "home station"; (5) filter from the database R The travel records whose entry and exit time is between 13:00 and 14:00 are stored in noon2. Delete the card numbers and travel records in noon2 that are the same as the card numbers in am, noon1, and pm. Sort by “station time”, select the travel record with the earliest entry time of each card number i, define its entry station number as home station and store it in the column “home station”, then the exit station number in this record is stored at the school station In the column "school station"; (6) merge am, pm, noon1, noon2 by row and store in database R1. Delete the columns of "home station" and "school station" in database R, and combine R and R1 by card number and store them in database S. So far, data S containing the home station and school station of each general student is obtained. In this step of processing, only the card numbers and their records with travel from 9:00 to 11:30 are eliminated.

3. Delete abnormal travel records and card numbers with too few school days and their travel records

In step 7, according to the schedule of work and rest in primary and secondary schools, delete the records of departure from the school station before the last school time in the morning and the trip from home after the latest school time in the afternoon from the travel records of all the card numbers identified with the home station and the school station. For the record of departure from the station, in this embodiment, the time requirement is appropriately relaxed, and the record of departure from the school station before 9:00 and the record of departure from the home station after 16:00 are deleted.

In step 8, the number of school days for each card number in the statistical database S is stored in a new column "School Days", and the minimum threshold of the number of school days should be determined according to the date span of the collected data, that is, the data collected for n consecutive weeks are collected. The minimum threshold for the data is n. Since this embodiment uses the subway card swiping records of three consecutive working days as a sample, the card numbers and their records with less than 3 days of school days are deleted. At this point, the final database Z has been obtained, including the columns "card number", "inbound date", "inbound time", "outbound date", "outbound time", "inbound station number", "outbound station number" , "inbound station longitude", "inbound station latitude", "outbound station longitude", "outbound station latitude", "home station", "school station", "days to school", some identification results are shown in Table 9 shown. The number of general students identified in this embodiment accounts for 40% of the total number of students in the original data.

Table 9 Examples of partial recognition results

Table 9 (continued) Examples of partial recognition results

Claims (8)

1. a general school identification method based on subway credit card data, is characterized in that, this method comprises the following steps: (1) Collect IC card swipe data and subway station coordinate data of each subway station within a certain period of time, preprocess the original card swipe data, and eliminate invalid data; (2) Calculate the frequency of occurrence of all stations of the card number for each card number, find out the station with the highest frequency of occurrence, and count the number of stations with the highest frequency of occurrence of each card number; (3) According to the number of stations with the highest frequency, find out the candidate stations of the home station and school station of each card number, including: (31) Determine the number of stations with the highest frequency of occurrence of each card number. If the number of stations is 1, the corresponding station is the candidate station Si1 of the card number; if the number of stations is 2, the corresponding two stations are the card number. The candidate stations Si1 and Si2 of the are the candidate stations Si1 and Si2 of the card number respectively. If the number of stations with the highest frequency after the combination is still greater than 2, the card number and the corresponding record will be deleted; (32) For the card number of only one candidate station Si1 found in step (31), calculate the frequency of occurrence of the station corresponding to the candidate station Si1, and find the other one of the home station or the school station according to the number of corresponding stations with the highest frequency of occurrence Candidate station Si2: If the number of stations with the highest frequency in the corresponding station is 1, the station is another candidate station Si2 for the card number; if the number of stations with the highest frequency in the corresponding station is greater than or equal to 2, the distance will not exceed the specified distance. The stations with the threshold are merged, and then the station with the highest frequency is Si2. If there is still a tie after the merger, the card number and the corresponding record will be deleted; (4) According to the schedule of work and rest in primary and secondary schools, the travel records between all card number candidate stations are divided into several categories according to the entry time; (5) For the trip records that fall into each category, the home station and the school station are determined according to the earliest or latest entry time, so as to obtain the trip records of school trips. 2. a kind of general school identification method based on subway card swiping data according to claim 1, is characterized in that, the original card swiping data in described step (1) comprises: card number, entry date, entry time, exit date , departure time, inbound station number, outbound station number, card type. 3. a kind of general school identification method based on subway card swiping data according to claim 2, is characterized in that, in described step (1), to original data preprocessing comprises: only keep card type as student card and the date of entry belongs to Records of working days, and delete abnormal data with the same inbound station number and outbound station number and different inbound and outbound dates. 4 . The general school identification method based on subway card swiping data according to claim 1 , wherein the duration of a certain period of time in the step (1) is not less than three weeks. 5 . 5. a kind of general school identification method based on subway credit card data according to claim 1, is characterized in that, in described step (4), travel category comprises: am: The pit stop time is before the last school time in the morning; pm: The pit stop time is after the earliest dismissal time in the afternoon; noon1: The entry time is within the range from the earliest dismissal time in the morning to the earliest beginning time in the afternoon; noon2: The entry time is within the range from the earliest school time in the afternoon to the last school time in the afternoon. 6. a kind of general recognition method based on subway card swiping data according to claim 5, is characterized in that, described step (5) comprises: four categories are sorted according to card number, entry time respectively; For the card number of am, the inbound station number of the first trip record is determined as the home station, and the corresponding outbound station number is the school station; for the card number with the trip record belonging to pm, the outbound station number of the last trip record is determined. It is determined as the home station, and the corresponding inbound station number is the school station; for the card number with the trip record belonging to noon1, the inbound station number of the first trip record is determined as the school station, and the corresponding outbound station number is the home station; For a card number with a trip record belonging to noon2, the outbound station number of the last trip record is determined as the school station, and the corresponding inbound station number is the home station. 7. a kind of general school identification method based on subway card swiping data according to claim 1, is characterized in that, also comprises: according to middle and primary school work and rest time arrangement, delete in all the travel records of the card numbers of the identified home station and school station Records of departures from the school station before the latest school time in the morning and records of departures from the home station after the latest school hours in the afternoon. 8 . The method for general school identification based on subway card swiping data according to claim 1 , further comprising: deleting card numbers and travel records whose days of school days are less than a specified threshold number of days. 9 .

Images