FR2871320A1 - SYSTEM AND METHOD FOR FILTERING ELECTRONIC MESSAGES - Google Patents

Abstract

This system for filtering (10) electronic messages comprises means (16) for receiving electronic messages (12a, 12b, 12c), means (18) for analyzing character strings contained in the received electronic messages and means (20) classification of these electronic messages based on results provided by the analysis means.The analysis means comprises means (24) for detecting scrambled character strings for providing at least a portion of the analysis results used by the classification means (20).

Description

{2871320

  The present invention relates to an electronic message filtering system and a corresponding method.

  More precisely, the invention relates to a system for filtering electronic messages, of the type comprising means for receiving electronic messages, means for analyzing character strings contained in the received electronic messages and means for classifying these electronic messages. on the basis of results provided by the means of analysis.

  For some applications in particular, it is interesting to classify the received electronic messages as acceptable or unwanted messages, For example, among the electronic messages that a user receives, some are unwanted and waste time. To overcome this drawback, there are known systems for filtering electronic messages to sort between messages likely to interest the user and others, excluded a priori as unwanted messages.

  In general, the known filtering systems comprise means for classifying electronic messages received on the basis of predetermined rules or learning mechanisms. When the messages contain text, the decision criteria are most often based on the analysis of strings contained in the electronic messages, including the detection of predetermined character strings, such as keywords or combinations of words. keys recognized by the system.

  For example, the scanned e-mail messages are discarded, as long as the probability that they are undesirable is sufficiently high, given the key words or combinations of keywords detected in their content.

  However, since the objective of the senders of unwanted messages is precisely to reach their receivers, it is more and more common to see such messages in which the keywords that can be detected are processed so as to pass through the message. filtering system while remaining intelligible to a user. For example, it is common to add symbolic characters between the letters of the keywords, to delete certain characters that do not prevent a user from understanding the word, or to replace certain characters with others that resemble them (a "i" by a "1", an "S" by a "$", etc.).

  These processes performed on the contents of unwanted messages, allow their transmitters to bypass the known filtering systems.

  The object of the invention is to remedy this problem by providing an electronic message filtering system capable of also detecting this type of unwanted message.

  The subject of the invention is therefore a filtering system of the aforementioned type, characterized in that the analysis means comprise means for detecting scrambled character strings for the supply of at least part of the analysis results used by the means of classification.

  Thus, the use of scrambled character detection means makes it possible to pre-process the received electronic messages, before transmitting them to the classification means, so that they can sort the electronic messages also on the basis of analysis results provided by these detection means. These analysis results are intended to facilitate the recognition of messages that have been scrambled.

  In particular, for some unwanted message detection applications, the classification means are means for classifying the received electronic messages as acceptable or undesirable messages.

  Optionally, the analysis means comprise means for providing, to the classification means, statistical parameters relating to the scrambled character strings detected.

  Indeed, since some unwanted electronic messages are processed to bypass conventional filtering systems, the provision to the classification means of statistical parameters, for example relating to the number of interference or to the types of interference detected, may allow it alone, or combination with the provision of other conventional classification data (keyword lists for example), to discard unwanted electronic messages.

  Also optionally, the classification means comprising means for detecting predetermined character strings in the content of these electronic messages, the analysis means further comprise means for descrambling the character strings detected as scrambled for the provision of E-mail messages unscrambled by means of classification.

  Indeed, since most known filtering systems are adapted to detecting predetermined keywords, if the system further comprises means for descrambling strings of characters detected as scrambled, then it is possible to integrate them simply in the means of d analysis, without the need to adapt the means of classification.

  In addition, the analysis means are for example means for analyzing the content of the received electronic messages according to a technique of automatic processing of natural language.

  Indeed, it appears that when the analysis of a possibly scrambled content message is performed according to a natural language automatic processing technique, the results of the analysis are particularly relevant to the classification of this message.

  Some interference is indeed sufficiently simple, such as insertions, omissions or character substitutions, to be easily detected using known algorithms based on natural language processing techniques.

  In order to detect different types of interference which may have been carried out on certain words of the unwanted electronic messages, the means for detecting scrambled character strings may comprise, more specifically, means for detecting substitutions, insertions, and / or deletions of characters in the strings of the received electronic messages.

  The subject of the invention is also a method for filtering electronic messages comprising a step of receiving electronic messages, a step of analyzing strings contained in the received electronic messages and a step of classifying these electronic messages based on of results provided by the analysis means, characterized in that, during the analysis, scrambled character strings are detected for providing at least a portion of the analysis results used in the classification step .

  An electronic message filtering method according to the invention may further comprise the characteristic that the content of the received electronic messages is analyzed according to a technique of automatic processing of natural language.

  The invention will be better understood with the aid of the description which follows, given solely by way of example and with reference to the appended drawings in which: FIG. 1 schematically represents the general structure of a filtering system electronic messages according to the invention; and FIG. 2 represents the successive steps of a method implemented by the system of FIG. 1.

  The installation shown in FIG. 1 comprises a system 10 for filtering electronic messages 12a, 12b and 12c. These electronic messages are transmitted to the filtering system 10 via a network 14 for transmitting data, such as the Internet network.

  Among the electronic messages 12a, 12b, 12c, some are for example purely advertising or more generally undesirable. The filtering system 10 therefore has the function of performing a preliminary sorting, allowing the final recipient of these electronic messages, not to waste time to consult messages uninteresting for him.

  Among the received unwanted messages, some are easily recognizable, because they contain predefined data, and known by the filtering system 10, allowing the latter to conclude that these emails are likely to be messages undesirable.

  The data contained in an electronic message, which makes it possible to recognize that it is undesirable, are for example its background color, the links it contains, the e-mail address of the sender, as well as all the words used in the message. body of the message or in its header.

  In particular, the search for keywords in the body of electronic messages makes it possible to discard many undesirable electronic messages.

  However, some unwanted electronic messages are scrambled, so that some words they contain are not recognized by the filtering system 10, but are always intelligible to the recipient.

  The interference made by their transmitters is generally easy to detect thanks to tools implementing techniques for automatic processing of natural language.

  Thus, it can be achieved a morphological type of scrambling, which is characterized by the insertion, omission, or substitution of characters of a word, so as to keep the word readability by the recipient, while making it undetectable by the filtering system 10. By way of example: - by insertion of dashes, the word "reverses" becomes "re-ver-ses", - by omission of a letter, the word "generic" becomes "genric" ", - by substituting one letter, the word" PILLS "becomes" P1 LLS ".

  Another jamming involves inserting random character strings. These random character strings have a very low probability of actually matching a word of a language. The addition of these random character strings makes it possible to modify the statistical representation of the content of the message without modifying the semantic content of the message.

  An insertion of characters can be replaced by the insertion of tags, for example of HTML type, having no meaning for an HTML parser, but making it possible to hide a word that should not be detected by the filtering system 10.

  For example the word "generic" can be concealed as follows: ge <kxlalx> ner <txwoyq> ic.

  Finally, another technique involves inserting a random sequence of words, each of these words being known but having no syntactic or semantic relationship consistent with the others. This does not change the semantic content of the message and the recipient is not disturbed by such interference. Like inserting random strings, this changes the statistical representation of the content of the e-mail message and makes it possible to not detect it as an undesirable message.

  These different types of scrambling act at the various levels, classically identified, techniques of automatic processing of natural language.

  In particular, known algorithms for detecting insertions, omissions, or character substitutions, which fall under standard typographic correction techniques, can be used to detect and replace words that have been scrambled by insertions, omissions or substitutions. by corresponding unscrambled words. There are also algorithms for detection and correction of scrambled words by detachment.

  To handle more complex cases, distance calculations between words may suffice.

  Thus, the filtering system 10 comprises means 16 for receiving the electronic messages 12a, 12b and 12c. It also comprises means 18 for analyzing character strings contained in the electronic messages received by the means 16, and means 20 for classifying these electronic messages as acceptable or undesirable messages on the basis of results provided by the means. Analysis 18.

  Finally, the filtering system 10 comprises means 22 for sorting the received electronic messages, 12a, 12b and 12c, according to their acceptable or undesirable nature. Thus, for example, the e-mail message 12a is discarded as an undesirable message, and the e-mail messages 12b and 12c are transmitted to the recipient as acceptable messages.

  The analysis means 18 comprise means 24 for detecting scrambled character strings associated with means 26 for descrambling the strings of characters detected as scrambled.

  For example, the detection means 24 comprise means 28 for detecting character substitutions in the character strings of the received electronic messages, means 30 for detecting character insertions in the character strings of the electronic messages received and means for detecting 32 for detecting deletions of characters in the strings of the received electronic messages. These detection means 28, 30 and 32 are means which, independently of the others, are known and conform to a technique of automatic natural language processing.

  The scrambled character detection means 24 are furthermore associated with a linguistic knowledge base 34 for the use of syntax and grammatical resources of this database for the purposes of the analysis performed by the means 18.

  The descrambling means 26 treats the character strings detected as scrambled so as to convert them into strings of unscrambled characters. The processing is directly related to the type of interference detected. For example, if it is a scrambling by character insertions, the characters recognized as inserted are deleted. In the case of scrambling by character substitutions, the characters recognized as substituting are replaced by the characters to which they substitute. If they are scrambled by character deletions, they are re-inserted into incomplete character strings.

  The results of the analysis carried out by the means 18 are provided to the classification means 20. The results of the analysis comprise: the e-mail address, that is to say the electronic message in which all the strings detected as scrambled have been replaced by new unscrambled character strings, and - quantitative statistical parameters relating to strings detected as scrambled, these statistical parameters being provided by the detection means 24.

  For this purpose, the classification means 20 comprise a first module 36 for classifying the received electronic messages, on the basis of the unscrambled messages provided by the analysis means 18. This module 36 is conventional and will not be described further. It can be taken over filtering systems known in the state of the art. In general, this type of classification module operates in two different possible ways: it can be an expert system based on rules resulting from the regular observation of acceptable or undesirable messages; or it may be a learning system driven on the basis of learning statistical rules, using the frequencies of the words or symbols identified in each of the messages considered acceptable or undesirable.

  An example rule for an expert system is as follows: if "Viagra" is in the message, then the message is undesirable, with a score of 0.9.

  An example of learning, which can be interpreted as an example of a statistical rule, is for example the following: if the words "Viagra" and "buy" are in the message, then the probability that the message is an undesirable message is 0 , 9.

  The classification means 20 comprise a second module 38 for classifying the received electronic messages, on the basis of the statistical parameters relating to the scrambled character strings detected, these statistical parameters being provided by the analysis means 18, and more specifically by the means detection of scrambled character strings 24.

  These parameters may include: - the total number of detected interference; the number of insertions detected, this number being able to be declined in as many parameters as of inserted type of characters detected; the number of substituted characters, which number can also be declined in as many parameters as detected substitute characters; a ratio expressing the number of words detected as scrambled on the total number of words; - etc. For example, if on a 614-word e-mail, 90 interferences have been detected, 70 concerning character insertions and 37 concerning character substitutions, one can decide on its classification as an acceptable or unwanted message on the basis of rules. such as: if the message contains more than 50 jamming, then it is an unwanted message, with an associated score of 0.9 on a scale of 0 to 1; the probability that an unwanted message contains more than 90 interference is 0.9, after counting the respective scrambling frequencies in the unwanted messages and in the acceptable messages.

  The method implemented by the system previously described is shown in FIG.

  In a first step 40, electronic messages 12a, 12b, 12c are received.

  During the next analysis step 42, possibly using the language knowledge base 34, the means 24 detect a certain number of scrambled character strings and update statistical parameters relating to the scrambled character strings detected. .

  Then, during a descrambling step 44, the means 26 converts the words detected as scrambled into words that have been unscrambled and updates a list of these unscrambled words.

  Finally, during a sorting step 46, the classification means 20 associated with the means 22, sort the received electronic messages, to distinguish the unwanted messages, such as the message 12a, acceptable messages such as electronic messages 12b and 12c.

  It is clear that a message filtering system as described above makes it possible to detect unwanted messages, not only by analyzing their contents, in order to detect particular keywords or predetermined expressions, but also by the detection interference, possibly made on the content of these electronic messages, for the provision of a descrambled message and / or statistical parameters relating to the interference, to the classification means 20.

Claims (10)

  A system for filtering (10) electronic messages (12a, 12b, 12c) comprising means (16) for receiving electronic messages, means (18) for analyzing character strings contained in the received electronic messages, and means (20) for classifying these electronic messages on the basis of results provided by the analysis means, characterized in that the analysis means comprise means (24) for detecting scrambled character strings for the provision of at least a part of the analysis results used by the classification means (20).   2. The electronic message filtering system (10) according to claim 1, wherein the analysis means (18) comprise means (24) for providing, to the classification means, statistical parameters relating to the scrambled character strings detected. .   An electronic message filtering system (10) according to claim 1 or 2, wherein, the classification means (20) having means (36) for detecting predetermined character strings in the content of these electronic messages, the analysis means (18) further comprise means (26) for descrambling the character strings detected as scrambled for the supply of unscrambled electronic messages to the classification means (20).   An electronic message filtering system (10) according to any one of claims 1 to 3, wherein the analyzing means (18) is a means for analyzing the content of the received electronic messages according to an automatic processing technique. of natural language.   An electronic message filtering system (10) according to any one of claims 1 to 4, wherein the classification means (20) is a means of classifying the received electronic messages as acceptable or undesirable messages.   An electronic message filtering system (10) according to any one of claims 1 to 5, wherein the scrambled character detection means (24) includes means (28) for character substitution strings of the received electronic messages.   An electronic message filtering system according to any one of claims 1 to 6, wherein the scrambled character detection means (24) includes means (30) for detecting character insertions in the strings. characters of the received e-mails.   An electronic message filtering system according to any of claims 1 to 7, wherein the scrambled character detection means (24) includes means (32) for detecting character deletions in the strings. received e-mail messages.   9. An electronic message filtering method comprising a step (40) for receiving electronic messages, a step (42) for analyzing strings contained in the received electronic messages and a step (46) for classifying these electronic messages. on the basis of results provided by the analysis means, characterized in that, during the analysis, scrambled character strings are detected for providing at least a part of the analysis results used during the analysis. classification stage.   10. The method of filtering electronic messages according to claim 9, wherein the content of the received electronic messages is analyzed according to a technique of automatic processing of natural language.