TWI880479B - Automated classification system and method for scalp sebum properties - Google Patents

Abstract

The invention relates to an automated system and method for classifying scalp sebum properties. A high-resolution image of a scalp section is captured and divided into non-overlapping patches. These patches undergo random rotation and position swapping for data augmentation and privacy preservation. Each patch is then converted into a one-dimensional vector, to which positional information is added through learnable embeddings. The vectors are fed into a Vision Transformer model that outputs a sebum classification label. The system and method offer advantages in speed, accuracy, and privacy preservation over traditional and other AI-based methods.

Description

Automatic classification system and method for scalp sebum characteristics

The present invention relates generally to the fields of computer vision and machine learning. More specifically, the invention relates to a system and method for automatically classifying scalp sebum properties using a visual transformation model.

Traditional methods for determining scalp sebum characteristics, such as dry, normal, or oily, usually rely on manual inspection and assessment. These methods are not only time-consuming, but also prone to human error and variability due to subjective judgment. In addition, the assessor's psychological state or professional knowledge may affect the manual method, resulting in inconsistent results.

Some existing computer vision techniques have been used to automate this process. However, these methods often have difficulty handling subtle feature differences that are critical for accurately classifying sebum. In addition, the data used for training and validation in these methods often suffer from class imbalance, requiring data augmentation techniques to improve model performance.

Additionally, privacy concerns have been raised regarding the use of high-resolution scalp images, particularly when these images are stored or transmitted for analysis.

Therefore, a more reliable, efficient, and privacy-preserving method for automatically classifying scalp sebum characteristics is needed. The method should be able to handle subtle feature differences and class imbalance in the data while ensuring the privacy of the individual scalp images being analyzed.

The present invention proposes a system and method for automatically classifying scalp sebum properties using a visual transformation model. The goal of this invention is to overcome the limitations of existing methods and provide a solution that is both effective and reliable. Traditional methods often rely on manual inspection, which is not only time-consuming but also prone to human error and subjectivity. The present invention automates this process, thereby eliminating these problems.

The system captures high-resolution images of sections of the scalp, which are then segmented into non-overlapping blocks. Each block is converted to a one-dimensional vector. Learned positional embeddings are added to these vectors to preserve spatial information lost during the flattening process. These enhanced vectors are then fed into a visual transformation model, which processes the data and outputs a classification label representing the sebum properties of the scalp, such as dry, normal, or oily.

A unique feature of this invention is the optional data augmentation module that randomly rotates and swaps the blocks. This not only improves the performance of the model and solves the class imbalance problem, but also adds a layer of privacy protection.

By automating the classification of scalp sebum characteristics and combining advanced machine learning techniques, this invention provides a comprehensive and effective solution to the challenges faced by existing methods. It effectively handles subtle feature differences, resolves class imbalance in training data, and ensures the privacy of the analyzed scalp images.

In order to make the above features and advantages of the present invention more clearly understood, the following is a detailed description of the preferred embodiment with the accompanying drawings.

100: Automatic classification system for scalp sebum characteristics

110: Imaging device

120: Block extraction module

130:Data enhancement module

140: Vector conversion module

150: Location embedding module

152: Feature extraction module

152A: First fully connected layer

152B: Second fully connected layer

154: Activate function module

154A:GELU activation function

156: Standardization module

156A: Sigmoid activation function

160: Visual transformation model

162:Self-attention mechanism

164: Forward Neural Network

166: Sebum classification label

S110~S170: Flowchart steps

S210~S230: Flowchart steps

Various embodiments will be described below with reference to the accompanying drawings, which are intended to illustrate and not to limit the scope in any way, in which like reference numerals represent like elements, and in which:

FIG1 is a block diagram showing one embodiment of the automatic classification system of scalp sebum characteristics of the present invention.

FIG2 shows a flow chart of one embodiment of the automatic classification method of scalp sebum characteristics of the present invention.

Figures 3A to 3C are schematic diagrams showing the process of converting an image into blocks in the present invention.

FIG4 is a flowchart showing the process of adding position information to a one-dimensional vector according to the present invention.

FIG5A shows the structure diagram of the location embedding module of the present invention.

FIG. 5B shows one embodiment of the position embedding module of the present invention.

FIG6 is a schematic diagram of the visual transformation model of the present invention.

The present invention is best understood with reference to the detailed description and accompanying drawings set forth herein. Various embodiments will be discussed below with reference to the accompanying drawings. However, those skilled in the art will readily appreciate that the detailed description given herein with respect to the accompanying drawings is for illustrative purposes only, as the methods and systems may extend beyond the described embodiments. For example, the teachings given and the requirements of a particular application may yield a variety of optional and suitable methods to implement the functionality of any detail described herein. Therefore, any method may extend beyond the specific implementation options described and shown in the following embodiments.

Please refer to FIG. 1 and FIG. 2. FIG. 1 is a block diagram of one embodiment of the automatic classification system for scalp sebum characteristics of the present invention, and FIG. 2 is a flow chart of one embodiment of the automatic classification method for scalp sebum characteristics of the present invention. Please refer to step S110. The automatic classification system for scalp sebum characteristics 100 disclosed in this embodiment first captures a high-resolution image of the scalp part. This process can be performed using various imaging devices 110 that can capture detailed images of the scalp surface. The captured scalp image serves as the main input of the automatic classification system for scalp sebum characteristics 100, providing raw data for determining sebum characteristics.

Once the image is captured, step S120 is performed and the input image is segmented into non-overlapping blocks. This segmentation process is performed by the block extraction module 120, which segments the high-resolution scalp image into smaller, more manageable parts. Each block represents a specific area of the scalp, and this block-based approach allows for a more detailed and localized analysis of scalp sebum properties at a later time.

After the block extraction, step S130 is executed, and the data enhancement module 130 randomly rotates and swaps the blocks. This step introduces variability into the data, which helps to enhance the robustness of the artificial intelligence model by exposing the artificial intelligence model to a wider range of data scenarios. In addition, the random rotation and position swap of the blocks also serve as a privacy protection measure. By disrupting the original arrangement of the blocks, the scalp sebum characteristics automatic classification system 100 makes it difficult to reconstruct the original scalp image, thereby protecting personal privacy.

Then, step S140 is executed, and each block is converted into a one-dimensional vector by the vector conversion module 140. This conversion process involves flattening the two-dimensional block into a one-dimensional array of pixel values. The resulting vector retains the pixel intensity information of the block, but its format is more suitable for processing by subsequent components of the system.

Next, step S150 is performed to add position information to the one-dimensional vector through a learnable embedding vector. This step is performed by a position embedding module 150, which adds a layer of spatial information to the vector. The learnable embedding vector encodes the relative position of block 11 in the original scalp image, allowing the scalp sebum characteristics automatic classification system 100 to retain some spatial context.

It is worth noting that the position embedding module 150 can have multiple possible configurations. Referring to FIG. 5A , the position embedding module 150 can include at least one feature extraction module 152, at least one activation function module 154, and at least one standardization module 156. The feature extraction module 152 is used to transform the one-dimensional vector of the block, the activation function module 154 is used to introduce nonlinearity, and the standardization module 156 is used to standardize the output value.

Next, step S160 is performed, and the vector with enhanced position embedding is input into the visual transformation model 160. The visual transformation model 160 is an artificial intelligence model designed specifically for image analysis tasks. It contains a multi-layer self-attention mechanism and a forward neural network, which work together to analyze the input vector and extract meaningful features related to the characteristics of scalp sebum.

Finally, step S170 is executed, and the visual transformation model 160 outputs a sebum classification label. This label indicates the sebum characteristics of the scalp portion represented by the input image, and can be one of the following three categories: dry, neutral, or oily. The classification label, as the final output of the scalp sebum characteristic automatic classification system 100, provides a concise labeling of the analyzed scalp sebum characteristics.

Below, the above-mentioned automatic classification system and method of scalp sebum characteristics will be described in more depth, please continue to refer to Figures 1 and 2. At the initial stage of the automatic classification process, that is, step S110, a high-resolution image 10 of the scalp portion is captured (as shown in Figure 3A). This image capture can be performed using various imaging devices 110, such as digital cameras, smart phone cameras, or specialized dermatology imaging devices. The imaging device 110 is positioned to capture a clear and detailed view of the scalp portion. The resolution of the captured image 10 is high enough to allow detailed analysis of the scalp surface, including texture, color, and other visual characteristics that may indicate sebum characteristics.

Once the high-resolution images of the scalp portion are captured, they are processed by the block extraction module 120. As shown in step S120, the block extraction module 120 is responsible for dividing the image into non-overlapping blocks 11 (as shown in Figure 3B). The segmentation process involves dividing the image into smaller parts, namely: blocks 11, each of which represents a specific area of the scalp. The size of the block 11 can be determined based on various factors, such as the resolution of the image, the level of detail required in the analysis, and the available computing resources. Each block 11 is treated as an independent analysis unit, allowing for a more localized and detailed examination of the scalp sebum characteristics.

The block extraction process is designed to ensure that each block 11 contains sufficient information for subsequent analysis. To this end, blocks 11 are extracted in a way that they cover the entire extent of the scalp portion depicted in the image without overlapping with other blocks. This non-overlapping arrangement ensures that each area of the scalp is analyzed only once, preventing redundancy in the analysis. In addition, the non-overlapping arrangement also helps to maintain the spatial integrity of the original image, as each block 11 corresponds to a clear and specific area of the scalp.

In summary, the process of capturing high-resolution images of scalp portions 10 and segmenting these images into non-overlapping blocks 11 ensures that the automatic scalp sebum property classification system 100 can obtain detailed and localized visual information about the scalp, which is then used to determine sebum properties in a reliable and efficient manner.

After extracting block 11 from high-resolution scalp image 10, scalp sebum characteristics automatic classification system 100 uses data enhancement module 130 to randomly rotate and swap the position of block 11 (as shown in FIG3C). This data enhancement module 130 introduces a certain degree of randomness, which has two main purposes.

First, the random rotation and position swapping of block 11 helps to enhance the robustness of the visual transformation model 160. In machine learning, robustness refers to the ability of a model to maintain its performance in the face of changes in input data. By introducing randomness, the scalp sebum property automatic classification system 100 exposes the visual transformation model 160 to a wider range of data scenarios, thereby encouraging the visual transformation model 160 to learn a more general and robust representation of scalp sebum properties. This is particularly beneficial in solving the problem of class imbalance, where certain sebum property categories may be underrepresented in the training data. By randomly rotating and swapping the positions of blocks, the scalp sebum characteristics automatic classification system 100 effectively increases the diversity of training data for each category, which helps to alleviate the impact of category imbalance and improve the overall performance of the model.

Secondly, the random rotation and position swapping of block 11 serves as a privacy protection measure. In this embodiment, privacy protection refers to protecting the identity of the individual in the scalp image being analyzed. By scrambling the original arrangement of block 11, it becomes difficult to reconstruct the original scalp image from the processed data. This means that even if someone obtains the data without authorization, the person cannot identify the individual from the scrambled blocks. This feature is particularly valuable in applications where personal privacy is critical, such as in medical diagnosis or personal care applications.

In summary, the data enhancement module 130 enhances the robustness of the visual transformation model 160 and protects personal privacy through random rotation and position swapping of block 11. Through random rotation and position swapping of block 11, the system is able to handle subtle feature differences and class imbalance in the data while ensuring the personal privacy of the analyzed scalp image.

After the data enhancement process, step S140 is executed and the scalp sebum characteristics automatic classification system 100 continues to convert each block 11 into a one-dimensional vector. This conversion is performed by the vector conversion module 140, which converts the two-dimensional block 11 into a one-dimensional array of pixel values. The conversion process involves flattening the block 11, which basically involves rearranging the pixel values from the two-dimensional grid into a one-dimensional sequence. This process retains the pixel intensity information of the block 11, but presents it in a format more suitable for processing by subsequent components of the scalp sebum characteristics automatic classification system 100.

Once the blocks 11 are converted into one-dimensional vectors, step S150 is executed, and the automatic scalp sebum characteristics classification system 100 adds position information to these one-dimensional vectors through learnable embedding vectors. This step is performed by the position embedding module 150, which is designed to encode the relative position of the block 11 in the original scalp image. The learnable embedding vector is essentially a set of parameters learned during the training of the visual transformation model. These parameters represent the spatial relationship between blocks 11, allowing the system to retain some spatial context despite the flattening process.

The process of adding position information to a one-dimensional vector involves a series of operations, please refer to Figure 4. First, as shown in step S210, each block 11 is assigned a position index according to its position in the original scalp image 10. This position index, as a kind of spatial metadata, provides additional context about the position of the block. Then, as shown in step S220, the position index is transformed into a higher-dimensional representation through a learnable embedding layer. This layer is essentially a lookup table with learnable parameters that maps the position index to a dense vector of real numbers. The resulting vector, called a position embedding, encodes the relative position of the block in the scalp image.

Then, as shown in step S230, the position embedding is added to the one-dimensional vector representing the block 11. This addition operation effectively combines the pixel intensity information of the block with the spatial information encoded in the position embedding. The resulting vector now contains both pixel intensity and position information and is then ready to be input to the visual transformation model 160 for further processing.

In summary, the process of converting each block 11 into a one-dimensional vector and adding position information through learnable embedding is a key step in the workflow of the automatic scalp sebum property classification system 100. This process ensures that the visual transformation model 160 receives comprehensive input data containing pixel intensity and spatial information, thereby achieving a more accurate and global-aware analysis of scalp sebum properties.

In addition, one embodiment of the architecture of the position embedding module 150 is shown in FIG5B . First, the position embedding module 150 includes a first fully connected layer 152A, which is used to perform a linear transformation on the one-dimensional vector of the block 11. The fully connected layer is a basic component in a neural network, and its function is to connect each input node (or neuron) to each output node. Here, the role of the first fully connected layer 152A is to perform a linear transformation on the one-dimensional vector of the block to extract higher-level features. In addition, the position embedding module 150 also includes a second fully connected layer 152B, which is connected to the first fully connected layer 152A and is used to perform a further linear transformation on the output of the first fully connected layer 152A, in order to further extract and transform features so as to better capture the complex pattern in the one-dimensional vector of block 11. The position embedding module 150 also includes a GELU activation function 154A, which is connected to the second fully connected layer 152B and is used to introduce nonlinearity to the output of the second fully connected layer 152B. The GELU activation function 154A is a commonly used activation function that can introduce nonlinearity, allowing the position embedding module 150 to learn and represent more complex functions. In addition, the position embedding module 150 also includes a Sigmoid activation function 156A, which is connected to the GELU activation function 154A and is used to convert the output value of the GELU activation function 154A to the range [0,1]. The Sigmoid activation function 156A is a function that compresses real numbers between 0 and 1, so that the output of the model can be interpreted as probability.

In FIG. 5B , the first fully connected layer 152A and the second fully connected layer 152B correspond to the feature extraction module 152 of FIG. 5A , the GELU activation function 154A corresponds to the activation function module 154 of FIG. 5A , and the normalization module 156 corresponds to the normalization module 156 of FIG. 5A . Of course, a person with ordinary knowledge in the field can. Of course, a person with ordinary knowledge in the field can adopt different implementation methods as needed. For example, the feature extraction module 152 can include more or better fully connected layers, the activation function module 154 can also be a Maxout activation function, a RELU activation function, or a SELU activation function, and the normalization module 156 can be a general normalization formula.

In summary, the position embedding module 150 is able to extract useful features from the one-dimensional vector of block 11 and transform these features into a form that enables the model to learn and predict better.

Please refer to FIG. 6 , the visual transformation model 160 is the core component of the scalp sebum characteristic automatic classification system 100 , which is responsible for processing the input vector and outputting the sebum classification label. The visual transformation model 160 is an artificial intelligence model designed specifically for image analysis tasks. It is based on the Transformer architecture, which was originally developed for natural language processing tasks but has been adapted for computer vision.

The visual transformation model 160 includes a multi-layer self-attention mechanism 162 and a forward neural network 164. The self-attention mechanism 162 allows the visual transformation model 160 to focus on different parts of the input data according to their relevance to the task at hand. In this embodiment, the self-attention mechanism 162 enables the visual transformation model 160 to focus on different blocks 11 of the scalp image 10, giving more attention to blocks that contain more information for the sebum classification task.

Each layer of the self-attention mechanism 162 in the visual transformation model 160 operates by computing a weighted sum of the input vectors, where the weights are determined by how relevant each vector is to the other vectors. This relevance is quantified using a metric called an attention score, which is calculated based on the similarity between vectors. The attention score is then used to weight the input vectors, allowing the visual transformation model 160 to focus more on vectors with higher scores.

The feedforward neural network 164 in the visual transformation model 160 is used to transform the weighted sum of the input vectors into a higher-level representation. These feedforward neural networks 164 are composed of multiple layers of neurons, each of which performs a linear transformation on its input and then uses a nonlinear activation function. The output of the feedforward neural network 164 is a set of feature vectors that capture the high-level patterns in the input data.

The visual transformation model 160 processes input vectors in a sequential manner, passing them layer by layer through the self-attention mechanism 162 and the forward neural network 164. At each layer, the visual transformation model 160 updates the feature vector based on the information extracted from the previous layer. This iterative process enables the visual transformation model 160 to gradually refine its understanding of the input data, resulting in more accurate sebum classification results.

Finally, the visual transformation model 160 outputs a sebum classification label 166 based on the final feature vector set. This sebum classification label 166 indicates the sebum characteristics of the scalp portion represented by the input image, and can be one of the following three categories: dry, normal, or oily. The sebum classification label 166, as the final output of the scalp sebum characteristic automatic classification system 100, provides a concise indicator of the analyzed scalp sebum characteristics.

In summary, two major technical improvements of the automatic classification system for scalp sebum characteristics 100 make significant contributions to its performance and privacy protection: random position transformation and rotation of block 11, and the use of learnable position encoding.

The random position shifting and rotation of block 11 is performed by the data augmentation module 130. The data augmentation module 130 introduces a degree of randomness into the data, which has two main purposes. First, it enhances the robustness of the visual transformation model 160 by exposing the visual transformation model 160 to a wider range of data scenarios. This is particularly beneficial when addressing class imbalance problems, where certain sebum property classes may be underrepresented in the training data. By randomly rotating and swapping the position of block 11, the scalp sebum property automatic classification system 100 effectively increases the diversity of the training data for each class, helping to mitigate the effects of class imbalance and improve the overall performance of the model.

Secondly, the random rotation and position swapping of block 11 serves as a privacy protection measure. By disrupting the original arrangement of block 11, the scalp sebum characteristics automatic classification system 100 makes it difficult to reconstruct the original scalp image, thereby protecting the privacy of the individual. This feature is particularly valuable in applications where personal privacy is critical, such as in medical diagnosis or personal care applications.

Another notable technical improvement in the automatic classification system for scalp sebum characteristics 100 is the use of learnable embedding vectors. This feature is implemented by the position embedding module 150, which adds a layer of spatial information to the one-dimensional vector representing each block, allowing the automatic classification system for scalp sebum characteristics 100 to retain some spatial context. This is particularly beneficial for the visual transformation model 160 because it allows the visual transformation model 160 to adapt to specific spatial relationships in the data, thereby achieving more accurate classification.

In summary, the random position transformation and rotation of block 11, as well as the use of learnable embedding vectors, are two major technical improvements in the scalp sebum characteristics automatic classification system 100 of the present invention, which improve its performance and protect privacy. These features ensure that the scalp sebum characteristics automatic classification system 100 can handle subtle feature differences and category imbalances in the data, while ensuring the privacy of the analyzed scalp images.

Although the present invention has been disclosed as above with preferred embodiments, it is not intended to limit the present invention. Anyone with common knowledge in the relevant technical field can make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of the patent application attached hereto.

S110~S170: Flowchart steps

Claims (10)

A method for automatically classifying scalp sebum characteristics comprises the following steps: Capturing a high-resolution image of a scalp portion via an imaging device; Dividing the high-resolution image into a plurality of non-overlapping blocks via a block extraction module; Performing random rotation and position exchange on the blocks via a data enhancement module; Converting each of the blocks into a one-dimensional vector via a vector conversion module; Adding position information to the one-dimensional vector via a learnable embedding layer via a position embedding module; Inputting the one-dimensional vector with the added position information into a visual transformation model; The visual transformation model outputs a sebum classification label according to the one-dimensional vector with the added position information. According to the method of claim 1, the step of adding position information to the one-dimensional vector through a learnable embedding vector includes the following steps: Transforming the one-dimensional vector through at least one feature extraction step; Applying at least one activation function to introduce nonlinearity to the output of the feature extraction step; and Standardizing the output value through a standardization step. According to the method for automatic classification of scalp sebum characteristics of claim 1, the sebum classification labels include: dry, neutral and oily. A method for automatically classifying scalp sebum properties according to claim 1, wherein the blocks are square. A method for automatic classification of scalp sebum characteristics according to claim 1, wherein the one-dimensional vector and the learnable embedding layer are concatenated before being input into the visual transformation model. A scalp sebum property automatic classification system comprises: an imaging device configured to capture a high-resolution image of a scalp portion; a block extraction module configured to divide the image into non-overlapping blocks; a data enhancement module configured to randomly rotate and positionally swap the blocks; a vector conversion module configured to convert each of the blocks into a one-dimensional vector; a position embedding module configured to add position information to the one-dimensional vector through a learnable embedding layer; a visual transformation model configured to receive the one-dimensional vector with added position information and output a sebum classification label. A system according to claim 6, wherein the position embedding module is configured to process the learnable embedding vector through a transformation sequence, and the position embedding module includes: At least one feature extraction module for transforming the block vector; At least one activation function module, connected to the feature extraction module, for introducing nonlinearity; A normalization module, connected to the activation function module, for normalizing the output value. According to claim 6, the automatic classification system for scalp sebum characteristics, wherein the sebum classification labels include: dry, neutral and oily. An automatic classification system for scalp sebum properties according to claim 6, wherein the blocks are square. According to claim 6, the automatic classification system of scalp sebum characteristics, wherein the one-dimensional vector and the learnable embedding layer are concatenated before being input into the visual transformation model.