AT527503A2 - Method for identifying a set of building blocks - Google Patents

Abstract

Method for identifying a set of building blocks (1), each with a shape and a color, wherein the following steps are carried out: - scanning the set of building blocks (1) and obtaining a set of recognized building blocks (2), each with an associated color shade (3), - selecting at least one reference block (4) from the set of recognized building blocks (2) by identifying a known shape of a known building block (5), - assigning a known color to the at least one reference block (4) and preferably carrying out a color calibration using the at least one reference block (4), - assigning known colors to the rest of the set of recognized building blocks (2) based on a known, preferably relative, frequency distribution (7) of the known colors of the known building blocks (5) and/or the results of the color calibration.

Description

A X x —_PYSm reyae 3 ATi Sm What SG ae be a SEE Zw ? a S a SS Yesayı uam The vorliie GO QUL Anch SATT Eriıfft a procedure for ae 3 Sa are ES: Mana am Stainen mi den ale ainer L Zereit KLDLELGS AA Eu ADS ALM es VON building blocks ML TEWSLLS SLIKEX Y N A _- Way 5 a a N a Ban en BF SCI VI S1LNSE FR £ E > X yar, a SON £ T % x Ca N N aa BANN Ar OA R x < KA U N : x . DRAN N sanders Bat a a N ee Önshesondere I ir AUSDESCONKSTE DEU SPLSLZEUG, LESDESONASTE 'Lu H S 535 sr In em sm 8m a > x Saw ss A < es Ya A X SO es am SAL SS OUT SR Dam $ AS CL US © SA Ko LA

a terer CS

ELITE An aan DBaı 5537 5 & vn BANN ATI NP Er LANAE =.

AN } AKA KL ALL a mil SS Sa MASS KALI VA

ji & Ran aea HAUSK

A A n SFEaN en m Se ext DS 5 & N S a yanmadr de Yirı SC so Y x ALT X ha kits UNSOTMLSF ung mixed in a Ka Sean ELLE CSN

is Nı u tersr transient Versuc Sean as - ae + m Haufer A USES ESErTEeEDLLON TRY; SLNZSSLNG 5 Sausages in the Haulen

5 FA clear KA Yı east inmtes Ohiekt ® 558 Rz at Ze BZ LEI AN si 0 DSSTLN En 3 ER S1L1NSeSsS ZBausatlzess n —_- Each 4 I A ar ae a ROHR an ON en we A eg 3 An Sammy han. learn about the zoriiegen! Year w aa to build together. The procedure of the VOrLLSdENGEN AYC is 3 + x

= = 5 : ” s 5 : A Ta a Fe Verswrrsen ne DET SEN Me Ss aa a ml N ae Orundi E&TE ür korreas DE KL TETS EOS ey Sa Is Is nn Qi Alias or the nei fieı eh annar Da yyassd a piece list or the dentifiızı missing components inside 5

Sa Ban © A e N AL N

The method for recognizing a set in the state of the art is generally initiated with a scan of the set of building blocks. For example, the set can be photographed using a smartphone or tablet, so that the scan saw, the photo is the basis for the object recognition process for identifying the shapes and colors. The procedure for recognizing the shape is

-Madel Lien,

LEN #

Meatchoden

Ka N ttals DVS YSSi

Sea le aa gt . SSL CL VE SS} © MN SS A

KEIL There

2

especially by means of machine learning such as deep neural networks, where the shape recognition is already relatively

works reliably.

However, the problem remains that color recognition has so far been unsatisfactory. The scans often exhibit very variable and/or poor quality, which is mainly due to unfavorable exposure conditions, so that the colors of the building blocks are often distorted, cannot be distinguished from one another, and cannot be correctly assigned. This challenge is particularly evident with grayscale,

Yellow, blue or red tones.

To solve this problem, state-of-the-art methods often use color calibration methods using color targets with reference color values, such as the Munseli target or the IT8 target. Unfortunately, this method requires that a color target must also be scanned. For example, US 2014329598 A1 proposes arranging the building blocks on a background with color targets, where the background could be a printed cardboard. Similarly, the method of CN 108986173 A requires a

single-color calibration board as background,

The requirement to provide a color target, a background or a calibration board (in short: color calibration element) in addition to the set of building blocks is

extremely impractical and disadvantageous for users,

The object of the present invention is therefore to provide a method for identifying objects that is improved compared to the prior art and that can be used in particular without

color calibration element.

- Scanning the set of building blocks and obtaining a set of recognized building blocks with an assigned color shading,

- Selecting at least one reference brick from the set of recognized bricks by identifying a known shape of a known brick from a set of known bricks,

- Assigning a known color to at least one reference stone and preferably performing a color calibration, in particular by means of a color calibration matrix, based on the at least one reference stone,

- Assigning known colors to the rest of the set of recognized building blocks based on a known, preferably relative, frequency distribution of the known colors of the known building blocks and/or the

Color calibration results.

The identification of the building blocks is based on the known frequency distribution of all known colors of all

known building blocks and/or on color calibration.

In short, according to a central aspect of the invention, the color calibration is carried out by detecting a shape of at least one reference element, the color of which is known with certainty or with a sufficiently high probability.

is.

the process to identified building blocks,

The great advantage of the invention is that the assignment of the known color of at least one reference brick and further to the rest of the set of recognized bricks is carried out directly via the recognized bricks. This means that the color calibration is carried out via the bricks themselves and not via an additional color calibration element. Thus, only the set of bricks to be identified needs to be scanned; scanning an additional

Color calibration element is not required.

The method according to the invention therefore represents a significantly simpler option with less effort compared to prior art object recognition methods. Despite the simplicity of the method, reliable color recognition and assignment to the building blocks

but at least as reliable.

Another advantage in this context is that no exact color matching between a scanned color and a color of a color calibration element is required; by using the known frequency distribution, a very high hit rate in color recognition is already achieved, especially of the at least one reference stone.

This results in a simplified assignment of a known

Color to at least one reference stone. Furthermore,

a simplified assignment of known colors to the rest of the

SZ Set of recognized building blocks based on a known

Frequency distribution of the known colors of the known

Building blocks and/or color calibration.

Depending on the lighting conditions and/or the number of reference bricks and/or the distinguishability of the colors of the detected bricks, it may be possible to dispense with the consideration of the frequency distribution and/or the color calibration. The worse the lighting conditions and/or the smaller the number of reference bricks and/or the lower the distinguishability of the colors, the more advantageous the consideration of the frequency distribution in combination with the

Color calibration.

The scanning of the set of building blocks is generally carried out

using a camera, which creates a digital image of recognized building blocks and makes them available for further processing.

The recognized building blocks are not yet

identified; but first recognized.

Preferably, the camera is integrated into a smartphone, tablet or similar, so that the further processing of the

recognized building blocks could be done using an app.

The recognized building blocks each have a color shade assigned to the respective building block. The color shades are initially unidentified or not yet assigned to a known color;

but first recognized.

The color shades and/or known colors are shown in

Generally preferably characterized as RGOB values.

In principle, any known building block can be selected as a reference block, which is a building block of the set of recognized building blocks. This means that only building blocks actually contained in the set of recognized building blocks can

actually be reference stones.

Particularly preferably, several, preferably at least four, reference stones are selected. Less preferably, exactly

a reference stone was selected.

The color calibration is carried out using at least one reference stone, preferably at least two, particularly preferably at least four, reference stones

be selected.

Particularly preferred are at least four reference stones in

four different shades of color,

Particularly in the case of very large quantities or heaps of building blocks and/or heterogeneous lighting conditions, the heaps can preferably be divided into different areas or sub-areas, with at least one reference stone preferably being selected in each sub-area.

and/or used for color calibration.

Preferably, two or more reference stones are present in each sub-area; particularly preferably, there are at least four reference stones in different color shades. In this case, the color calibration is

for each sub-area separately.

Alternatively, the piles can also be manually divided into sub-areas

be divided so that the most homogeneous

is present.

Preferably, at least one reference brick is selected whose known shape has a complex surface contour compared to all known shapes of the known building blocks, i.e. is in particular asymmetrically formed and/or characterized by a plurality of intermixing mathematically

writable surfaces.

Furthermore, at least one reference stone is preferably selected, the color shade and/or anticipated known color of which is likely, i.e. with a probability of more than 50% and/or more than 70% and/or

over 90%, a known color isC,

The recognized building blocks can also be printed with at least one additional color, whereby the additional color can differ from the color shade assigned to the recognized building block. However, for the execution of the method, preferably only the assigned

Color shading used.

The well-known yellow man's head (design) is particularly preferred as a reference brick for a set of LEGO bricks.

Number 3626) is used,

Other possible reference bricks for a set of LEGO bricks could be green trees or plants and/or red roof tiles, as they, like the yellow man’s head, have a comparatively large

Probability in the aforementioned colors,

.y x 5 5 x | - x Sm Ss { al Han SETS Re} Az street sets {1 ZAMAST KYEUZUNg, curve) OdSX So arı br za hr ae © Satan Pr Si ae Wr OS A Ki senbahnset 3 LETSN KESSEL EN VELWS © WEL < AL ‘ s St en N. x x 3 Ta + ekannt an Warıyen ni FE KERN x 3 nn OU En 3 N Ya aranert sr hekenten Fa x} I. > aM ar A N yet Assigning the known colour to at least its A a A m A es je n x ss x X Ta 8 Tania £ en sven zstein ısi sSheasandeaere KV en Sy ES Type NO ETAGE x BETEN ZSCCClN LS sbesoand e AS S18 Ka LDILE TALTLG des a a u a N Sata em es 55 ch Ss N Sa ar War I wi u yes en Pin reference stone secret of its colour SPESEN, Sa N & Bee an AN a Da anal 54 a A < arte ilih: Savas 1er for this colour Ian 4 STE a Advantageous this is for this colour AL LOXLSTUILKT no de Kal 343 em SE x ze a ZN al \bhrierungse lLemernt VERY additional! ZNES FSaTDbKaL LED LET WS LSCIMMSILE | HOLWeNdi LT x a % X m A an KE = wre of all recognized building blocks Amount of recognized caustics plus the at least el: Pefearenzatein, iet in the sinue of a back calculation colors taking into account the bel zen frequency distribution a YET OS of the parb calibration Sa an \ 1 31 READING NOSES NS 5a a a X of the SKSALEEL MMO A N . x Bart: x WLDOG Ten Ce ZuUS TESTS LE > ‘ ‘ . x \ a cbheshading of a te em naar raster a FESTDSCHATT LETUN SS BALL A ET KATILSIN D building block is assigned an N x x = 5 known color; and var inde BEE u neknown color to TESECTANST}; UNO SWÄT 7 ndem due to this A EL it: mE ES De ne BO Jana Ce Sa way n = avertbeli SE x Sn N ar Tan ap Tan ann 5 bye GEN AND annten frequency sverler Lun AS NO CAS SE FarDKaL LED LOXKIKT x & 3 a EN x - aM Br er Naar ya 1 . STSLNS 4 5 mn any the madness ha 4 N ae SALES ience to a match of the color shading * . & x a. > Ka . 3 a x | SON ade > Da SL x ea er x AN ANNE N + anbe ar $fır + tv SE Ay 3 nen AN SLKSX SAFT A LO Zus Führ LEAMGS SWEK LANE SSL el annten Bau EXCLUSION SL LEST SS CE EEK

En

u em r PR x EI ya an Sm Ss { al Han SETS Re} Az road sete {1 ZAMAST KYEUZUNg, curve) OdSX

So arı br za hr ae © Satan Pr Si ae Wr OS A Ki senbahnset 3 LETSN. X 11 EEG al S8eXr 1 of x Gqı LLC

w HN N it. = F A a st EAN We Fixing stones, GO ax 3 MASS SL As fi nd Lets LS TASTE & 5 = . en N. x x 3 Ta + ekannt an Warıyen ni FE KERN x 3 nn OU En 3 N Ya aranert sr known Fa x} I. > aM ar A N yet Assigning the known color to at least its A a A m A es je n x ss x X Ta 8 Tania £ en sven zstein ısi sSheasandeaere KV en Sy ES Type NO ETAGE x BETEN ZSCCClN LS sbesoand e AS S18 Ka LDILE TALTLG des a a u a N Sata em es 55 ch Ss N Sa ar War I wi u yes en Pin reference stone of its color SPESEN, Sa N & Bee an AN a Da anal 54 a A < arte ilih: Savas 1er for this color Ian 4 STE a Advantageous this is for this color AL LOXLSTUILKT no de Kal 343 em SE x ze a ZN al \bhriesrungse lLemernt VERY additional! ZNES FSaTDbKaL LED LET WS LSCIMMSILE | HOLWeNdi LT x a %

recognized building blocks, i.e. all recognized building blocks, set of recognized blocks plus at least 1: peer group, which in the sense of a backward calculation colors taking into account the bel frequency distribution

a YET OS

the Parb calibration

Sat on \ 1 31 READ NOSES

NS 5a a a X the SKSALEEL MMO A N . x Bart : x x On aa han Alan auasredefnaier ae Sugsardnste understand, ÄAnder AUSGS expresses :; WLDOG Ten Ce ZuUS TESTS LE

> ‘ ‘ . x \ a shading of a te em naar raster a FESTDSCHATT LETUN SS BALL A ET KATILSIN Dbaustein anl N x x = 5 known color assigned; and var inde BEE u neknown color to TESECTANST}; UNO SWÄT 7 ndem due to this

A EL it: mE ES De ne BO Jana Ce Sa way n = avertbeli SE x Sn N ar Tan ap Tan ann 5 bye GEN AND annten frequency sverler Lun AS NO CAS SE FarDKaL LED LOXKIKT

x & 3 a EN x - aM Br er Naar ya 1 . STSLNS 4 5 mn any the delusion ha 4 N ae SALES ience to a match of the color shade * . & x a. >

Ka X aa * Sa 5 myımbarı Se KATıf SE Wach NZ

ML er Sestimnmcen DeKAMnlen Vals VOL .

. 3 a x | SON goodbye >

Da SL x ea er x AN ANNE N + anbe ar $fır + tv SE Ay 3 nen AN SLKSX SAFT A LO To the lead LEAMGS SWEK LANE SSL X OA AUT en

Spa N nuten Fa ben zun sn AT Sn firs of known colors To the rest MILE the recognized:

Maya em SS 3 an TS SR 5 untie Raselie Se a el annten Bau AUSSCHL SL LEST SS CE EEK

Frequency distribution of the known colors and the

Color calibration,

In another variant, the assignment of known colors to the rest of the set of recognized building blocks is carried out exclusively by taking into account the known

Frequency distribution of known colors,

In a further variant, the assignment of known colors to the rest of the set of recognized building blocks takes place exclusively under consideration of the

Color calibration.

> x © YA

T Cr EN a Ya ER er “x rt < N ae ar Cr X Ka a Des TO al ee

LS SMS For information £ actual

Aa + X ad x

SS Can there vr N FA A m x SOON NS 5 een SEN NEN CN IQ Es aa a ATS Aa I N x Sr A

UDere ss" LMU AaNndgEeSOoXN« will be, so that 8118 8NQqJuLilıLgs

A x rn Ara 833 MS Aa Ye Sr A Aare earı EYE CE TO £ Ward AS & KARA

x} An Sn Sr Te NSS De ©} A SS Lan

m m _ an A WALES syn ayıceh a HE en x © MEN

The tenders: VE SLLST LUNG RATE &UON SQL ME VAGS

< es ne _—_ & EN DAS u ANNE ONE & A AN AN N SE CST ei am SCEYUS TOLLES Estimate VS SG SSSNEN W STOSS, LS

> x x > ‘ Ya ayats Au a as iYmar NW + N A ENTE 53 Aa SSL LANA MT ALEX SR SO zat KÜLSLUL AT SEN ALL SL 3 x LS . Yu “> ra vr Am + am an m x” An a. 5 is Zr N

x a3 ex mean < ara aa Sy a N sm TUT an AS

ao Ye Laer ex ae X Ni DAaSLOXST SEM

WS

& on

x AT WOEFCYSS TON TE Ar 5 m Sy Seatarıe any 5 ZN CI AFCHEATSTZOMEES EL ea. DB. that WeEnLTSCENS et LS x NY x N ni Ar Year ee an 9 ae an VAREL all color sound ı ecundgen

ax a

ae he: ayayı te NY oeveann SS] Kan AS ATS A ANA is RN “

3 SS

there are sche Sras

AFC

Ss Sf ESSEN

x x She le Shasande N ea er TEN S ALL

AREA FE

the Sn

A Li TC anrn 1 AG LLL A p

LESS

x x = SAME Hin x Say ee CT Tal x ER Sr x WE) LEG BO EAN SO OCT 88 gr = N U e x „x x HA at least ea an De pairs Sb mia ans

However, the current frequency distribution of the detected building blocks can be easily determined informatively and does not necessarily have to be used for a corrected assignment of the detected

Building blocks can be used to form known building blocks.

For example, current frequency distributions can be stored at least temporarily and/or used to update the known frequency distribution

be used.

For evenly distributed, spatially recognized building blocks under ideal lighting conditions, e.g., under laboratory conditions, the correspondence of the color shade to a known color would be relatively exact. In this case, the recorded ROB values of the assigned color shade would correspond to the RGB values of one of the known colors of the known building blocks, so that the known frequency distribution and/or the color calibration results are theoretically not relevant for the assignment of the recognized building blocks to known building blocks.

should or should be used.

Further preferred embodiments and developments of the

Invention are defined in the dependent claims,

It is preferably provided that the known frequency distribution of the known colors, preferably

continuously and/or regularly updated.

The update of the known frequency distribution can be achieved by user input, by taking into account current frequency distributions of the sets of recognized

Building blocks and/or based on publicly available data

to frequency distributions of known and/or new

Building blocks are created.

It can be provided that each known color of the known building blocks has a variety of color shades

is and/or will be assigned.

The multitude of shades of the respective known color can, for example, be data that is used in combination with an image recognition process to

Assign color shade to a known color.

It is particularly preferred that the frequency distribution of the known colors, known in particular for the at least one reference brick, is two-dimensional and the known colors correlate with known shapes of the known building blocks. In other words, the colors and shapes of the building blocks are known and are represented in particular for the at least one reference brick by the

known frequency distribution related to each other,

It is preferably provided that the selection of the at least one reference brick involves a search for a known building block with a known color and a known shape

contains.

Particularly preferred is the selection of at least one

Reference stone with the search for a known building block

It is preferably provided that the known form

known frequencies assigned to the frequency distribution

Color is present with a probability of over 50%.

With LEGO bricks, the man-heads, trees, plants, roof tiles and railway tracks in particular have a probability of over 50% as yellow man-heads, green trees, green plants, tole roof tiles and grey

Bisenbahn tracks,

It is particularly preferably provided that the assignment of the known colors to the rest of the set of recognized building blocks follows directly or not directly after a color calibration of the known color of the at least one reference brick, wherein the known color serves as the calibration basis,

especially for a color calibration algorithm,

There are basically various algorithms that can be used for color calibration, with the choice of algorithm depending on the number of different color shades or known colors to be assigned. If, for example, only one or two color shades or known colors to be assigned are present, the well-known 3x3 diagonal model can be used. If, for example, four or more different color shades or known colors to be assigned are present, the well-known color calibration algorithm by S. Wolf (S. Wolf; “Color Correction Matrix for Digital Still and Image Imaging Systems”; NTIA Technical Memorandum TM-04-406; 2003) can be used. In general, a higher number of reference stones improves the quality of the

Color calibration is positively influenced.

It is particularly preferred that, preferably by means of the two-dimensional frequency distribution and/or the results of the color calibration, the known shapes of the known building blocks are added to the rest of the set of recognized

Building blocks are assigned.

It is preferably provided that the assignment of the known shapes takes place in conjunction with an image recognition method, in particular one based on an AI method. Such image recognition methods are in and of themselves state-of-the-art

the technology,

It can be provided that the known shapes of the known building blocks are each assigned a plurality of perspective representations of the respective known shapes 18€

and/or will.

The plurality of perspective representations of a particular known shape can, for example, be data which are used in combination with an image recognition method to determine the recognized shape of a known shape

to assign.

It can be provided that absolute dimensions, such as length, width and height, and/or relative dimensions are and/or will be assigned to the known shapes, whereby the dimensions can be used to

to assign a recognized shape to a known shape.

E@ can be provided that at least one grouping of color-identical and/or shape-identical and/or color- and shape-identical recognized building blocks takes place into at least one group, preferably with at least one counting of the recognized building blocks within a group. Thus, a sorting of the recognized building blocks can be performed, thereby providing an overview of the set of recognized building blocks, in particular of an unsorted pile of building blocks.

The identified building blocks are in this

In this case, fully identified building blocks, i.e. both their

Both shape and color are identified.

It is preferably provided that a list is created from the recognized or identified building blocks, wherein the recognized building blocks are each identified by a known color and known shape and a number

have.

In this case, it may be provided that the list is compared with at least one assembly instruction and/or parts list of a kit containing building blocks and is checked for identical color, shape, and number of building blocks. This can, for example, be used to determine the extent to which a selected kit is available and/or soft building blocks to complete the selected building block.

miss.

It could also be provided that the list of at least one parts list of at least one construction salt is assigned if the list is a complete subset of the at least one parts list. This way, it can be shown which kits are compatible with the existing identified

building blocks can be fully assembled.

It is also conceivable that identified missing components of a parts list of a known kit are consolidated and, for example, sent to users

It is also possible that missing modules are automatically

Users can retrieve the identified

You can also procure missing building blocks yourself.

It is also conceivable that functionally equivalent replacement modules could be proposed for the missing modules.

become.

Since the method allows to clearly distinguish known colours and known shapes,

already assembled and/or visually homogeneous appearing assemblies are segmented in such a way that known building blocks

can be identified within this assembly.

A preferred variant of the method according to the invention is described below using an example:

- first, a pile of building blocks is scanned so that scanned or recognized building blocks are obtained;

- four reference elements are selected from the scanned or recognized building blocks and/or found among the scanned or recognized building blocks: e.g., a yellow head, a red roof tile, a gray stone pick and a green tree;

- the respective RGB values of the above-mentioned reference bricks are determined in the scan and compared with the respective RGB values of the known original bricks;

- the required color calibration matrix is determined from the difference between the ROB value pairs (original building blocks va, scan of the reference bricks);

“- the color calibration using the required color calibration matrix leads to an improvement in the agreement of the respective RGB values of the scan of the reference bricks with the ROB values of the original bricks, whereby in the rarest of cases an exact

Agreement will be present;

- the results of the color calibration are applied to the RGB values of the remaining recognized blocks and converted;

- the converted RGB values are compared with the known frequency distribution, so that all recognized building blocks are assigned known colors and are thus identified; optionally, a current frequency distribution of the identified building blocks is determined and compared with the known frequency distribution; if there is an approximate match, the process is terminated; if there are major differences between the current and the known frequency distribution, another

recalculated and/or recalibrated.

Further advantages and details of advantageous variants and developments of the invention emerge from the figures

and the corresponding character descriptions. Showing:

Fig. 1-3 a schematic explanation of an application of the method according to the invention,

Fig. 4 is a schematic representation of the first method step of the method according to the invention,

Fig. 5Sa-5c a set of detected building blocks under different lighting conditions,

Fig. 6a-Sc a schematic representation of the second process step of the process according to the invention,

Fig. 7-8 schematic representations of the third process step of the process according to the invention,

Fig. 9 a known frequency distribution,

Fig. 10 is a schematic representation of the third and fourth process steps of the process according to the invention,

Fig. 11 is a schematic representation of a further development of the method according to the invention,

Fig. 12 shows a schematic explanation of a related application of the method according to the invention,

Fig. 13-16 schematic representations of further developments of the method according to the invention,

Fig. 17 a recognized assembly,

Fig. 18 is a further schematic representation of the fourth method step of the method according to the invention and

Fig. 19a-19c schematic representations of a

Further development of the method according to the invention.

Figs. 1-3 show a schematic explanation of an application of the method according to the invention: Construction kits 6, in particular for toys such as LEGO, are usually delivered with assembly instructions 8 for assembling one or more objects 14. Initial assembly is supported by dividing the construction kits 5 into specific subsets of the construction kits 12 for assembling partial objects 13 and preparing them separately (see Fig. 1). Complete objects 14 are then assembled from the partial objects 13 (see Fig. 2).

The kits 6 are designed to be disassembled and reassembled at a later date. Since many users mix their kits 6 together and thus create a confusing amount of

Building blocks 1, there are several difficulties for

reassembling one or more objects 14, vol. Fig. 3. For example, when there are several construction sets 6 of a similar type (e.g. LEGO City), it is particularly difficult to assign the set of building blocks 1 to specific objects 14. However, the problem also exists with very complex individual

on

kits 6 or if known building blocks 5 are lost.

Fig. 4 shows a schematic representation of the first method steps of the method according to the invention, wherein the set of building blocks 1 is scanned and a set

recognized building blocks 2 is obtained, where each recognized

Building block 2 is assigned a color shade 3.

Figs. Sa-Sc each show the same set of recognized building blocks 2 with color shades 3 assigned to recognized building blocks 2 under different lighting conditions. This schematic representation is intended to illustrate that the problem is that the quality of the scans can be very different or poor, which makes it difficult to differentiate and assign the colors with sufficient accuracy. In the case of Figs. Sa-Sc, the problem is further complicated by the fact that the recognized building blocks 2 are located in a cluster in which

the building blocks overlap each other.

Figs. 6Sa-6Sc each show a schematic representation of the second method step of the method according to the invention under different exposure ratios. In this second method step, at least one reference brick 4 is selected from the set of recognized building blocks Z, wherein the selection is made by identifying a

known form of a known building block 5 of a set

known building blocks 5.

In the present exemplary case, which is also retained in the following figures and figure descriptions, the cylindrical recognized building block 2 is

Reference stone 4 selected.

This recognized building block 2 and/or reference block 4 can have different color shades 3 depending on the lighting situation (cf. Fig. Sa-5c). At the same time, the recognized building block 2 and/or reference block 4 can have one of the known colors from a list of known colors or a color list

for a known module 5, see Fig. 7.

Preferably, there is a work list for each known building block 5, with which the recognized color of the recognized building block 2 and/or the reference block 4 can be compared.

can.

The reference brick 4, which is also a recognized brick 2, can be, for example, a LEGO head of a LEGO man

be, see Fig. S.

Fig. 9 shows a known, preferably relative, frequency distribution 7, which shows the frequency of a known building block 5 and thus also of the recognized building block

2 and/or reference stone 4 in the known colors

reproduces.

Preferably, for each known building block 2, a

such frequency distribution 7 is known.

A Se ae ha > Day x dan Far Var + A TUNCTAT z A ARAM MAC Gen AN hs KEN DLS SA

ES Council San

en © oo yayvar AS ea

be compared.

Fig. 10 shows a schematic representation comprising the third and fourth method steps of the method according to the invention. In the third step, the known color is initially assigned to the reference brick 4. In the fourth step, further known colors are assigned to the remainder of the set of recognized bricks 2 based on the known, preferably relative, frequency distribution 7 of the known colors of the known bricks 5 and/or the results of the

THERE

Color calibration assigned.

This means that from the initially recognized building blocks 2,

a successful comparison with known building blocks 5

identified building blocks 10,

See ae SC QSTr

ea er - bear > 3 en eren known 5 CO TO THE Mande

x

Yamada or TREE en "medium a SZ BAUS TDSLNG WAL ader indirectly on ns 5 ea Car SH ES

TO ANAL AND ANAL

\ esse DSKZMNLS

AN Ka 7

A N HS & STLOL

a8 Are A AA LATS 5

u x N E x = ) Ya says by ST Sa ande ie AFery I x In a special bevo AST siante des ern Sn

Ss x ———_—_— ex SYS Ser er uEi LETKEL LS LAG f

t x poor © im? LE A ALINA

A N 165% LO AAEALU

'Ar we XS x N it een a U ERS EEE ER Nir as SEE XNOTFELLETSN ML

— a Xu © a Say, E NY NANNTEN AS ON n ae ent a, AT Sanders WC Cie DEKAN

One

a CT

x N SAY as nes 5 to the rest

ae

. can X

Ya N en 4 ea ed Ya ss a N © SIT NEN 5X Eat SE St x NEN EV ALL Br SS ATS AL © är VE

Ahynteres

Dear Ama Ss A LAS TRAIN 3 AS s \ 4 $ SH an Si as HH as A SELECTION SD BON SLIM

aaa de Sa

Xor = CAN 5 MX

AS

5 Am LS

N GSW

a ANAL EL

known color assigned to a known shape using the frequency distribution 7 with a probability of

over 50%.

This search for an anticipated reference brick 4 refers to the set of bricks 1, but takes place in

Essentially based on the identified building blocks 2.

Fig. 12 shows a schematically illustrated explanation of a related application of the method according to the invention: With very large quantities of building blocks 1, for example many construction sets 5 with many building instructions 8 and in particular in the form of an unsorted pile, a large number of recognized building blocks 2 are obtained, whereby the problem of the "needle in the haystack" merely generates a second, analogous problem of "a second needle in a second haystack", Val. for this purpose the large number of highlighted recognized

Building blocks 2 in Fig. 12, which is again confusing.

To enable users to grasp the corresponding recognized building blocks 2 as quickly and efficiently as possible, the limitations of human perception must be taken into account and information must be provided in such a way that users can use it appropriately. Schematically shown in Fig. 13, the assembly instructions 8, in conjunction with an underlying parts list 11, are preferably divided into individual subsets of the kit 12.

segmented.

The data obtained, in particular the segmentation obtained, can be processed in a database 16

become.

Ya os wear mi ST As as ea Sa m evaar Cr Fa Sa ea Then MLUCSIS ASS STIL Ya SS according to procedure

er ES nt A 5 = 5 > NH x af F I Ay + N AG SE AMMS ar aM 5 ey 3 mm. ea Se ot & NS, A ns} ES OT ade LA ar AUSANMEN HNOau 1 DES LOT Aa nl x A Do rat nam who aa a 33 ee an 2 PS ex 8 ATS LO eKusS 18; VS LSValıl El SVTKEADNMLEN BAUS SLNGS AS BE LAMSE Bye aa SS 45 Sachırn h A ne Sa N

BREAST SL . TE AAN ; 43 thereby represent AL that

Ay Meat AST at x ea y SCH en 5 5 ey ale ass a

CN AAN 1 El MAR S Sa MARS EN SL N AS AUON

2 ' © Yale yo Yen N FACH abe b ANNE A N Se ame EM st ertle Baustel KVOHESGSILODEN S ind). © Demue STE Snüber who

FOR LS BEUSCSLN VO the

nn

x © NY 5 x each :x Km = A FA

NVE. Ws LAN a 4 aa A aebn f «a al will only be no6 Ale relevant QALTES LS VL LG Sall will only be QdLe TSiLEeVANCSN

Pd

en AN Senn aa Sa en m ea; A aan an yın tan Rausteauın 3 x ei imencges x 5 A ne SS 4x Imensen des x 3 OEsaNnS Sy + > EN el an roof completed unit ' EB, SLR SL Dat 5 355 Todaam 1a körnnen we % $ a wen NS FATPUODMEN ds EAN ARE KOT WEL TSUE UN EAN NAMES SALARZL yiekte 1 ardean ob 76 ke 1 WETCSTL, WA es 5 = x ey Sn > mat 5 Ka Mi SS xy Ag NAT 5 a sr We % $ he COAST TO SL bes N AR ALL Verne, re fun Asıma how Ye 7 ns recognized building blocks seh KARA A ANENS DEW, LEARNING ALL SAUSC Ak A Oaavtns easy Min var PN Re N NY A sn sin Mn 5 N SCAN QSer Mande on building blocks and after S16 TO £ x & A x Y z “ BA aastarn building blocks 15 aa Yemen There Ye As verified blocks were LO, SUCCESS SLVE FROM Gi N x na | ie _ 3 n originally ar CAacsamt a a ar recognized VEN SEM ag] general overview . of the recognized A a A . and as NN __- N SS are removed. N + 2 3 x 305 en Sa hahar am geguchten T Dallimer ATS ES BAUS ZER dm AO TESHODEN,: NECHKEM x Da DS m x es X a Sr Gar A BE ya a ea a is Te & Ay Te OREN Sen MY SENATE a 7 er u... X \ ex en Kausatktzes LLC D81m SCANNING TEST an DW SELL alized x x SL 5 n—_—_- £ artohoane Ray MUS Rn a Aa afundane rei Pat Was ({SMAXr ZOOS IN SQET Middle 1; WLTO of the GEST MMS VS iLevanles

ai x ya eh Asa

x am x, X ® n asien Ka © a Dunı Sal 5 oe hir mn en ON a a ey SrkKnte DBausilein

Er A gr m AN N 5 u AR Ta en A - es} oA ai AN aaa an A sen bir x right Seiis OA LEN a N arbiiebenen er ea anten Ra > ver? N warden VL RDSTLEN STK Dal Te dam time DU ak ; SSe1LSDLeLSt by means of Tartend IE ZN NMZUGTS bern WR x da Asa 7 DLR AL PSMEN s BELEG, N A An “x qegamten = me SiNes 33er 2 = Is 3C j STSLLS ELSE SSL CS TO AM 5 5 3 an x Y. s us Yan ae AN As ey Manrı = ran Ra marvel WS LA LET DEW. LO EL MOL En BUILDING SITE TA ae Ss As FO Sea m Tom eye enlarıe Tinden . WOODS QLS from SausteiLnen © au various 5 x $ > vr ya 3 in ie aa aaa Ra = LEEDS A KARLS or T VERSCHLGECSNEN Sa CO X ex |. 1 en ser ahean ann DA dem Mn 4 AM Sn SSTSNEN 5 3 can Nu CN SD, A Da . . a x x DS 3 pr % % = A _- "—. An an an . a Rn 5 Wahyrarnt Ka ch = 5 es A an RN EN SuM sed Sm = 5 the probability is that s10 G6SLLMMLES OD e a | m & En EA Kan PS Pi. ns nn NO de an A ei mas Say ei £ Say va TH a ar SHE SAUSSE 5 ESTULG POSITIONED N Kan. x x 3 en RN A RN Yo AA ea SumaaT x Seven se Del ABS GLTM AUuUSatZes IL LUX QeT MASSE WET SVENS x % Ya N x a sd & Ne aa lan eÄnnen ur} am Bine een Sr 15 verarpeitel werden KOMMEN UT Her AOCSE Sinne VOSTDITÜZUN Sa Hammel Sarmicdante Rlenmäfßlier 1dente Ba wa HE MALE OL LEN AL STE X & Se X de HNÜ& DSaUSCSL can be carried out. Preferably, the comparison is carried out using a list created from the recognized building blocks 2, whereby the recognized building blocks 2 are each identified by a known color and known shape and have a number.

a N arbiieben er ea anten Ra > ver? N warden VL RDSTLEN STK Dal

Ay hi nk Bazar ine uf an AN Sn aA OF a Ser A ass en R AU ıkbar, users at Te dam Zeit DU ak ; SSe1LSDLeLSt by means of Tartend IE ZN NMZUGTS bern WR x da Asa 7 DLR AL PSMEN s BELEG, N A An “x

5 U

We are playing

= me SiNes 33er 2 = Is 3C j ELSE SSL CS TO AM 5 5 3 an x Y. s us Yan ae AN As ey Manrı = ran Ra marvel WS LA LET DEW. LO EL MOL En CONSTRUCTION SITE

TA ae Ss As FO Sea m Tom eye enlarıe Tinden . WOODS QLS from SausteiLnen © from various

5 x $ > vr ya 3 in ie aa aaa Ra = LEEDS A KARLS or T VERSCHLGECSNEN Sa CO X ex |. 1 en ser ahean ann DA whom Mn 4 AM Sn SSTSNEN 5 3 can Nu CN SD,

A There . . a x

x DS 3 pr % % = A _- "—. An an an . a Rn 5 Wahyrarnt Ka ch = 5 es A an RN EN SuM sed Sm = 5

The probability is that s10 G6SLLMMLES OD

e a | m & En EA Kan PS Pi. ns nn NO de an A

ei mas Say ei £ Say va TH a ar

SHE SAUSSE 5 ESTULG SET N Kan.

x x 3

x Ü x 5 Pe Sinse da en RN A RN Yo AA ea SumaaT x Seven se Del ABS GLTM AUuUSatZes IL LUX QeT MASSE WET SVENS x % Ya N x a sd & Ne

"+ N Sad a A & ©

\ veararheitet aa lan eÄnnen ur} am Bine een Sr 15 verarpeitel Werden COMMEN UT Her AOCSE Sinne VOSTDITÜZUN Sa Hammel Sarmicdante Rlenmäfßlier 1dente Ba wa

HE MALE OL LEN AL STE X & Se X de HNÜ& DSaUSCSL

can be done.

Preferably, the comparison is carried out using a list created from the recognized building blocks 2, whereby the recognized building blocks 2 are each identified by a known color and known

are identified in a form and have a number. This way,

For example, it can be determined which and/or how many known building blocks 5 gof. would have to be acquired in the course of a replacement purchase. The acquisition can be automated and/or

be done manually,

Fig. 17 shows an assembly 15, which consists of a set of recognized building blocks 2 with associated different color shades 3. Here, the recognized building blocks 2 are located in an assembled

Form-fitting condition.

Fig. 18 shows a further schematic representation of the fourth method step of the method according to the invention, wherein the recognized modules 2 of an assembly 15 on

Basis of color calibration using reference grade 4

known colors are assigned.

Fig. 19a-19c show schematic representations of a further development of the method according to the invention, wherein by means of known dimensional ratios or relative dimensions, e.g. length: width: height, assemblies 15 can be segmented into their individual recognized building blocks 2. In this way, visually homogeneous appearing assemblies 15 can be segmented to such an extent that the presence of known building blocks 5 in

such an assembly 15 can be derived.

Fig. 19a illustrates that the boundaries between the detected building blocks 2 are not easily recognizable, for example, due to poor scan quality. Fig. 19b illustrates that the boundaries can be identified using known dimensional ratios and/or color calibration. Fig. 190 8511 illustrates the state with identified boundaries of the detected

Represent building blocks 2.

10 11

12

13 14 15 16

25

Legend

to the reference numbers:

(Set of} building blocks {Set} of recognized building blocks Assigned color shade Reference brick Known (r} building block {({s} kit/kits Known frequency distribution Building instructions Two-dimensional frequency distribution Identified building blocks Parts list of at least one building block Subset of at least one building block Sub-object (s)} Object Assembly {n} Database

Claims (1)

20230818 256 claims L. Method for identifying a set of building blocks {1} each with a shape and a color, where the following steps must be carried out: - scanning the set of building blocks (1) and obtaining a set of recognized building blocks {2} each with an assigned color shade (3), Selecting at least one reference brick (4) from the set of recognized bricks ({2) by identifying a known shape of a known brick {5}, - Assigning a known color to the at least one reference stone {4} and preferably performing a color calibration based on the at least one reference stone (4), - Assigning known colors to the rest of the set of recognized building blocks (2) based on a known, preferably relative, frequency distribution (7) of the known colors of the known building blocks {5} and/or the results of the Color calibration. Method according to claim 1, wherein the known frequency distribution {7) of the known colors, preferably continuously and/or regularly, is updated. p. 8. 27 Method according to one of the preceding claims, wherein each known color of the known building blocks (5) is assigned a plurality of color shades and/or will. Method according to one of the preceding claims, wherein the known frequency distribution {7} of the known colors is a two-dimensional frequency distribution (9) and the known colors are associated with known shapes of the known building blocks (S) correlate, Method according to one of the preceding claims, wherein selecting the reference brick (4) involves a search for a known building block (5) with a known color and a known shape. Method according to one of the preceding claims, wherein the known color assigned to the known shape by means of the frequency distribution {7} with a probability of over 50%. Method according to one of the preceding claims, wherein the assignment of the known colors to the rest of the set of recognized building blocks (2) follows a color calibration of the known color of the reference brick (4), wherein the known color serves as a calibration basis, in particular for a Color calibration algorithm. Method according to one of the preceding claims, wherein, preferably with a two-dimensional Frequency distribution (9) according to claim 4, the AL, We S 14. 28 known forms of the known building blocks {5} are assigned to the rest of the set of recognized building blocks ({2}) become. Method according to the preceding claim, wherein the assignment of the known shapes in connection with a Image recognition process is carried out, Method according to one of the preceding two claims, wherein the known shapes of the known building blocks (5) are each assigned a plurality of perspective representations of the respective known shape and/or becomes. Method according to one of the preceding claims, wherein the known forms are each assigned absolute and/or relative dimensions are and/or will be assigned. Method according to one of the preceding claims, wherein at least one grouping of color-identical and/or color-identical and/or color- and form-identical recognized building blocks (2} is carried out into at least one group, preferably wherein at least one counting of the recognized building blocks (2) within a group. Method according to one of claims 8 to 12, wherein a list is created from the recognized building blocks (2), wherein the recognized building blocks (2) are each identified by a known color and known shape and have a number. Method according to the preceding claim, wherein the List with at least one parts list of at least one kit (11) is comprehensively matched building blocks and from’ color- Building blocks Procedure List of a building 28 and form-identical and numerically identical is checked. according to the preceding claim, wherein the at least one parts list of the at least atzes {11} is assigned when the list a complete subset of at least one Parts list of at least one kit {11}.