WO2019221618A1

WO2019221618A1 - An electronic device for ordering eyewear and system and method for ordering eyewear comprising such an electronic device

Info

Publication number: WO2019221618A1
Application number: PCT/PL2018/050022
Authority: WO
Inventors: Eryk MAKOWSKI; Michal JARON; Stanislaw Nowak
Original assignee: Optimy Sp. Z O.O.
Priority date: 2018-05-18
Filing date: 2018-05-18
Publication date: 2019-11-21

Abstract

An electronic device (100) for ordering eyewear comprising first camera (201a) for taking pictures of the customer eye region, a control unit (202) comprising processor (203) and memory means (204) for managing components of the device (100), the control unit (202) being adapted to acquire and process data on lenses required for the customer, communication means (207) for sending orders characterized in that it further comprises at least one additional camera (201b) for taking pictures of the customer face, disposed at different height than the first camera (201a), means (209) for reading marking from eyeglass frames (400) for accessing it's model data from the database (302) wherein at least said two cameras (201a, 201b) are calibrated so as to take at least two pictures of the customer face which match geometrically and wherein the control unit (202) is adapted to process said at least two pictures acquired by at least said two cameras (201a, 201b) so as to find characteristic points of the customer face and is adapted to calculate pupillary distance PD and half- height SH based on the at least 2D model data of the eyeglass frame (400) from the database (302) and said characteristic points of the customer face, and wherein the control unit (202) is further adapted to automatically complete an order for eyewear. The invention also relates to a system and a method for ordering eyewear.

Description

AN ELECTRONIC DEVICE FOR ORDERING

EYEWEAR AND SYSTEM AND METHOD FOR ORDERING EYEWEAR COMPRISING SUCH AN ELECTRONIC DEVICE

[0001] This invention relates to an electronic device for ordering eyewear and system and method for ordering eyewear comprising such an electronic device. The invention is applicable in the field of taking measurements performed by an optician / optometrist in order to let optical workshop swiftly and most accurately get parameters aiming at defining optical center and level of progression on progressive lenses before preparing ordered eyewear.

Prior art

[0002] The eyewear market requires very high standard of customer service, ach customer should be individually served to meet its own needs. Such services require for e ample a possibility of measuring the distance between the pupils. When the number of customers in a stationary shop increases, it becomes more and more difficult to keep up the high standards of customer service. The optician store staff is often short of time. Moreover, the orthoptist or technical staff member will make use of relatively unprecise and unadapted tools to measure pupillary distance PD .

[0003] ven though there are some fairly costly digital tools available pupilometer , the practice still often is to use a ruler and to mark lenses. This often results in badly centered lenses - even when delivered by reputable optical retailers.

[000 ] If the customer needs to purchase progressive lenses another important parameter must be determined, namely the height between pupils and eyeglass frame bottom edge SH . Again, the measuring of segment height SH is always done in the simplest way the optician places frames on the user s nose and uses a small ruler to measure height vertically under the pupil. This method often called DAT M is the most widely used.

[000 ] Sometimes the method called bo ing is used, that uses the lower tangent to the eyeglass frame, ut this may result in even less precise result.

[000 ] Above-described methods may frequently result in releasing unsuitable spectacles to a user who will either live with unprecise correction resulting from wrongly defined optical center in mounted lenses or involve return to optical store to swap spectacles - that involves additional delay and cost to both customer and optician.

[0007] Moreover, wrong measurement of the PD and SH still happen very commonly when purchasing glasses.

[0008] In addition, the measurement-taking process is a long and boring procedure, that demotivates people from consulting eye-doctors as often as they should to preserve their vision from deterioration.

[0009] Moreover, nowadays more and more customers prefer self-service over contacting support agents. This phenomenon relates also to optician services. It has been found already in other brands that the self-service is recognized as the fastest and the most cost-effective way to provide customer support.

[0010] Self-service systems in the form of kiosk are known from the priori art, however they are not adapted for places where there are space constraints. Moreover, known kiosks do not provide measurements of pupillary distance PD and half-height SH of good quality as well as allow choosing only a limited number of eyeglass frames available within the kiosk. Another disadvantage is that it is often not possible to be measured while wearing chosen eyeglass frames, thus several actions have to be performed by the user before he can order chosen eyewear.

[0011] The object of the invention is to provide a compact self-service device and system for ordering eyewear while offering high accuracy measurement of pupillary distance PD and half-height SH and moreover simplifying and making the process of ordering new eyewear more efficient.

[0012] In other words, the invention is aimed at providing an effective self-service tool and system for ordering eyewear while getting two critical measurements, namely pupillary distance PD and half-height SH, in a hassle-free funny way and with the accuracy up to 0,5 mm.

[0013] According to the first aspect, the invention relates to an electronic device for ordering eyewear comprising a first camera for taking pictures of the customer eye region, a control unit comprising processor and memory means for managing components of the device, a user interface for providing inputs from the customer, a display for prompting the customer , means for accessing a database with at least 2D model data related to eyeglass frames upon purchase characterized in that it further comprises at least one additional camera for taking pictures of the customer face, disposed at different height than the first camera, means for reading marking from an eyeglass frame for accessing model data of said eyeglass frame from the database, wherein at least said two cameras are calibrated so as to take at least two pictures of the customer face which match geometrically and wherein the control unit is adapted to process said at least two pictures acquired by at least said two cameras so as to find characteristic points of the customer face and is adapted to calculate pupillary distance and half-height based on the at least 2D model data of the eyeglass frame from the database and said characteristic points of the customer face.

[0014] By providing a second camera a parallel process of acquiring image data is obtained which makes measuring process shorter. Moreover, by disposing the second camera at different known height a higher accuracy of measurements is achieved.

[0015] Thanks to the fact that both cameras are calibrated so as to take pictures of the customer face which geometrically match, a higher accuracy of measurements is achieved.

[0016] By using means for reading marking from a chosen eyeglass frame in order to access its model data from the database it is possible to take pictures while wearing chosen eyeglass frames and no additional equipment is required to measure pupillary distance and half-height.

[0017] Advantageously, the device comprises a frame and a two-way mirror arranged on at least a part of a front side of the frame so as to cover said at least two cameras and constitute the display surface.

[0018] The electronic device for ordering eyewear thanks to the presence of the two- way mirror eases the trial of new eyeglass frames, allowing shortsighted users to see themselves immediately while wearing new spectacles.

[0019] By providing a two-way mirror on the front side of the device for ordering eyewear a compact object looking like a typical mirror to be hang is achieved which enhances its use in optician shops in addition to typical shelves with eyeglass frames .

[0020] According to the second aspect, the invention relates to a system for ordering eyewear characterized in that it comprises:

- a backoffice system

- an external server for managing orders - an electronic device for ordering eyewear comprising

- first camera for taking pictures of the customer eye region

- a control unit comprising processor and memory means for managing components of the device, the control unit being adapted to acquire and process data on lenses required for the customer

- communication means for sending orders

- at least one additional camera for taking pictures of the customer face, disposed at different height than the first camera

- means for reading marking from eyeglass frames for accessing its model data from the database

- wherein at least said two cameras are calibrated so as to take at least two pictures of the customer face which match geometrically and

- wherein the control unit is adapted to process said at least two pictures acquired by at least said two cameras so as to find characteristic points of the customer face and is adapted to calculate pupillary distance PD and half-height SH based on the at least 2D model data of the eyeglass frame from the database and said characteristic points of the customer face.

- wherein the control unit is further adapted to automatically complete an order for eyewear.

[0021] According to the third aspect, the invention relates to a method of ordering eyewear, comprising steps of:

- taking a first picture of the customer eye region from a first height

- acquiring at least 2D model data related to eyeglass frames upon purchase

- acquiring data on lenses required for the customer

- sending order

characterized in that before sending an order it further comprises steps of

- taking at least a second picture of the customer eye region from a different height

- reading marking from eyeglass frames for accessing its model data from the database wherein said at least two pictures of the customer face match geometrically

- processing of said at least two pictures acquired by at least said two cameras so as to find characteristic points of the customer face and - calculating pupillary distance PD and half-height SH based on the at least 2D model data of the eyeglass frame from the and said characteristic points of the customer face automatically completing the order for eyewear.

[0022] Advantageously, inputs from the customer are provided with the use of a user interface built in the device for ordering eyewear.

[0023] Advantageously, inputs from the customer are provided with the use of a user interface of the mobile device.

[0024] Advantageously, acquiring data on lenses required for the customer is performed by scanning a prescription with a scanner.

[0025] Thanks to the use of two cameras and accurate calibration of them as well efficient image processing algorithms measuring with high accuracy of so-called half height to adjust progressive lenses to patient's visual parameters and pupillary distance is enabled.

[0026] By embedding the device for ordering eyewear into a system for ordering eyewear comprising, among others, a back office as well as an external server, high standards for customer service in stationary optician shops are set and maintained.

[0027] Thanks to output commands displayed on the two-way mirror covering the device for ordering eyewear unassisted purchase of glasses by customers in a self- service mode is enabled.

[0028] Thanks to the intelligent device for ordering eyewear with the two-way mirror and mobile application the process of selecting and purchasing eyeglass frames and lenses is facilitated in stationary optician shops - in a traditional way as well as a self- service.

[0029] The device for ordering eyewear contains built in cameras that allow potential customers to take pictures and send them to customers' mobile devices via different media.

[0030] The device for ordering eyewear enables biometrical measurement of important elements of the face (especially distance between pupils) using high quality pictures.

[0031] The system for ordering eyewear allows the customer to purchase the eyeglasses in self-service manner via cell phone or dedicated self-service device. [0032] It allows for the most accurate measurement of both above-mentioned measurements that are critical to adapt optical lenses mounting to selected eyeglass frames.

[0033] The electronic device and the system for ordering eyewear according to the invention allow an interactive electronic shopping experience that can be accessed in store.

[0034] Additionally, thanks to the presence of a connection between the electronic device for ordering eyewear and the external server the system for ordering eyewear allows for promptly transferring indications required for the mounting of lenses on eyeglass frames from store to workshop.

[0035] The electronic device for ordering eyewear according to the invention, namely "a digital mirror" is designed for automating and fine-tuning measurements necessary to match optical lenses with eyeglass frames, to provide end-users with perfectly fitted eye-glasses.

[0036] A method for ordering eyewear according to the invention allows to automate preparatory works to fine-tune the cutting and mounting of ophthalmic lenses on eyeglass frames thanks to the process of automatically measuring pupillary distance (PD) and half-height (SH).

Brief description of the drawing

[0037]

Fig.l is a general view of the electronic device for ordering eyewear, namely 'a digital mirror', according to the invention placed at the optician store;

Fig.2 is a block diagram of the system for ordering eyewear according to the invention; Fig.3 is a block diagram of the device for ordering eyewear, namely the digital mirror, according to the invention;

Fig.4 is a flowchart of the eyewear ordering process;

Fig.5 is a flowchart of the PD calculation algorithm;

Fig.6 is a flowchart of the SH calculation algorithms;

Fig.7 is a general flowchart of PD and SH calculation process;

Fig.8 illustrates exemplary pattern sheets for the calibration process

Fig.9 illustrates geometrical relations between points in 3d space in the camera calibration process; Description of detailed embodiments of the invention

[0038] As described above the electronic device and the system for ordering eyewear according to the invention are aimed at helping the customer with eyeglass frame and lenses selection by prompting for user information and automatically deriving required optical parameters based on taken pictures and sending an order. They also allow to order new eyeglasses without assistance of the optician.

[0039] Fig.2 is a block diagram of the system 100 for ordering eyewear. It comprises an electronic device 200 for ordering eyewear according to the invention, also called a digital mirror 200, which is the main device of the system 100 for ordering eyewear. Among other functionalities, since the electronic device 200 for ordering eyewear is a central module, it communicates with all other modules of the system 100, including receiving the requests from other modules and responding to them. The digital mirror 200 serves for selecting and gathering details of an order. It also serves for measuring pupillary distance PD and half-height SH of a person wishing to purchase new eyeglasses.

[0040] As shown in Fig.l, the electronic device 200 for ordering eyewear, namely the digital mirror 200 has a compact form. In particular, the electronic device 200 for ordering eyewear has a frame 230, preferably of substantially cuboid shape. The frame 230 is at least partially covered by a two-way mirror 219 which causes the digital mirror 200 to look like a typical mirror. The digital mirror 200 might be mountable to various places in the optical store. As shown in Fig.l, the digital mirror 200 may be located right where customers would try on eyeglass frames 400 such as a display area. It should be noted that customers try on eyeglass frames 400 from a certain distance d which allow proper measuring process. In reality, said distance d is comprised within a certain range of allowable distances.

[0041] The digital mirror 200 comprises a computer hardware in the form of a control unit 202, comprising a processor 203 for controlling components of the digital mirror 200 with the use of various applications and memory means 204 for storing code of various software applications running on the processor 203 as well as various data. Memory means 204 might be any type of memory like hard drive, solid state drive or memory card. One of the applications running on the processor 203, namely a digital mirror application 215 enables registering / logging in / logging out of the system 100 with the use of a given username and password. Only one user might be logged in at a given time. Other exemplary applications which run on the processor 203 are: measurement application 213 for measuring and calculating characteristic values PD and SH, admin interface application 214 for administrative and maintenance access. There can be also an application 216 serving hardware interfaces as well as FrontEnd application 217 for guiding the customer through the purchase process and BackEnd application 218 for communication with BackOffice 300.

[0042] As already mentioned in one embodiment the digital mirror 200 according to the invention may comprise a frame 230 and at least one mounting plate (not shown) for attaching internal components. As shown in Fig. 2 and 3, inside the digital mirror 200 at least two cameras 201a, 201b are disposed at two different heights for taking pictures of a person standing in front of it. Advantageously there might be three cameras (201a, 201b, 201c) disposed at different heights for making measurements more accurate. Pictures taken by the cameras (201a, 201b) are used to determine the pupillary distance PD and half-height distance SH of the customer by a measurement application 213 as explained later. Those two parameters are used to design properly different types of lenses like mono-focal lenses and progressive lenses.

[0043] The digital mirror 200 may also comprise an RFID reader 209 connected to the control unit 202 for reading marking means 401 ( tags/ tag chips 401) attached to different eyeglass frames 400. The customer takes selected eyeglass frame 400 off the shelf and puts in proximity of the RFID reader 209, advantageously while wearing it on one's nose, to cause the RFID reader 209 to read the information from the chip tag 401 disposed on the selected eyeglass frame 400. In other embodiment means 209 for reading marking from eyeglass frames 400 may have another form depending on the type of the marking of the specific eyeglass frame 400.

[0044] The digital mirror 200 comprises advantageously an internal built-in display means 206 connected to the control unit 202 for displaying messages and instructions to the user of the digital mirror 200 in order to help him to get thorough the whole purchase process. In the idle state, the internal display means 206 displays advertisements or other audio and video material. The two-way mirror 219 arranged on the frame 230 of the digital mirror 200 constitutes a display surface covering said display means 206. [0045] The internal built-in display means 206, for example a screen 206 enables displaying information read for identified eyeglass frames 400, for example dimensions, price, picture and accessible options (for example eyeglass frame color, material etc., type of lenses) from internal storage means 204. Data related to a selected eyeglass frame 400 are associated directly with the information read from the tag chip 401.

[0046] As mentioned earlier, in order to get the look of a real mirror the front side of the device 200 for ordering eyewear according to the invention is covered by a two-way mirror 219 for example a Venetian mirror 219. The two-way mirror 219 on one hand allows the purchaser standing in front of it to try on various eyeglass frames 400 and on the other hand to see and read messages displayed on the internal display means 206. As shown in Fig.2 and Fig.3, the device 200 for ordering eyewear according to the invention, namely the digital mirror 200 comprises advantageously a user interface 211, for example a touch pad or touch screen or a keypad. In other embodiment, the user interface 211 might be implemented as a part of an external mobile device 500, for example as a user interface of a mobile phone 500. The user interface 211 allows inputting various information required to place an order, for example details of refraction, namely distance vision, near vision information, such as a spherical correction, cylindrical correction, axis, prism, base as well as mailing details, payment details. The customer may also make with the user interface 211 selections related to the details of chosen eyeglass frame 400 and lenses. In another embodiment, the information from the customer may be inputted in various manner, for example as voice messages or thumb scan or eye scan etc.

[0047] As shown in Fig.3, also, there might be a modem 207 connected to the control unit 202 for communicating with external devices. Advantageously the modem 207 is any type of transceiver communicating via wireless connection or wired connection. Especially modem 207 supports communication with the mobile device 500 and the BackOffice system 300.

[0048] Moreover, the digital mirror 200 advantageously comprises a payment interface 208 (for example build -in card terminal 208) for enabling payment for the service by the customer and a scanner 212 for scanning prescriptions (as shown in Fig.3) for acquiring alternative or additional data relating to the eyes of the customer and eyewear to be ordered. The payment interface 208 is connected to the control unit 202. The scanner 212 is also connected to the control unit 202. It is advantageously an optical scanner 212. In another embodiment the payment interface 800 and the scanner 801 for scanning prescriptions are external components (as shown in Fig.2), namely external devices connected to the digital mirror 200.

[0049] As shown in Fig. 3, the digital mirror 200 can also use an electronic card reader, namely RFID reader 209 for acquiring personal data required for logging in or for creating an account, like address, email details, contact details. Such data may be stored on an electronic card provided with an electronic chip.

[0050] As shown in Fig. 2, the system 100 for ordering eyewear according to the invention further comprises a BackOffice system 300 which among other comprises a database 302 with information on eyeglass frames 400 upon purchase. It also comprises a user interface 303 implemented for example as a set of xml documents with declarative user interface logic description and html and JavaScript documents defining user interface. Flowever, the person skilled in the art will appreciate that another implementation of the user interface 303 is also possible. Said BackOffice system 300 serves among other for receiving data from the digital mirror 200 relating to the customer order, obtained measurements, prescription scan, customer contact data and coordinating the order handling process between every involved party (customer, workshop, lens manufacturer, shop, etc.).

[0051] The system 100 for ordering eyewear also comprises a set of eyeglass frames 400 with tag chip 401 on (or alternatively other type of marking means 401) and a customer mobile device 500 with a user interface application 501. The mobile device 500 comprises at least hardware means for QR code scanning, namely QR code scanner 502, user interface application 501 and a QR code reading application 503. It allows setting up a connection between the digital mirror 200 and the mobile device 500. Namely a QR code to be scanned is displayed on the front part of the digital mirror 200 once the customer inputs into its mobile application a request for setting up a connection with the digital mirror 200. The internal QR code generator 205 embedded in the control unit 202 of the digital mirror 200 generates a unique QR code to be displayed. The customer approaches the digital mirror 200 for scanning said QR code.

[0052] The system 100 for ordering eyewear contains also a web application 701 for handling orders running on an external server 700, wherein it enables logging in / logging out of the system 100 via web browser by using a username and a login. It might be accessed by the customer or staff of an optician store or by glasses manufacturers. For example, the customer, with the use of its mobile device 500 may see, change and manage a list of his orders, details of any of her/his orders, including status, date, final price, ordered frames and lenses, delivery address while the manufacture may see and modify details of any issued order. Thanks to the order handling application 701 running on an external server 700 the customer is notified using SMS and/or email when her/his order state is changed by glasses manufacturer using settings (enable/disable, telephone number, email address).

[0053] Now, eyeglasses ordering process will be described in relation to Fig. 4. The person wishing to purchase new glasses visits the optical store or other type of retail establishment in order to select and order a set of eyeglass frames 400 and lenses that are fitting well. The optical store includes special display areas, for example in the form of various shelves for displaying available models of eyeglass frames 400. Typically, the purchaser takes the eyeglass frame 400 that he is interested in and try it on before a normal mirror. In the system 200 for ordering eyewear according to the invention, the purchaser does not have to change his habits. The only difference is that instead a normal mirror he tries on selected frames 400 in front of the digital mirror 200 according to the invention.

[0054] Firstly, the customer switches on or activates the digital mirror 200 via the user interface 211, for example by touching the touchscreen or by pressing an appropriate button on the keypad. In other embodiment, the customer activates an application on his mobile device 500 which communicates with the digital mirror 200, particularly via a user interface application 501 running on the mobile device 500 (as shown in Fig.2). Another software application, namely a display application 216 stored in the memory means 204 causes the internal display means 206 to display instructions for the customer relating to next actions to be performed by him.

[0055] As a next step logging in is required by inputting username and a password via available user interface 211. This process is performed via digital mirror application 215 running on the digital mirror 200. Advantageously, this step might be a last step before inputting payment details. [0056] Next, the customer picks-up a chosen eyeglass frame 400 and encouraged by the displayed instruction he waives the eyeglass frame 400 in front of the digital mirror 200, which allows, thanks to the RFID reader 209, for identifying which eyeglass frame 400 model is involved in the measurement-taking process. The RFID reader 209 reads the information included in the tag chip 401 on the picked eyeglass frame 400 and based on this the device 200 for ordering eyewear, namely the digital mirror 200, access external data base 302 for more information about said eyeglass frame 400.

[0057] Advantageously, the frame model 400 appears on the screen 206 in a corner of the digital mirror 200. Also, another associated information is displayed as explained later on.

[0058] As already mentioned, all eyeglass frames 400 in the optical store are provided with a special marking 401 able to be read automatically by the digital mirror 200. For example, the marking 401 may be in the form of an RFID tag 401. The tag 401 include an individual code of a specific eyewear frame 400. The individual code is associated with a set of data concerning a specific frame model 400, for example its 3D model as well as data on relative coordinates of the characteristic points of the specific frame model 400. Other data about specific frame 400 model might be color, material, possible patterns etc. A precise data on measurement of selected elements of eyeglass frames 400 are stored in the external database 302 and are used by measuring application 213 involved in image-processing to analyze a picture of the face of an ophthalmic lenses user taken with the selected eyeglass frame being worn on. Said picture with additional information is then converted for measuring a distance on the picture in order to obtain the PD and SH with accuracy up to a hundredth of mm. The external database 302 of eyeglass frame 400 details is maintained on the BackOffice server 300 and regularly synchronized with digital mirror 200 internal storage means 204, to keep the internal storage up to date.

[0059] In other words, on the customer side, the method of ordering eyeglasses involves among other steps picking-up a chosen eyeglass frame 400 and waiving it in front of the digital mirror 200 for accessing information about said eyeglass frame 400. Next step is wearing it in front of the digital mirror 200.

[0060] Encouraged further by the messages displayed on the screen 206 the customer faces the digital mirror 200 at an indicated distance d with the eyeglass frame 400 worn on. Image acquisition happens at that time: at least two cameras 201a, 201b mounted at different heights behind the Venetian one-sided mirror 219 take a series of pictures of the customer with the eyeglass frame 400 on. The cameras 201a, 201b capture at first basic pictures, next high-quality pictures for biometric measurements.

[0061] Specialized software, namely measurement application 213 then precisely measures pupillary distance PD and half-height SH based upon eyeglass frame 400 characteristics stored in the database 302 and other information acquired from the pictures taken by at least two cameras 201a, 201b.

[0062] Basic pictures might be shared by the customer with others for example by sending pictures via email using digital mirror application 215 on the digital mirror 200. It is possible to save and cancel any picture taken by cameras 201a, 201b using digital mirror application 215. The digital mirror application 215 also allows to modify pictures, for example by filtering on the customer request. Said request can be input by the user interface 211 build-in the digital mirror 200 or by using user interface being integral part of the mobile phone 500 as well as the user interface application 501 running on the mobile phone 500. Once the pictures are taken the customer may share the photo through text messaging, email, social networking websites or other online posting. In another embodiment, having pictures taken while trying on eyeglass frames 400 and displaying them later on the screen 206 it allows the user to see himself/herself while wearing glasses and also allows for side by side comparison of different eyeglass frame 400 models.

[0063] To complete the order the customer provides lacking information, particularly prescription details, for example by scanning it with the use of the optical scanner 212 built in the digital-mirror 200. The customer selects further details of the chosen eyeglass frame 400, details of the lenses type etc. as well as provide payment details.

[0064] Regarding lenses details, the customer may choose via the user interface 211 different types of lenses and additional parameters like the material of the lenses (glass, plastic, polycarbonate, trivex), coating type (hardening - anti-scratch, anti-reflecting), high refractive index (for large diopters), computer lenses, single vision/progressive lenses etc.

[0065] As mentioned earlier, to provide prescription details and payment details the customer needs to be registered before he begins the whole process of ordering eyewear or have to register at the end of the process by creating his own profile or logging in into the digital mirror application 215 using an existing profile.

[0066] Regarding optical characteristics of the customer's eyes, it is assumed that the customer knows one's refraction parameters or ideally, has one's eye-doctor's prescription with oneself. In such a case, as already mentioned, the customer is asked to wave one's prescription under the scanner 212. If not, the customer inputs it manually in accordance with the displayed instructions using the user interface 211. For example, the customer enters manually optometric data known to him such as: pupillary distance PD, the diopters for both distance and near vision of both eyes, spherical, cylindrical, axis (for cylindrical lenses), add (for progressive lenses) etc. Generally, such information may be obtained from a recent doctor's prescription or past prescription. The user may provide different types of prescriptions (e.g., reading, distance, bifocal, trifocal, progressives, etc.). This gathered information may be saved in the user's profile for later access to be used to order eyewear.

[0067] In one embodiment data related to mailing details and generally to the customer account details may be acquired automatically from an electronic card reader, namely RFID reader 209, the customer holding any type of electronic card with a RFID readable chip. In another embodiment, the account details may be provided earlier via the user interface application 501 open at the mobile device 500 like a smartphone.

[0068] Once the details of the chosen eyeglass frame 400 and chosen lenses have been provided and other inputs completed the process ends by the payment step. Information displayed on the screen 206 allows to choose the payment step to begin. The payment may be performed using special hardware like payment interface 208 integrated in the digital mirror 200 or the payment interface 800 provided externally and linked via an available network with the digital mirror 200, for example an external card terminal 800. It is also possible to choose a payment option - cash on delivery.

[0069] The ordering process continues by providing the customer with payment confirmation and transaction details as well as sending the order to the manufacturer. In one embodiment, the customer may be provided with a date on which the ordered eyeglasses will be delivered to the customer. Next, all required information is forwarded from the digital mirror 200 to a chosen manufacturer who executes the order and produces ordered lenses and place it within the chosen eyeglass frame 400. The flow chart of steps of the method of ordering eyewear according to the invention has been illustrated in Fig.4.

[0070] The customer can manage later on its order via a user interface application 501 which is running on its mobile device 500. It allows the customer to reach the order details forwarded from the digital mirror 200 to external server 700 for storing and managing by manufacturer. The web application 701 for handling orders running on an external server 700 enables logging in / logging out of the system 100 via web browser by using a username and a login. It might be accessed by the customer or staff of an optician store or by glasses manufacturers. For example, the customer, with the use of its mobile device 500 may see, change and manage a list of his orders, details of any of her/his orders, including status, date, final price, ordered frames and lenses, delivery address while the manufacture may see and modify details of any issued order.

[0071] Now a method for acquiring pupillary distance PD and half-height SH will be described in referent to Fig.5 and Fig.6.

[0072] Two sources of data are used for calculating pupillary distance PD and half-height SH according to the invention. On one hand the pupillary distance PD and half-height SH are calculated based on the individual set of data associated with the eyeglass frame 400 that the customer registered via RFID reader 209 and wears later on. Data related to the selected eyeglass frame 400 may be used as reference points or as a known dimension. On the other hand, the pupillary distance PD and half-height SH are calculated based on characteristics points of the customer face found on the pictures taken by the device 200 for ordering eyewear.

[0073] As already mentioned, at least two cameras 201a, 201b built in the device 200 for ordering eyewear, namely the digital mirror 200, are involved in taking pictures of the customer standing in front of the digital mirror 200 at an indicated distance d. The digital mirror 200 displays on the screen 206 visual and/or written instructions that direct the customer through the steps required to complete proper measurements of predetermined optical parameters. For example, displayed instructions direct the customer to take a specific position such that both eyes of the customer are substantially in a plane parallel to the surface of the two-way mirror 219 of the digital mirror 200. Another instruction may direct the customer to position himself at a certain distance d from the digital mirror 200 while the customer may see his face matching with a frame of reference displayed on the screen 206 of the digital mirror 200. When the customer is in place he is asked to press an appropriate button or provide another input via the user interface 211 to begin measurements. In another embodiment, cameras 201a, 201b may use facial recognition software to identify when a face is in view of both and then automatically capture picture of the customer.

[0074] At least two cameras 201a, 201b disposed on different heights take one or more pictures of the customer. Said two cameras (201a, 201b) are calibrated so as to take at least two pictures of the customer face which match geometrically. The control unit 202 of the digital mirror 200 manages the cameras 201a, 201b via an application and then process the acquired data with the aid of the measurement application 213 which runs on the processor 203.

[0075] Once the pictures are taken the measurement application 213 processes the acquired data. It outputs two parameters in proper units, namely pupillary distance PD and height between pupils and eyeglass frame bottom edge SH, namely half-height. Two different algorithms are implemented in order to output those two dimensions.

[0076] As shown in Fig. 5 a method of determining the pupillary distance PD according to the invention, performed by the measuring application 213 comprises at least four further steps.

[0077] First, an image smoothing step is performed. The captured image needs to be smoothed, e.g. by a Gaussian filter, in order to avoid problems that arise due to bright outliers such as reflections of glasses. This helps in detections of pupils' centers. Moreover, it is required to convert the captured image into greyscale image.

[0078] Next operation is a face detection step, for example performed by the OpenCV. Based on the position of detected face and anthropometric relations, there are extracted rough eye regions relative to the size of the detected face. Those regions are used to estimate the eye centers accurately in the following step.

[0079] The next step is an eye center detection step. Eye centers detection may be performed for example by using the algorithm described in detail in "Accurate eye center localization by means of gradients" , Fabian Timm and Erhardt Barth, Institute for Neuro- and Bioinformatics. It bases on strong contrast between iris and sclera. It is assumed that there will be a peak of grey scale gradient to detect in each direction of the captured image. Next a procedure which looks for an intersection of most of directions of those gradients is performed. Because of circular eye shape the intersection points will estimate eye center.

[0080] Further in the next step, the measurement application 213 calculates pupil position in the two captured pictures and further reconstruct 3D pupil position based on known physical location of both cameras 201a, 201b using the algorithm for measuring real distance as described below.

[0081] A method for measuring real distance from captured pictures according to the invention comprises at first a calibration step. This step involves calculating camera angles of rotation and finding position of the cameras, which are measured after fixing the cameras in permanent positions. Various calibration methods are possible.

[0082] In one embodiment the calibration algorithm involves simulated annealing with minimization of mean squared 3d reconstruction error. Said task comprises calculating the position of cameras: namely their coordinates in the space 3d (x,y,z) as well as their inclination in three planes (angles of rotation): angles alfa, beta, gamma.

[0083] In the first step, input data is a series of input pictures of a predetermined pattern from both cameras, as shown in Fig. 8. Said predetermined pattern is a panel with two aruco markers, placed in a predetermined distance (for example 64 mm).

(http://docs.opencv.Org/3.l.0/d5/dae/tutorial aruco detection.html).

[0084] In the second step, the 2d coordinates of said two markers are found on each of the pictures using a simple detection algorithm (for example using OpenCV). Said coordinates are used in all further steps of the calibration process.

[0085] Then the iterative algorithm is performed using some preset initial values P = (x,y,z, alfa, beta, gamma) according to project assumption of the mirror for both cameras.

[0086] In each iteration of the algorithm better values P=(x, y, z, alfa, beta, gamma), are searched in proximity of input values, namely such that for which mean squared error is lower.

[0087] The initial range for searches is set to

for x,y,z and +/- 0.1 degree for angles alfa, beta, gamma. In further iterations said range is narrowed.

[0088] The iteration of the algorithm comprises random generating of new proposition for P'=(x', y', z', alfa', beta', gamma') within the defined ranges, calculating of mean squared error of the 3d reconstruction. For this purpose, for each pair of pattern pictures the distance between aruco markers is calculated. The error under examination is the error for the whole series of pictures. The error for a single picture is a difference between the calculated distance and the real distance (for example 64 mm). If the error for said new proposition P^J is better than the error for the initial value, then said new proposition P' becomes initial value for the next iteration (P=P'). Said calculations are repeated till the value of the error becomes lower than the assumed value or are repeated for a preset number of iterations, for example 10000.

[0089] In another embodiment, the calibration step can be done using a known pattern (i.e. aruco markers, as shown in Fig. 8), in the following way. The pattern sheet is placed parallel to the front surface of the two-way mirror 219 of the digital mirror 200 at lm distance and markers are in such position that if both cameras 201a, 201b were mounted perfectly (perpendicular to the digital mirror plane), optical axes would intersect the calibration sheet in those marker points. Each camera 201a, 201b takes picture of the sheet with pattern (as shown in Fig.8). Then processing of data from said taken pictures is performed. Based on the assumption that the center of each photo is an intersection point of optical axis with the pattern sheet, there can be found the number of pixels between that center point and each marker. This results in obtaining input data related to the position of said markers. Said data allows calculation of camera angles of rotation.

[0090] Further based on known camera zoom and geometric relations, rotation angles of cameras are calculated as follows: first angles of rotation around real optical axes are calculated separately for both cameras (pixels are counted separately for each photo):

[0091] Then, new coordinates of the marker (without calculated angles of rotation) are calculated:

[0092] In the above-mentioned formula (2) the number of pixels from the center of each photo to the nearest marker (pix and pix₂' ) are used. [0093] Then, the remaining two angles of rotation a and b for both cameras are calculated. Based on geometrical relations as presented in Fig. 9 following formulas are derived:

a) h = k * pix d = m * pix₂

[0094] Where:

f - focal length of camera lens

a, b - width and height of camera matrix

nl,n2 - number of pixels horizontally and vertically between the nearest marker and the center of the photo.

[0095] As a result of mathematical transformation, final formulas for both angles of rotations are obtained:

[0096] Knowing cameras angles of rotation is identical with knowledge about directions of cameras optical axis in space. Other known input information required to complete calculations is: positions of cameras and markers (in 3D), dimensions of cameras matrixes and their focal lengths.

[0097] Referring again to Fig. 5, once the directions of both optical axes are known from the calibration step, namely 3D pupil position reconstruction step is completed, one may start to search for pupillary distance PD based on the pictures taken by at least two cameras 201a, 201b.

[0098] Firstly, distances from the center of each photo to both pupils are determined and transformed using the equation (2). Next an algorithm basing on formulas for direction vectors that starts on camera and goes through pupil, solves a set of linear equations.

[0100] Where:

Xi, yi, Zj - coordinates of camera position (X axis is normal to mirror plane, Z axis is vertical)

t - ancillary variable

Li - distance from camera to pupil

[0101] The solution of those equations gives a relative position of the first pupil. The same is calculated for the second pupil. Based on coordinates of said two points a pupillary distance PD is calculated. In other words, the control unit 203, in particular measuring application 213 is adapted to process at least two pictures acquired by at least said two cameras 201a, 201b so as to find characteristic points of the customer face (i.e. pupil centers in 2D space) which are relative positions of pupils. Then the measuring application 213 calculates pupillary distance PD based on said characteristic points of the customer face.

[0102] At the end, advantageously, a false detection filtering algorithm can be applied. The measurements are compared to the set of rules to ensure the accuracy of the result, i.e. average physiological traits of face among society.

[0103] Now a method of determining the half-height SH will be described in reference to Fig. 6. Said method according to the invention comprises also several steps.

[0104] The first couple operations are the same as for the algorithm for determining pupillary distance described above, namely acquiring reference data from the selected eyeglass frame 400, capturing at least two pictures by two different cameras at two different heights, smoothing, face detection and eye center detection steps. [0105] The next step comprises forehead area extraction and then image enhancement. In that case the Otsu's method is used. It reduces greyscale image to a binary image.

[0106] In the next step, the height between pupils and frame bridge top edge SD is determined, based on the location of pupils that was found earlier and frame bridge top edge, using the same method for distance measuring from taken pictures as in pupillary distance calculation. By scanning pixels, the measurement application 213 reads all pixels from the top of the image line by line until a first object pixel is found (i.e. first black pixel is found). Said pixel is a frame bridge top edge. The measurement application 213 reconstructs in the next step 3D frame bridge position based on calculated directions of both optical axes of both cameras 201a, 201b.

[0107] The last step of calculation is determining the half-height SH, based on height between pupils and frame bridge top edge SD from previous step and the at least 2D model data of the eyeglass frame 400 from the database 302. Additionally, like for the pupillary distance calculation, a false detection filtering algorithm can be applied at the end.

[0108] A general flowchart of PD and SH calculation process has been shown also in fig.7. It shows more directly that said calculation processes comprise several steps in which basis geometrical relations are used to calculate specific distances, namely pupillary distance and half height in 3D space.

[0109] Advantageously, the measurement application 213 and the said at least two cameras 201a, 201b are configured to acquire the measurements PD and SH in near real time fashion, in order to allow multiple measurements of one customer, further allowing the filtration and refinement of the most accurate results. The measurements, as a part of the method of ordering eyewear according to the second embodiment, are executed in a loop, comprising steps of:

- taking at least two pictures from a different height

- putting the pictures on dedicated stack.

- choosing a set of most recent pictures from the stacks, in such way that the time of capture of the pictures is most equal

- processing of said chosen set of pictures so as to find characteristic points of the customer face by - performing 2D measurement on each picture.

- performing 3D reconstruction on set of pictures

- adding the result of the picture processing, relating to at least characteristic points of the customer face, to a set of measurements.

[0110] Regarding said second embodiment of the method, there is a dedicated stack for each camera 201a, 201b. Dedicated stacks are stored in the internal memory 204 of the device for ordering eyewear. Thanks to adding the result of each loop to a set of measurements a way for comparison of multiple results for single customer is provided. By repeating measurements in a loop more customer related data is obtained allowing more accurate final result. Processing of a set of pictures instead of only two or three allows to implement a method of filtering false positive results by applying measure of quality for each result. The condition of the end of the loop is obtaining sufficient measurements above the required quality threshold.

[0111] Once the loop is ended, the method of ordering eyewear according to the second embodiment further comprises calculating pupillary distance PD and half-height SH based on the at least 2D model data of the eyeglass frame 400 from the database (302) and said characteristic points of the customer face.

[0112] As mentioned above the device and the system 100 for ordering eyewear according to the invention will enable customers to select and purchase eyeglass frames 400 and lenses without the support of shop assistant also via a mobile device 500, for example via a cell phone. In another embodiment a method of operating the digital mirror 200 (as shown in Fig. 4) is performed with the use of a mobile device 500, a smartphone for example. Advantageously according to said embodiment the whole process of eye glasses configuration and ordering, except measuring pupillary distance and half-height, might be completed via the mobile device 500 communicating with the digital mirror 200. This is possible thanks to a set of mobile applications that run on the device 200 for ordering eyewear according to the invention and on the mobile device 500 like smartphone. Appropriate computer instructions may be provided in the form of an app downloaded to the mobile device 500 or may be executed within a browser running on the mobile device 500. The connection between the digital mirror 200 and the user interface application 501 might be advantageously set by generating and reading QR code. This is possible thanks to a QR code generator 205 build in the digital mirror 200 as well as QR code scanner 502 and QR code reading application 503 on the mobile device 500. It is assumed that a control unit (not shown) incorporated in the mobile device 500, for example smartphone is configured to execute computer instructions associated with the display means application 216 running on the digital mirror 200 for displaying information and prompts on the screen 206 and for receiving input from the user via a user interface being an integral part of the mobile device 500. At first logging in or registering a new account is required. The process of registration or logging in is managed by the digital mirror application 215 running on the digital mirror 200. Next during the ordering process, the displays might include advertisements and user prompts for the frame selection feature, the lens/lens option selection feature, etc.

[0113] Advantageously, the device 200 for ordering eyewear and system 100 allows for providing the customer with purchase recommendations based on computer aided analysis of customer appearance, traits and choices, by facilitating the use of image processing algorithms and Al based algorithms.

[0114] Reference numbers

100 - system for ordering eyewear

200 - device for ordering eyewear/ digital mirror

201 - screen 201a - camera 1 201b - camera 2 201c - camera 3

202 - control unit

203 - processor

204 - memory means

205 - QR code generator

206 - display means - communication means / modem for communicating with external devices - embedded payment interface

- means for reading marking from eyeglass frames / RFID reader

- user interface

- embedded scanner for scanning prescriptions

- measurement application

- user interface application / admin interface application

- digital mirror application

- an application serving hardware interfaces

- FrontEnd application

- BackEnd application

-two-way mirror

- frame

- BackOffice system

- processor

- database with information on eyeglass frames 400 upon purchase - user interface

- eyeglass frame

- marking means / tag chip / reading tags TAG1, TAG2

- mobile device

- user interface application

- QR code scanner

- QR code reading application - external electronic card reader - external server

- web application for handling orders - external payment interface - external scanner

Claims

1. An electronic device (100) for ordering eyewear comprising first camera (201a) for taking pictures of the customer eye region

a control unit (202) comprising processor (203) and memory means (204) for managing components of the device (100), the control unit (202) being adapted to acquire and process data on lenses required for the customer

communication means (207) for sending orders characterized in that it further comprises at least one additional camera (201b) for taking pictures of the customer face, disposed at different height than the first camera (201a)

means (209) for reading marking from eyeglass frames (400) for accessing its model data from the database (302) wherein at least said two cameras (201a, 201b) are calibrated so as to take at least two pictures of the customer face which match geometrically and wherein the control unit (202) is adapted to process said at least two pictures acquired by at least said two cameras (201a, 201b) so as to find characteristic points of the customer face and is adapted to calculate pupillary distance PD and half-height SH based on the at least 2D model data of the eyeglass frame (400) from the database (302) and said characteristic points of the customer face wherein the control unit (202) is further adapted to automatically complete an order for eyewear.

2. Device according to claim 1, wherein the device (100) comprises a frame (230) and a two-way mirror (219) arranged on the frame (230) so as to cover said at least two cameras (201a, 201b) and constitute a display surface.

3. Device according to claim 1 or claim 2, wherein the device (100) comprises display means (206) for prompting the customer and a user interface (211) for providing inputs from the customer.

4. Device according to any preceding claims, wherein the device (100) comprises a QR code generator (205) for enabling establishing connection with a mobile device (500).

5. Device according to any preceding claims, wherein the device (100) comprises a scanner (212) for scanning prescriptions.

6. System for ordering eyewear characterized in that it comprises

- a backoffice system (300)

- an external server 700 for managing orders

- an electronic device (100) for ordering eyewear comprising first camera (201a) for taking pictures of the customer eye region

communication means (207) for sending orders

at least one additional camera (201b) for taking pictures of the customer face, disposed at different height than the first camera (201a)

means (209) for reading marking from eyeglass frames (400) for accessing its model data from the database (302) wherein at least said two cameras (201a, 201b) are calibrated so as to take at least two pictures of the customer face which match geometrically and wherein the control unit (202) is adapted to process said at least two pictures acquired by at least said two cameras (201a, 201b) so as to find characteristic points of the customer face and is adapted to calculate pupillary distance PD and half-height SH based on the at least 2D model data of the eyeglass frame (400) from the database (302) and said characteristic points of the customer face. wherein the control unit (202) is further adapted to automatically complete an order for eyewear.

7. The system according to claim 6, wherein the device (100) comprises a frame (230) and a two-way mirror (219) arranged on the frame (230) so as to cover said at least two cameras (201a, 201b) and constitute a display surface.

8. The system according to claim 6 or claim 7, wherein it further comprises a mobile device (500) for interacting with the electronic device (100) for ordering eyewear.

9. The system according to any preceding claims, wherein it comprises an external payment interface (800) and/or an external scanner (801) for prescriptions.

10. A method of ordering eyewear, comprising steps of:

- taking a first picture of the customer eye region from a first height

- acquiring at least 2D model data related to eyeglass frames (400) upon purchase

- acquiring data on lenses required for the customer

- sending order characterized in that before sending an order it further comprises steps of

- taking at least second picture of the customer eye region from a different height

- reading marking from eyeglass frames (400) for accessing its model data from the database (302) wherein said at least two pictures of the customer face match geometrically

- processing of said at least two pictures acquired by at least said two cameras (201a, 201b) so as to find characteristic points of the customer face and

- calculating pupillary distance PD and half-height SH based on the at least 2D model data of the eyeglass frame (400) from the database (302) and said characteristic points of the customer face

- automatically completing the order for eyewear.

11. Method according to claim 10, wherein inputs from the customer are provided with the use of a user interface (211) built in the device (100) for ordering eyewear.

12. Method according to claim 10, wherein inputs from the customer are provided with the use of a user interface of the mobile device (500).

13. Method according to claim 10 or 11 or 12, wherein acquiring data on lenses required for the customer is performed by scanning a prescription with a scanner (212, 801).

14. A method of ordering eyewear, comprising steps of:

- acquiring data on lenses required for the customer

- sending order characterized in that before sending an order it further comprises following steps executed in loop :

- taking a first picture of the customer eye region from a first height,

- taking at least a second picture of the customer eye by at least a second camera from a different height,

- putting the first picture and each next picture on a dedicated stack,

- choosing a set of most recent pictures from the stacks, in such way that the time of capture of the pictures is most equal,

- processing of said chosen set of pictures so as to find characteristic points of the customer face by:

- performing 2D measurement on each picture.

- performing 3D reconstruction on set of pictures

- adding the result of the picture processing, relating to at least characteristic points of the customer face, to a set of measurements and once the loop is ended the method further comprises a step of:

- calculating pupillary distance PD and half-height SH based on the at least 2D model data of the eyeglass frame (400) from the database (302) and said characteristic points of the customer face and - automatically completing the order for eyewear.