CoVision – an AI-Based Classification System of Rapid CoVid Tests for Visually Impaired and Blind People

Background

​

In the last years the world has learned how to live in the situation of a global pandemic. While current versions of CoVid-19 are less deadly, the number of infections – and consequently the number of many severe consequences – is still high, especially in the cold periods of the year. Nonetheless, majority of the countries have reduced their CoVid preventive measures, so that the prevention of the spreading of this disease has become more and more a personal responsibility for each of us.

 

One of the most important tools to reduce the spreading of the virus and to protect yourself and your beloveds is personal rapid antigen test that can be performed and evaluated at home without the help of medical staff. Almost everyone did already take such a test and many people perform them on regular basis, especially in the case of first flu-like symptoms. Because of the simplicity, convenience, and privacy of these tests, they have become extremely popular all over the world. Unfortunately, there is a group of people that has largely none or very difficult access to the personal CoVid tests – these are the visually impaired and blind people.

 

While these people are able to perform tests on their own, they are not able to read out the results from the test, so, that they are dependent on help from other people. Because many of blind and visually impaired people live alone, they are enforced to seek help from people outside like their neighbors or to send photos of their tests to their family and friends via mobile devices. The   lack of privacy and considerable inconvenience associated with seeking such help make rapid CoVid tests less attractive and difficult to use for visually impaired people. In the worst case it might lead to less adoption of this technology among blind and visually impaired which would put them at a greater risk.

 

However, two modern technologies can help these people to get a better, more convenient, and private access to the CoVid tests. First, accessibility of mobile devices for blind people has become much better in the last years thanks to technologies like voice-over, and manufacturers of mobile devices / operation systems are setting great value on improving accessibility technologies and making their products usable for visually impaired people. In particular, it means that blind people are able to shoot photos. Second, Computer Vision – an Artificial Intelligence (AI) discipline that is focused on analyzing photos and videos – can be applied for classification of CoVid test results on photos made by mobile devices. And this is exactly what MI4People will realize with this project.

 

Motivated by the original idea from Dr. Stefanie Lämmle from InnovationLab of City of Munich and discussions with Steffen Erzgraber from Bavarian Federation of the Blind and Visually Impaired a group of students has created a first working prototype of this application called CoVision. The team  consisting of Simon Farshid, Raphael Feigl, Brigitta Jesica Kartono, and Lennart Maack did the prototype within 48 hours during the TUM.ai Makeathon and won the first prize at this event!

 

The current prototype shows that such application is feasible but must be improved both in terms of accuracy and usability to deliver highest possible value to the blind people. Such improvements need additional resources and time for research and requires access to further Machine Intelligence talents. Therefore, CoVision team decided to join MI4People and further their applied research work on this topic as a project within MI4People!

​

​

Vision

​

A free-of-charge and easily accessible open-source Computer Vision app that can classify CoVid rapid test results using mobile devices.

​

​

Expected Impact

​

This system will enable blind and visually impaired to perform rapid antigen CoVid tests by themselves. It will increase the convenience and privacy for this group of people and make the tests more accessible. Overall, it is expected that the app will help protect the health of blind and visually impaired and better integrate them into the measures against the pandemic.

​

​

Project Phases

​

• Phase 1: Collection of training data, increasing the Computer Vision model accuracy, and improving usability
  Since the current prototype was created with only few examples of CoVid rapid tests, the currently used Computer Vision model is not stable yet. This phase is dedicated to the collection of a bigger data set that will be used to re-train and improve the AI model. In this phase, we also will collaborate with Bavarian Federation of the Blind and Visually Impaired to research on how to increase the usability of the app for blind and visually impaired people.

• Phase 2: Field Study and incorporation of the feedback
  Using the results from Phase 1, we will perform a field study in which we will collect feedback from blind and visually impaired users to measure the effect of the app on the quality of their lives.

• Phase 3: Release to Community
  In Phase 3, MI4People will make relevant software code, data, MI-models/tools, and any other intellectual property created during the project publicly available and will create a community of enthusiasts who will proceed to maintain and improve the application as an open-source project.

​

​

Current Status

​

Phase 1 is running.

​

​

Opportunities for Contribution

 

We are currently searching for MI and domain experts who would be ready to contribute to the project as volunteers.

​

We also would be very grateful for any donations in cash or kind to support this initiative.