Computer vision applications have become ubiquitous nowadays. It’s hard to think of a domain where the ability of computers to “see” what’s going on around them has not yet been leveraged.
Delivering process automation and accuracy, computer vision technology is expected to have even greater momentum due to the COVID-19 pandemic as organizations have rushed to adopt automation on a larger scale.
More nuanced use cases of computer vision in different industries are also predicted to emerge with the next evolutionary leaps in artificial intelligence development; a field computer vision is a part of. Let’s look at the inroads already made by computer vision, its vast applications in different industries, and its benefits.
What is computer vision, and how has it evolved?
Computer vision (CV) is a subset of AI that enables systems to interpret information from digital images and react to it with action or recommendations. The goal of computer vision technology is to emulate human vision for performing monotonous or complex visual tasks faster and even more efficiently.
Historically, it all started with a lot of manual coding until machine learning progressed enough to allow developers to program smaller computer vision applications and apply statistical learning algorithms for specific CV-related tasks such as pattern recognition. With AI making major strides, ML algorithms have been largely replaced by deep learning or hybrid models that rely on neural networks to transform patterns into mathematical equations for information classification. After the launch of the first commercial software back in the 1970s, computer vision applications have evolved from enabling reading devices for the blind to transforming entire industries.
Some systems powered by computer vision have achieved 99% accuracy and can even surpass human performance (for instance, in diagnostic radiology).
The key drivers behind the surge in computer vision applications are:
- Spreading mobile technology that allows us to create and share billions of images a day
- Increasingly affordable computing power to process this massive amount of visual data
- New hardware
- A burst of deep learning algorithms
What can computer vision do? — Major computer vision techniques
With the growth in visual data and advances in computing power to process it, present-day computer vision applications rely on the following technology capabilities:
- Object classification to assign objects in a photograph or video to predefined classes. With binary classification, the system answers a simple question whether a particular object belongs to, say, a class of apples, tourist attractions, or cats.
- Object localization to locate an object in an image by enclosing it into a bounding box.
- Object detection to do both of the above to many objects in an image, assigning labels and localizing objects by drawing bounding boxes around them.
- Semantic segmentation to understand every pixel of an image and associate it with a class label (a car, a person, etc.) by creating object masks, with objects of the same class presented as a single entity.
- Instance segmentation to do semantic segmentation and additionally identify different instances of the same class so that you don’t get just a one-color mask for three parked cars in a street view photo but label the vehicles with three different colors, identifying their boundaries.
Computer vision applications — Benefits and real-world examples
Applications for computer vision have been expanding at a rapid pace over the last decade to reach a frantic level with the onset of the COVID-19 pandemic. Organizations now invest heavily in AI-driven solutions, from retail software and healthcare platforms to advanced manufacturing and government systems. Many are even concerned that AI technology is moving too fast for them, according to a survey of business leaders conducted by KPMG.
While there is still an overall lack of AI regulation that comes with AI explainability challenges and bias risk, computer vision technology is a safer playing field due to its maturity. Given the investment speed and immediate benefits delivered by computer vision implementation, the worldwide market of computer vision solutions is projected to grow at a CAGR of 7.6% to $19.1 billion from 2020 to 2027.
We’ve compiled a list of industries and use case examples to demonstrate how companies leverage advanced computer vision techniques to boost their results:
- Retail & Ecommerce
- Education
- Healthcare
- Fitness & Sports
- Precision Agriculture
- Manufacturing & Mining
- Cross-Industry Applications
Computer vision applications in retail and eCommerce
Benefits :
- Minimizing human interactions in stores to boost safety, deliver digital shopping experience, and lower labor costs
- Personalizing customer experience for increased engagement and more effective upselling and cross-selling strategies
- Leveraging next-gen in-store analytics to prevent stockouts, enhance store layout designs, and optimize staff scheduling
Use cases:
- Automated checkouts and cashierless stores combine computer vision techniques with shelf sensors and deep learning to recognize shoppers, detect items they place into their carts, and automatically charge them on their accounts for items bought when they leave the store. Apart from the famous Amazon Go example, automated stores using sophisticated computer vision technology have also been launched by Chinese giants Lenovo, JD, and Alibaba. In the startup space, AI provider Tiliter’s self-scanning scale solution can automatically identify fresh produce, allowing customers to check out without the need for barcodes or packaging.
- In-store navigation systems, powered by computer vision algorithms, can find the most efficient route to products on a customer’s in-app shopping list and recalculate the route if the person decides to look at other items. Currently, home improvement chain Lowe’s tests such an indoor mapping app developed with Google. Combining computer vision and augmented reality software, the app reportedly locates users in an aisle more accurately than Wi-Fi or mobile phone systems while also providing access to product reviews.
- Product information display apps feature Scandit’s mobile barcode scanning software based on computer vision that recognizes objects for shoppers to get a personalized offer or information on the product in the store.
- Visual search solutions that leverage deep learning algorithms make it more convenient for online shoppers to discover products, returning visually similar results. eBay was among the first to offer its customers this engaging computer vision experience.
- Personal recommendations are enabled by an array of computer vision-based systems. Among them are virtual mirrors that use augmented reality, allowing shoppers to try on various clothes virtually (like on Amazon’s site) or experiment with makeup products (like via Sephora’s app or the Bourjois Magic Mirror). In-store virtual dressing room technology, embedded in Findmine’s Complete the Look solution, offers shoppers a touchscreen display with a CV-powered camera that recognizes items they are wearing to create an outfit based on images from the retailer’s catalog. Ecommerce sites are eyeing computer vision-enabled body scanning technology like the Bodygram service to provide size or clothing recommendations. The skin care brand Neutrogena invites their customers to try the Skin360 app that measures their skin health by assessing facial attributes and skin pixels and recommends a skin care routine. The Lolli & Pops candy store uses facial recognition to identify regular shoppers for offering them personalized product recommendations or loyalty discounts.
- Inspection systems based on computer vision methods help retailers improve inventory management. Examples range from the Shelfie technology that uses shelf-mounted computer vision cameras to alert staff about out-of-stock or incorrectly displayed items to the mobile robot Tally that not only notifies shop assistants of gaps on shelves but also can detect damaged packaging and inaccurate pricing.
- Connected stores use computer vision-powered cameras to analyze customer journeys and get product-related insights. Among them are Samsung’s famous pop-up store infused with capabilities to analyze dwell time, demographics, and other customer data, and Serbian fashion retailer Legend World Wide’s store built in collaboration with Deloitte to gauge customer movement heatmaps.
Computer vision applications in education
Benefits:
- Understanding students’ learning behaviors to drive personalization and improve learning experiences
- Automating classroom monitoring to deter cheating in tests
- Assessing students’ papers to reduce the burden on educators
Use cases:
- Student engagement detection and personalization. With computer vision-powered platforms, educators can measure students’ mood and behavior to capture signs of engagement or distraction both online and in a classroom. Solutions in this space include Emotuit that uses facial recognition to analyze students’ emotional response to online content and the Little Dragon learning app that also reads facial expressions to detect frustration or boredom and adapt learning content. In addition, computer vision-driven insights help teachers regroup students into a more comfortable environment to improve learning.
- Attendance monitoring and automated online proctoring has been made easy with computer vision-powered webcams that are used to identify students and flag cheating behaviors by tracking their postures or eye movements. Examples include UAuto with multi-factor authentication launched by the ProctorU proctoring service and automated proctoring solutions like Respondus Monitor and Mettl.
- Handwritten character recognition is an area where computer vision is also expected to shine, as advanced algorithms are able to not only recognize responses written by students but also assist with their automatic evaluation.
Computer vision applications in healthcare
Benefits:
- Improving patient identification to prevent wrong person procedures
- Delivering more accurate diagnosis through medical imaging analysis
- Providing assistance in surgery training and real-world surgeries for better outcomes
- Delivering rehabilitation assistance to patients
Use cases:
- Patient identification systems use computer vision-based cameras that help improve facial authentication of patients from check-in to discharge to prevent wrong person procedures.
- Medical image analysis assisted by computer vision is transforming radiology, helping practitioners interpret X-ray, CT scans, MRIs, and even microscopic images of cellular structures more accurately when diagnosing breast, brain, lung, or skin cancer. Computer vision applications in medical imaging also feature solutions to estimate a human pose in analyzing symptoms of neurological and neuropsychiatric disorders, monitor blood loss to optimize blood transfusions, diagnose eye conditions, and even detect COVID-19 (a deep convolutional neural network called COVID-Net has showed 90% accuracy in diagnosing COVID-19 based on chest X-ray images).
- Surgical simulation and assistance leverages computer vision technology to increase surgical precision. Apart from assistant surgical robot systems, there are solutions like Proprio Vision that combine computer vision with ML and VR to create 3D visualizations for surgeons in the operating room. In the surgical training space, Touch Surgery is a famous mobile simulator that provides a detailed guide to surgical procedures.
- Rehabilitation applications feature computer vision systems that are being developed to supervise exercise routines as part of at-home rehabilitation for rheumatoid arthritis, sports injury, brain injury, or stroke.
Computer vision applications in fitness and sports
Benefits:
- Capturing performance data to aid coaches in training sessions and athletes in self-training
- Introducing advanced player or ball tracking methods to improve viewing experience or help referees in decision-making
- Collecting performance statistics for scouts, sports betting sites, and other industry professionals
Use cases:
- Tracking systems powered by computer vision-enabled cameras detect and track moving players or balls in an array of games such as soccer, tennis, baseball or golf. Top examples include SentioScope, designed by Sentio for soccer player tracking and analysis, and the SportVU 2.0 optical tracking technology that gives football coaches a holistic view of matches. Computer vision-based systems are also used for improving shooting accuracy in basketball training (Noah System), help swimmers improve their techniques by collecting data from stroke rates to real-time velocity and turn times (FINIS LaneVision), and even can take over in part the job of the umpire in professional tennis matches (the Hawk-Eye ball tracking solution).
- Self-training solutions that are based on computer vision techniques like pose estimation for motion analysis can help users improve their self-training, with Zenia, for instance, claiming that its fitness app powered by computer vision and ML can recognize yoga asanas with 95% accuracy. Such fitness solutions can also use markers as the Carbon intelligent fitness mirror does, unlocking data on user performance with the help of weight sensors to offer intelligent suggestions.
- In-depth data analytics platforms analyze the actions of players on the ice or field, producing meaningful insights either for building better game strategies or making smart decisions on players in the scouting market, or engaging viewers. Among famous examples are the Sportlogiq system that collects raw data from video feeds to produce game models and the AutoSTATS solution that can analyze any recorded basketball game for performance improvement and scouting insights.
Computer vision applications in precision agriculture
Benefits:
- Identifying pests and weeds with greater accuracy to optimize the application of chemicals
- Monitoring crop development and the environment to maximize yields and produce better quality according to rising customer expectations
- Automating livestock management to prevent flock and herd losses and reduce the need for feet in the field
Use cases:
- Pest and weed detection systems feature Blue River Technology’s See & Spray solution equipped with intelligent cameras that can distinguish between crops and weeds to apply herbicides to the right plants. Computer vision methods are also used for automatic identification and counting of flying insects and even for identifying apple diseases early.
- Observation, harvesting, and prediction systems are developed to detect the ripeness of fruit, including through color ratings of cherries in an outdoor environment or pick vegetables in a greenhouse using robots. Computer vision-aided monitoring solutions include SolarXOne, a 100% autonomous solar-powered drone system from XSun, that provides farmers with HD images capturing crop and soil conditions. AI solutions from Brazil’s Cromai can gauge information about the color, shape, and texture of crops. High-definition cameras from SWIR Vision Systems are equipped with sensors to help monitor soil moisture for yield prediction.
- Livestock management systems using drone technology can perform automatic counts, detect sick or injured animals, find strays, spot grazed areas, and even move cattle. A fair example is autonomous drone technology developed by Israeli firm BeeFree Agro to herd cattle.
Computer vision applications in manufacturing and mining
Benefits:
- Implementing automated quality control to increase manufacturing accuracy, improve productivity, and produce better quality
- Deploying monitoring solutions to cut inspection time, minimize safety risks, improve operator productivity, and increase cost-efficiency
- Reduce human involvement to protect workers from hazardous environments
Use cases:
- Next-level quality control can be enabled by intelligent computer vision-powered cameras directed at a manufacturing line. Examples include Pharma Packaging Systems’ machinery for the pharmaceutical industry to automatically count tablets or capsules on production lines and WebSPECTOR, a surface inspection system that identifies defects in items, stores images, and collects image-related metadata to classify errors by type and grade. Сomputer vision methods are also used to guide assembly operations, like assembly verification solutions from Acquire Automation that measure product components versus manufacturing specifications, check caps and fill levels, and verify packaging components.
- Robot palletizing systems guided by machine vision automatically load or unload boxes and items to and from pallets.
- Predictive maintenance systems assisted by computer vision technology and sensors have made it much easier to track the condition of critical infrastructure and determine when maintenance is needed. For example, FANUC’s Zero Down Time takes photos and collects metadata to uncover any potential problems in the machine. Oil and gas giants such as Shell, ExxonMobil, and BP are using computer vision-powered predictive maintenance to anticipate failures in their equipment.
- Intelligent monitoring solutions, including drone-assisted systems, allow companies to conduct remote inspections of their sites and assets. This application of computer vision is especially important in mining, an unsafe industry for workers, where operators need to collect visual data in difficult areas. Visual inspections of well sites using the Osprey Reach system, for instance, have enabled operators to reduce routine site visits by half.
Computer vision in cross-industry applications
- Warehouse management systems using computer vision technology reduce inventory times from hours to minutes, delivering huge savings on operational costs. Such systems feature the Gather AI computer vision-based platform that uses drones and connects to IoT devices for scanning and counting inventory. Another example is Amazon rolling out the Pegasus robot technology at its sorting centers, claiming that the robot can improve sorting accuracy by 50%.
- Mobile computer vision enables a contactless delivery process for retail, logistics, post, and parcel businesses, transforming smartphones and other smart devices into computer vision-enabled barcode scanners like solutions from Scandit. The frictionless way to pick items from warehouses, distribution centers, or retail rooms is growing in popularity in the age of COVID-19. In retail, barcode scanning software is used for order fulfillment while allowing shoppers to safely collect items with one contactless mobile scan.
- Safety monitoring solutions using computer vision help keep public spaces safe during the pandemic by detecting ill employees or students as well as monitoring social distancing or exposure times. At construction sites and on the manufacturing floor, computer vision systems like IRIS monitor behavior-based safety and can protect people working around hazardous zones by alerting machine operators of dangerous events or incidents.
What's next?
It might be hard to believe, but we can uncover more computer vision applications and benefits with the advancement of technology such as edge computing, emotion AI, mixed reality, and embedded vision. And they can be quite incredible as artificial intelligence will get as sophisticated as we humans are.
Drop ITRex a line if you want to explore the benefits that computer vision can bring to your organization. Their AI experts will be happy to help you with complex or straightforward computer vision initiatives to address your specific business needs.
Previously published on https://itrexgroup.com/blog/computer-vision-applications-in-different-industries/