Captions are a great way to help people with accessibility issues understand the content they’re consuming, and now Microsoft wants to make them even better. The company has announced that Microsoft Edge will now “caption” images so that those that are hard of sight know what’s going on.
Microsoft Edge’s New Image Descriptions
Microsoft made the announcement on the Microsoft Edge Blog. The captioning is meant to enhance narrators, which read out the content on a webpage.
Sometimes, website owners will set text on images themselves (called “alt text”) which a narrator will read out. This means that those with reduced eyesight can “hear the image” and understand what’s going on on the website.
However, sometimes website owners don’t set text, which leaves a narrator in the dark about what the image is. If this happens in Microsoft Edge, it can upload the image to the Computer Vision API for Azure Cognitive Services.
Microsoft states that the API works with most of the more well-used image formats, such as JPEG, PNG, and even GIFs. Once the API thinks it knows what’s going on, it’ll create its own caption for the user’s narrator to use. The API can identify both what’s going on in an image, and any text printed on it.
There are a few limitations to the feature, though. It won’t process the following:
- Images that are marked as “decorative” by the web site author. Decorative images don’t contribute to the content or meaning of the web site.
- Images smaller than 50 x 50 pixels (icon size and smaller)
- Excessively large images
- Images categorized by the Vision API as pornographic in nature, gory, or sexually suggestive.
It will also skip over any images that the website owner has already set text for. This means that human-written alt text will always be prioritized over what the API thinks it should say.
If the API does generate its own description, the narrator will state that the image “appears to be…” before describing it. Any text it detects will be introduced with “appears to say…” so that listeners understand that the narrator is making a guess using the API, instead of using the website owner’s own text.
An Accessibility Win for Edge
Browser developers are always trying to one-up each other to become the top dog, but a race to become the most accessible browser on the market is a win for everyone. People who are hard of hearing or sight can browse the internet better than ever as companies fight to give the best experience.
Other companies outside of the web browsing market are offering their own take on this tech. For instance, in January 2021, Facebook made improvements to its AI image description service to help those who need aid.
As such, those who need help surfing the net may find solace in Microsoft Edge. And given how it comes pre-installed with every copy of Windows 10 and 11, it means there’s minimal setup to get this feature working.
Making the Internet Accessible, One Feature at a Time
Any news to aid those who need help is good news, and Microsoft Edge’s image description service seems to be a winner. We’ll have to see if other browsers respond in kind to this development.