This is a working demo for working with Azure Cognitive Services. It analyzes uploaded images and generates a caption text and tags. As a bonus it generates a smart cropped thumbnail. Project contains Image API storing images and smart AI-cropped thumbnails in Azure Storage with metadata tagging from Azure Vision API
Although the basic cropped thumbnail could be improved with some extra effort, example below shows power of AI. It looks at the image what the focus point should be before creating the thumbnail.
For running this demo you need a (free) Azure account where you need to create the following 2 Azure resources:
Needed for storing uploaded images and generated thumbnails. After creation add the connection string to the placeholder in appsettings.Development.json. BTW, the containers 'photos' and 'thumbnails' will be create automatically.
Used for Vision API for analyzing image conten to retrieve caption text and all related tags. After creation of account, copy the key and endpoint values to the placeholders in appsettings.Development.json
When running the application locally you can use the Swagger interface to upload an image (see Samples folder). After the image is uploaded you can use List API method to retrieve image data with the recognized captain and tags from Vision API. Example output:
{
"caption": "a canal with buildings along it",
"name": "Colmar.jpg",
"tags": "building outdoor house narrow town residential",
"thumbnail": "https://<something>.blob.core.windows.net/thumbnails/Colmar.jpg",
"url": "https://<something>.blob.core.windows.net/photos/Colmar.jpg"
}
I use the following packages:
- Azure.Storage.Blobs - used for BlobContainerClient for store/retrieve images from storage account
- Microsoft.Azure.CognitiveServices.Vision.ComputerVision - used for sending images to retrieve captain and tags
- Imageflow.AllPlatforms - used for creating thumbnail images
- MimeTypeMapOfficial - used for mapping extension to corresponding mime-type
- Swashbuckle.AspNetCore - used for Swagger API documentation page
This implementation is inspired by the older Microsoft vision tuturial. The tutorial is using .NET core 2(?) with razer pages. I choose to modernize it and use .Net 6.0 with Swagger API.
For resizing with ImageFlow, I used this documentation page: Querystring documentation for BuildCommandString