The Microsoft Azure team introduced the Sketch2Code Web tool to convert user interface sketches to HTML codes. Service speeds up the process of developing projects, eliminating the need to manually create markup based on the drawing.
Sketch2Code works with artificial intelligence. The process of converting a picture into code consists of five steps.
- First the user uploads an image through the website.
- A custom vision model predicts what HTML elements are present in the image and their location.
- A handwritten text recognition service reads the text inside the predicted elements.
- A layout algorithm uses the spatial information from all the bounding boxes of the predicted elements to generate a grid structure that accommodates all.
- An HTML generation engine uses all these pieces of information to generate an HTML markup code reflecting the result.
All images, results and grouping information are stored in the BLOB storage. The connecting link between all services is Azure Functions. The tool is developed in collaboration with Kabel and Spike Techniques. Details about Sketch2Code are provided in the GitHub documentation.