Now, Microsoft has expanded the capabilities of Sketch2Code. Thanks to its ease of use, does the app put dev’s who work on bringing service from prototype to real products under threat? That’s probably a little hyperbolic, but there’s no doubt Sketch2Code streamlines development. Users can draw sketches and export them to a code translator that will convert sketches into source code. Microsoft Garage developed an artificial intelligence system to automate code translations from wire frame drawings.
We have often heard how AI will transform workplaces and make many human jobs obsolete. While Sketch2Code is very much an early step, an evolution of automatic coding clearly puts the role of frontline developers in doubt through the long-term. That said, at the moment Sketch2Code is best described as handy tool and not game-changer. At the moment, the service supports basic app elements in visual implements. So, text, images, buttons, and labels can be transitioned from sketches to source code.
AI Model
Microsoft has leveraged a highly trained AI that can detect objects. Sketch2Code create HTML results from different objects within a design. Below are the capabilities of the services:
A Custom Vision Model trained to perform object recognition against HTML hand drawn patterns is used to detect meaningful design elements into an image. Each detected element is passed through a Text Recognition Service to extract handwritten content. The information of the detected objects and their position inside the image is feed into an algorithm that generates underlying structure.
A valid HTML is generated accordingly to the detected layout containing the detected design elements.
