One of the new features in iPadOS 14 that has stood out the most is called Scribble. This allows you to make freehand notes written with the Apple Pencil so that it automatically transforms into typed text. This can be very useful to take freehand notes very quickly and to transform them into much more readable text. Apple’s own senior vice president of software engineering has given much more insight into how it works.
How Scribble works uncovered
You must understand the difficulty of this function that has been integrated with iPadOS 14. The fact of writing by hand in any text field so that it becomes typed text means that users do not have to switch between the keyboard and the Apple Pencil when taking notes. Obviously a lot of data gathering has to be done for this feature to work properly. You have to know that not everyone writes in the same way and everyone has a very different style of writing.
Federighi through Popular Mechanics has declared all the work they had to do to make this function see the light. In short, Apple found people from all over the world to write things with the Apple Pencil on a surface that monitored all the movements they made to write a preset text. Specifically, he has explained it in the following way:
When it comes to understanding strokes (handwritten), we collect data. We met people all over the world and asked them to write anything. We give them a pencil, and we make them write fast, we make them write slow, write inclined inclined, in short with all the possible variation. We needed to see the strokes that made up each letter. Ultimately, if you understand the strokes and how they are made, it can be used to remove ambiguity from what was being written.
Ultimately, what you do is train artificial intelligence with the equipment in question on site. This is really complicated since an iPad must be used to perform all the monitoring of the movements. It is obvious to think that it is necessary to have enough computational power so that the level of processing is carried out locally. The iPad in its vast majority of modes does not have any type of problem with the processor that it includes, so it must have been a relatively simple task in this regard.
Obviously the fact of having to recruit hundreds of people to observe the type of writing is the most complicated thing that has been found. In the end there is a function that interprets numerous fonts and that although it has a small flaw at specific moments in future updates, it will surely end up being solved. In the end, with the associated complexity it works quite well, appreciating the good work that Apple has done when doing this training.
