Let's make circuit diagrams accessible: Part 2
In the last post we looked at different ways to represent the information that’s usually presented in a circuit diagram, also called a schematic. The goal is to start automatically converting existing circuit designs into a form that’s accessible to blind people. But before we can produce such an output, we need to get the circuit information into a format that our program can understand.
Reading schematics from images
My first impulse is to see if we can read schematic images directly, since an image is the most common way that circuits are shared online. Software for optical character recognition (OCR) is quite good these days, and there are many libraries for extracting text from images. But when it comes to reading circuit diagrams, there are only a couple of limited experimental projects with published code. Most recently Researcher Charles Kelly trained an OCR model that can recognize a few different component symbols and extract the connection graph, publishing a research paper and a GitHub repository. A library called CircuitNet can read hand-drawn circuits consisting of 6 types of components. There are several other research papers on the topic, and it may be possible to obtain code for their implementations as well.
These research projects are very promising, but none of them are mature enough to use as the basis for a translation system. The set of recognized symbols is still too small; none of these OCR models appear to recognize common things like transistors or integrated circuits. Bringing them to a level where they could handle most circuit diagrams would be a significant project, and would require computer vision skills that I don’t have yet. I decided to set image recognition aside and look for circuit designs that were already in a machine-readable format.
Reading schematics from CAD files
Most schematics are drawn using computer aided design (CAD) programs. While the user sees a graphical circuit diagram, these programs save the circuit in a machine readable format so they can be modified later. Common tools in the hobbyist community include KiCad (open source), Circuit Maker (proprietary), [EAGLE](https://en.wikipedia.org/wiki/EAGLE_(program) (proprietary), Fritzing (open source), EasyEDA (proprietary), and many, many more. Each of these tools has its own file format, although they can often import files from other popular tools. We’ll need to choose one to focus on first.
Most designers tend to use these tools to draw the schematic, then export an image to share online. It’s much less common to share the actual design files that we could parse. The exception to this is Fritzing. Fritzing has some unique features aimed at beginner and intermediate hobbyists, including the ability to visually lay out your circuit on a solderless breadboard. And it has a project directory where makers have uploaded the actual .fzz files for more than 5,000 beginner and advanced projects. What’s more, each project in the directory is tagged with a specific license - many of them are Creative Commons licensed. If we can parse Fritzing’s file format, we gain access to these 5,000 projects to test our translator. We can even automatically filter them by license, and republish an accessible description of the projects when the license permits.
A closer look at Fritzing
Fritzing is a somewhat unique tool in the circuit CAD / EDA space. It’s focused on a very different niche from projects like KiCAD and Eagle - while they are most often used to design printed circuit boards, Fritzing is most often used to lay out circuits on a breadboard. Each project in Fritzing can have multiple different representations - a schematic layout, a breadboard layout, and a PCB layout. Fritzing also lets you edit code for your projects’ microcontroller right there and save it with the project. And all three are accessible from different tabs in the application.
Many projects only populate one or two of these views (using Fritzing to make PCBs is much less common since the PCB tooling is more basic). You can start with a schematic, and when you switch to the breadboard view Fritzing will show floating “air wires” showing which parts need to be connected on the breadboard using breadboard rows and jumpers. Conversely, you can lay out a project first on the breadboard, and you’ll get “air wires” in the schematic view showing parts of the schematic you need to finish drawing.
It’s important to note that these “air wires” are not considered a finished part of a view and aren’t saved in that view when you save the project.
Fritzing has a fairly large library of built-in components and there are several additional libraries that can be downloaded on demand. These typically have SVG images for the components in the different views, metadata about pin labels, and some kind of textual description of the part’s function. Some parts have parameters that can be manually set (e.g. resistance or voltage rating), but this option is only available for certain core parts that have a special case coded into Fritzing itself.
From a programming side this means each unique part configuration has a unique part ID, unless it’s a core part with a special case. This part ID can be used to look up more information about the part in the library, to give a better description of the part. The parts are defined in XML files that are easy to parse, as are the projects themselves. Fritzing’s project file format is .fzz, which is actually a ZIP file full of XML files, images (for custom parts) and code (if used). That means that even though Fritzing is written in C++, I can use the language of my choice (in this case Python) to write my circuit describer.
What’s next
My first focus is on the schematic layer, and building a tool that can read an arbitrary Fritzing file and describe the schematic’s connections. In fact, this is working already in a basic way. I’m currently refining the process and handling the various special cases better. In many cases this has required me to delve into Fritzing’s codebase to get an understanding of how something works.
This extracted connection information is being stored in data structures that are passed to a separate “describer” module, which produces textual description of the circuit like the ones described in the previous blog post. That means that in the future, we can swap out either the circuit parsing (e.g. to read from Eagle files or images) and the description (to provide different formats for different tastes and audiences).
by Troy