Previously I wrote about how I handle Kicad libraries. That all is still quite valid for the current version of Kicad (5.1.6).
I thought that I should add both the Numbers spreadsheet that is my “parts database,” as well as the Python script I use to create an orderable BOM from the BOM exported from a Kicad schematic and that parts database.
The Python script is here.
The Numbers spreadsheet parts list is here. Export the spreadsheet as a CSV file for use with the script.
In a previous post, I discussed two approaches to Kicad library organization. Here, I will go into detail on how I’ve set up my libraries, which use the atomic parts paradigm. Remember: the goal here is to save time and effort. With atomic libraries, you research and vet all parts once, at the time they are put into the library, instead of doing it for each part after completing the layouts for each design.
Before we can put parts into a library, we have to know what parts we want to put into the library! Sure, that’s a tautology, but it’s also true. And it implies that we must have a scheme to keep track of parts.
So, what, exactly, is a “part” (or “component,” which we will use synonymously here)? Simple: it is anything that we want to use in a design. Generally, that means the usual electronic bits (resistors, transistors, op-amps, etc) as well as mechanical parts (standoffs, screws, even enclosures) and things that complete the design but don’t get soldered to a board (external power supplies, user manual). All of these things are kept in a Master Parts List, and everything gets a Company (or House) Part Number. This is all for the convenience of the whole organization: engineering needs to know what parts are allowed for use in designs, purchasing needs to know what to buy, assembly techs need to know what to put in kits, and so forth.
A master parts list can be maintained in a system as complicated as an enterprise-resource system, or in a simply MySQL database, or even as a spreadsheet. A minimal master parts list should contain the company part number (the key), a vendor name, that vendor’s part number, and a description. Nice to have, especially for keeping track of inventory, are entries for quantity on hand and estimated cost. My parts list also includes which library the part lives in, the footprint, and the footprint library associated with the part. My master parts list “database” is nothing more than an Apple Numbers spreadsheet.
My “company part numbers” follow a simple scheme. It’s best explained with an example: A-1000-1 where:
My categories include A for ICs, B for resistors, C for capacitors, D for transistors, E for diodes, F for electro-mechanical (switches, encoders, connectors) and so forth. My category G is for mechanical parts, such as screws, knobs, light pipes, enclosures.
My footprint libraries, all in the .pretty format, are broadly organized by category, and each library name is prefixed with asp_ to avoid collisions with other libraries. Most footprints have an associated 3D model, where I could find one. There is otherwise nothing non-standard about them.
My schematic symbol libraries, all in the Kicad 4.0.x .sch format, are broadly organized by category, and each library name is prefixed with asp_ to avoid collisions with other libraries. None of the symbols have hidden power (or any other type) of pin. All parts have a useful description.
All symbols are for atomic components. This requires that they have a preselected footprint, and the footprints are in my libraries. A part called OPA551PA, which comes in the DIP-8 package, has its Footprint field filled with asp_ic:DIP-8. Also, each part must have a custom PN field, and it is in this field that I enter my company part number.
Yes, this means that each dual op-amp and each transistor and each whatever in the library will have essentially identical symbols. A reasonable response to concerns about this is, “So what!”
Naming the part symbols.
A simple requirement, yes? For most things, yes. ICs tend to be straightforward in this regard. Because the package is defined with atomic parts, the name of the part in the library should reflect it. Thus, my library has both LM317T (in the TO-220 package) and the LM317EMP (the SOT-223 package). These are, of course, the TI full part numbers, which encode the package type. The assumption is that the engineer has read enough of the data sheet to know the package codes so s/he can choose the desired device. The device description should indicate the package, so it’s obvious, and post-4.0.5 nightly builds include the footprint in the schematic symbol browser, so there shouldn’t be any complaints about the part number and “remembering what the extensions mean.”
Connectors and switches and the like are where things can get ugly, quickly. Consider a board-mount male XLR connector. Neutrik has a dozen choices, Switchcraft has many more, and there are other vendors. Most footprints (and panel cut-outs) are incompatible. So calling a library part XLR-M is clearly insufficient, unless you know you will choose exactly one type of this part and never consider another. What I’ve done is to rely on my parts list, Kicad’s symbol browser and its description field and call the part by its manufacturer’s part number. I am familiar enough with Neutrik’s parts to know that the NC3MAH is a board-mount right-angle male XLR connector. What about JB15LPF-JF? Well, my parts list tells me that’s an NKK momentary pushbutton switch with a green LED. Since my parts list doesn’t have every imaginable switch, this is manageable.
Variants and Families and Part Numbers and Symbols
Above I mentioned that my library has the LM317T and LM317EMP devices. Two symbols, two distinct devices … but two company part numbers? Yes, but — I use the part number variant field to distinguish between the two. Thus, LM317T has a part number A-1000-0 and LM317EMP has the part number A-1000-1. This is trivial, I admit, but there is a rule here. A variant indicates that all of the parts with the same category and part number are the same except that they vary in exactly one respect. In the case of our voltage regulators, that one variable is the package. In this manner, one might wish to define a part number for a microcontroller with 32 kB flash and its pin-compatible smaller brother with 16 kB flash. The two can be distinguished by the variant.
Now, the big one: what to do about parts such as resistors, where an entry for each individual value, package and tolerance would quickly cause the library to grow to a ridiculous size? (I should note that I worked for one employer whose library really did include one entry for each resistor value, package and tolerance that they used.) This is where the concept of a “family part number” comes in.
The idea is that the category and the part number indicate a family of parts which vary only in value. The family is represented by only one symbol. Consider the standard 0805 1% resistor. We create a symbol called R0805_1%. Its Footprint field is populated in the usual way. Its custom PN field has B-1000 entered (no variant included). The Value field is left at the default (which is the part name, in this case R0805_1%).
As the engineer places the resistors on the schematic, s/he must edit the part’s Value field appropriately. What does this mean? In my Master Parts List, for these resistors the variant part of the part number is the value. A 100Ω 0805 1% resistor is part number B-1000-100R, so the user enters 100R in the Value field. A 2.67kΩ resistor in that family is part number B-1000-2k67, and as such 2k67 is entered in the value field. And so forth.
When a BOM is exported from EESchema using the bom2csv.xslt script, the resulting CSV file will include the part number and the value for each part in the design. To get a final BOM, with orderable part numbers and manufacturer names and quantities for each part, I parse the CSV against the Master Parts List with Python script. The script “knows” that for parts in the Resistors category B, it must concatenate the part Value with the PN to get the Part Number key into the Parts List, and from there it can pull the manufacturer part number. Parts such as ICs, where the “family” concept isn’t used, have the full Part Number key already from the part’s PN field.
There are likely better ways to build the final BOM. One would be to have the Python script work directly on the schematic, rather than rely on the XSLT script to create the intermediate BOM. Another way would be for the Master Parts List to be built in a proper SQL database, which would import the Kicad CSV BOM and work on it without a Python script.
The topic of how Kicad (and indeed, any other EDA package) libraries should be organized comes up regularly on forums such as the Kicad User forum. There are many opinions, and user preference seems to be mostly based on experience with previously-used systems. The preference also seems to be split down a “professional” versus “hobbyist” point-of-view. This post will attempt to present the two prevailing preferences for Kicad library organization. After that, I will explain the basic system I have created for my own use.
As of the current stable version of Kicad, 4.0.6, Kicad’s libraries are divided into three separate types:
Schematic symbol libraries (which have a .sch file extension) hold all of the symbols the designer can put on a schematic using EESchema.
Footprint libraries (which have a .pretty file extension, and are actually directories which contain the footprints, which have a .kicad_mod file extension) hold the footprints which are used with pcbnew to define how a real component is attached to the board and connected to other parts.
3D model libraries aren’t really libraries, rather, they are just collections of 3D models in VRML format, and for nightly builds also STEP.
These three library types are loosely connected. Each schematic symbol has a “Footprint” field, which can be populated or not. A part might come in more than one footprint, so a user might wish to not have the library symbol include a default. As for footprints, it should be obvious that common footprints are used by thousands of devices, so having a generic SOIC-8 footprint is more useful than having a TL072-SOIC-8 footprint. Finally, all footprints have a 3D Model field, which calls out such.
Users can decide how to organize their libraries. One might reasonable decide to have exactly one schematic library and one footprint library, and those libraries include only parts that have been vetted and put onto an approved parts list. Others might prefer to have a “digital” library, an “analog” library, a “passives” library and so forth for schematic symbols, and an “IC” library, a “passives” library, a “connectors” library and so forth for PCB footprints. Kicad is flexible, indeed agnostic, in this respect.
Since schematic symbols in libraries are not required to have a populated footprint field, Kicad allows for essentially two separate library paradigms, which I will call the “CvPCB flow” and the “atomic library” concept.
This is named for the Kicad tool CvPCB, which is used to map footprints to schematic symbols prior to generating a netlist for pcbnew.
Many users prefer to draw a schematic without concern for the actual components to be used on the board. For example, a user designing an op-amp circuit may wish to place a generic op-amp symbol on the sheet, and then choose generic resistor and capacitor symbols to complete the design. Before layout, obviously footprints must be chosen, so the CvPCB tool is run to do this association. The user may select DIP-8 for the op-amp and standard ¼-watt through-hole parts for the resistors and a standard through-hole ceramic part for the capacitors. (Part values are uninteresting for the layout, so it doesn’t much matter whether the op-amp is a TL072 or an NE5532.) After associating all symbols with a footprint, CvPCB back-annotates the selections to the schematic. The footprint field in each symbol is populated with what was chosen. Then a netlist is generated and pcbnew is launched. The netlist is imported, and the layout starts with all of the chosen footprints.
This flow has advantages. One is that the designer may not know exactly what op-amp is being used, or whether through-hole or surface-mount passives are going to be used. This flexibility is prized by hobbyists.
This flow has disadvantages. The main one, in my view, is that it is utterly disconnected from any reasonable way of generating a bill of materials. Matching the resistors and op-amps put on the schematic to something one can actually buy requires generating a generic BOM from Kicad and then manually editing it to match parts from various vendors. And this manual parts matching occurs for every design, every time. For the hobbyist doing one board design a year, this inefficiency is probably acceptable.
“Atomic parts” is a term used to indicate that the symbol in the schematic library is more than just something generic without footprint or part-number information. It means that the symbol and the footprint (and the 3D model) are combined into one inseparable unit. All symbols have a footprint pre-selected. Ideally, each symbol includes a custom “Part Number” field. This part number field is critical, as it is the hook into a parts list database. This database is not part of Kicad, and can be as simple or complex as the user requires.
A generic “dual op-amp” symbol is no longer allowed. Instead, the user must create, for example, an OPA1652AID symbol with the SOIC-8 footprint set. A USB Type B jack is similar, in that it will explicitly call out the proper footprint. Rather than generic “resistor,” the library will contain RES_0805_1%, indicating that the part is the 0805 size and has a tolerance of 1%. Value of the resistor is given in the symbol’s VALUE field, as one might expect.
The disadvantage to this paradigm is that parts have to be fully defined before the design can begin. The professional user will argue, though, that this requirement is really an advantage. When an OPA1652AID is placed on the schematic, pcbnew already “knows” that it uses the SOIC-8 package. The BOM generated by EESchema also has the correct part number; there is no need to look up parts and create the BOM by hand. The hard work of that parts lookup and vetting is done once and is then used for each design. It is tedious work that is not repeated.
In a following post, I will explain the simple atomic parts library system I’ve created for my own use.
I have created a set of Kicad schematic and footprint libraries. The library archive is available here.
These instructions are for Mac users.
After downloading, copy the archive to your user application support directory at /Users/YourAccountName/Library/Application Support/ and unarchive it (double-clicking in the Finder suffices). It will create a folder in the application support directory called kicad. This is the standard directory for Kicad libraries. Some users might wish to put the libraries into the global library directory, /Library/Application Support/kicad.
Inside that directory are the schematic symbol libraries (in library), the footprint libraries (in modules) and any 3D models I might have (in modules/package3d). The scripting directory has some plug-ins, notably the Python scripts which are used to generate footprints.
The plugins directory has some XSL scripts, most notably the bom2csv.xsl script which is used to generate bills of material from EESchema.
The template directory has a couple of default project templates.
The help directory has the PDF-format manuals for all of the Kicad programs.
You need to tell EESchema where to find the schematic symbol libraries. From EESchema’s main menu, choose Preferences => Component Libraries. A dialog pops up:
Initially it might be populated by Kicad default libraries. (You may have gotten dire warnings from EESchema when you first launched it.) Go through and select and delete each of those default libraries.
Next, make sure that the libraries you unarchived earlier are on one of the paths listed under “Current search path list.” If not, add the path to the search list by hitting the “Add” button next to “User defined search path” and mousing around to that directory (in my case, it’s /Users/andy/Library/Application Support/kicad/library).
Then, hit the “Add” button next to the list of Component library files and selecting each library. Now all of the libraries should be available to use for schematic entry.
Please note: As you add components to the schematic, they will also be added to a file created automagically in your project directory called projectname-cache.lib. DO NOT delete this file! If you do, all of the symbols you added to the design will be lost. (Unlike pcbnew, the current eeschema file format does not embed the symbols in the sheet schematic file, and instead uses this cache. The Kicad developers indicate that they will revise the schematic file format to eliminate this weirdness and embed the symbol in the schematic file.)
Once the library list is set up, you should be able to browse the libraries from within EESchema and place the symbols onto your sheets.
You tell pcbnew where to find footprints by setting an environment variable and then using that environment variable in a list called a “Library Table.” This sounds complicated but it’s very simple.
First, we’ll set the environment variables. These are mainly useful for those who use Kicad on different machines which might have different library locations, for example working on a Mac and then on a Windows machine. From the Kicad project manager main menu, choose Preferences => Configure Paths. The following dialog pops up:
For our purposes, ignore the KIGITHUB line; that is useful if one wishes to access the community footprint libraries that are served by Github.
The KISYSMOD and KISYS3DMOD paths must be set to the directories where you find the footprints and the 3D models, respectively. You can see that here I’ve set them to the correct locations. If they are not correct, or do not exist, then double-click on an incorrect path and enter the correct one. Click OK to save.
Next, you need to create or modify a library table. This is the file that maps the footprint library locations to a pointer that is used in pcbnew to find the footprints. Again, this is all so that you can store the libraries anywhere in the file system on any computer, and as long as the entries in the file table are correct, pcbnew can find the footprints.
By default, the footprint library table is called fp-lib-table and it lives in the directory /Users/yourname/Library/Preferences/kicad. It is a text file and can be opened and changed by any editor. My current library table file, which references all of my PCB libraries, is here.
Modifying the library table is easy. From the pcbnew main menu, choose Preferences => Footprint Libraries Manager. (You may also prefer the Wizard.) This opens a dialog like this:
Here you the location and the contents of my library table. As of Kicad stable 4.0.0, the footprint library is actually a directory with the suffix “.pretty,” and inside that directory are the individual footprints within that library.
For each library, you give it a “nickname,” which is a short name for the library. I recommend making the nickname the same as the library directory sans the .pretty suffix. (Don’t confuse yourself.) If the nickname is the same on all of your different machines, pcbnew will never have a problem finding the library.
Next, note the library path includes the ${KISYSMOD}/ prefix. This text string invokes the environment variables for the library location, and it’s the hook which lets you put the library anywhere in the filesystem and pcbnew will be able to find it. The environment variables are shown at the bottom, and you can change them as you need.
For the most part, “Plugin Type” will be Kicad, which is the “.pretty” format. “Legacy” libraries are in an ancient format Kicad used to use and is now deprecated. Those libraries can be read but not modified. (If they are modified, they are automatically upgraded to the new format.) You can also directly use, but not modify, EAGLE- and gEDA-format footprint libraries. (Note that you cannot use EAGLE or gEDA symbol libraries in your schematics.)
Ignore the “options” for now. And if you prefer, you can enter a short description of the libraries, in case the name itself is insufficient.
Once the library table is set up, it can be used when importing a netlist generated by EESchema, as long as the symbols in the schematic have proper “footprint” entries (all of mine do). If you create your own symbols, it is not strictly necessary to embed a footprint link in them; you can use the CvPCB program within EESchema through the Tools => Assign Component Footprint” menu option. But I don’t recommend this design flow. Create symbols with reasonable names that somehow indicate footprint, and then marry the footprint to the symbol by making sure that the symbol’s “Footprint” field has a valid entry.
You will note that each of the symbols library has a custom part number field called PN, and that field is always filled. This field holds a key into a master parts list I created. The parts list is here, as a zipped Apple Numbers spreadsheet, and here as a zipped CSV export of that database.
A python script called bomgen.py is used to parse the BOM exported from EESchema and look up the parts against the CSV parts list. Put the script and PartsDatabase.csv into the same directory as the CSV BOM exported by EESchema. Run the script from the Terminal like such:
$ python bomgen.py DesignBOM.csv PartsDatabase.csv
You will see messages as the BOM is parsed, and will be greeted at the end with a success or failure message. The result is a CSV file called FinalBOM.csv.
