Bug #699012 (sf-1075715) “File format documentation out-of-date” : Bugs : gEDA project

Revision history for this message

danmc (danmc) wrote on 2004-11-30:

#1

you are correct, the manual is out of date. I haven't
figured out what the "right" answer (for some reasonable
def. of "right") here is. If you take a look at
src/parse_y.y you'll see the full story, but using a yacc
program as a substitute for a manual is obviously lacking.
The tough thing is that there have been several versions of
PCB's file format and the current pcb program can read all
the old versions. Also various footprints in the library
have been created in various versions.

So I guess I'm open to suggestions on what the most useful
thing would be to have in the manual.

mask is soldermask, clearance is how much to clear when the
pad/pin is enclosed by a plane. I think they are both
diameters, but I'd have to probably do an experiment to say
that with even moderate confidence.

Revision history for this message

whitis (whitis-users-sf) wrote on 2004-12-29:

#2

Download full text (8.6 KiB)

The answers to the questions abut mask and clearance don't
begin to cover it. People need to know if they are
radiuses, diameters, and whether they are absolute (measured
from center) or relative (measured from the outside of the
feature). "clearance" polygon clearances are measured from
the outside of the copper feature, but are off by a factor
of two. "mask" soldermask clearances are measured from
hole center and
are a diameter (for round pins, edge length for square
pins). So all clearances are off by a factor of two and
soldermask clearances are off by the pin/pad thickness.
Pad soldermasks are even more confusing because the shape is
almost never round (many pins aren't round either).
So, not a single clearance parameter (polygon or mask) seems
to actually be expressed as a true clearance (the distance
between two
copper features).

Here are exceprts from my fix_soldermask program and show how
to convert from true clearance to the non-standard forms
used by xpcb.
All true clearances are in 0.01 mil units measured from the
outside edge
of a pad to the inside edge of the polygon or soldermask.

pin_p->mask = pin_p->thickness + new_mask_clearance * 2;
pin_p->clearance = min_pin_clearance *2;
pin_p->drillinghole = new_hole_diameter;
via_p->mask = via_p->thickness + new_mask_clearance * 2;
via_p->clearance = min_via_clearance *2;
pad_p->mask = pad_p->thickness + new_mask_clearance * 2;
pad_p->clearance = min_pad_clearance * 2;
line_p->clearance = min_line_clearance * 2;

The documentation should at least describe the latest file
format and mention there are older, incompatable, formats.
When the file format is changed, the descriptions could be
archived. It is important to retain
old file format documentation to future proof data.

The existing mish mash of formats that need a yacc/lex
parser is a nightmare. The program should have been
written with an extensible
file format. A properly designed file format would:
   - be readable by all future versions of the software (or
other programs
     that read the format such as utilities and other CAD
software)..
   - be readable by older versions of the software (or other
programs)
     except that newer information will be ignored.

This could be accomplished by:
   - including a resolution parameter. The older format
could have been
     handled by using a default value if resolution was not
specified.
    Switching to square brackets instead of parenthesis was
a very bad
    way to handle this. This is then as simple as adding:
          x *= file_scale;
          y *= file_scale;
          drillinghole *= file_scale;
          etc.
     to the existing parser. People could even look at the
new format
     files with the old program, the board would just have
been magnified
     100x.
   - preferably using an XML file format or at least a
format with named fields.
   - never deleting or moving positional fields
   - add new poisitional fields only to the end of the list
and define default
     values.
   - if not using a sensible format (XML or one with named
parameters), include in the file header lines that name the
positional fields for each record type:
    - if n...

The answers to the questions abut mask and clearance don't
begin to cover it.   People need to know if they are
radiuses, diameters, and whether they are absolute (measured
from center) or relative (measured from the outside of the
feature).  "clearance" polygon clearances are measured from
the outside of the copper feature, but are off by a factor
of two.   "mask" soldermask clearances are measured from
hole center and
are a diameter (for round pins, edge length for square
pins).   So all clearances are off by a factor of two and
soldermask clearances are off by the pin/pad thickness.  
Pad soldermasks are even more confusing because the shape is
almost never round (many pins aren't round either).
So, not a single clearance parameter (polygon or mask) seems
to  actually be expressed as a true clearance (the distance
between two
copper features).

Here are exceprts from my fix_soldermask program and show how
to convert from true clearance to the non-standard forms
used by xpcb.
All true clearances are in 0.01 mil units measured from the
outside edge
of a pad to the inside edge of the polygon or soldermask.

pin_p->mask = pin_p->thickness + new_mask_clearance * 2;
pin_p->clearance = min_pin_clearance *2;
pin_p->drillinghole = new_hole_diameter;
via_p->mask = via_p->thickness + new_mask_clearance * 2;
via_p->clearance = min_via_clearance *2;
pad_p->mask = pad_p->thickness + new_mask_clearance * 2;
pad_p->clearance = min_pad_clearance * 2;
line_p->clearance = min_line_clearance * 2;

The documentation should at least describe the latest file
format and mention there are older, incompatable, formats.
When the file format is changed, the descriptions could be
archived.   It is important to retain
old file format documentation to future proof data.

The existing mish mash of formats that need a yacc/lex
parser is a nightmare.   The program should have been
written with an extensible
file format.   A properly designed file format would:
   - be readable by all future versions of the software (or
other programs
     that read the format such as utilities and other CAD
software)..
   - be readable by older versions of the software (or other
programs) 
     except that newer information will be ignored.

This could be accomplished by:
   - including a resolution parameter.  The older format
could have been
     handled by using a default value if resolution was not
specified.    
    Switching to square brackets instead of parenthesis was
a very bad
    way to handle this.    This is then as simple as adding:
          x *= file_scale;
          y *= file_scale;
          drillinghole *= file_scale;
          etc.
     to the existing parser.   People could even look at the
new format
     files with the old program, the board would just have
been magnified
     100x.
   - preferably using an XML file format or at least a
format with named fields.
   - never deleting or moving positional fields
   - add new poisitional fields only to the end of the list
and define default
     values.
   - if not using a sensible format (XML or one with named
parameters), include in the file header lines that name the
positional fields for each record type:
    - if not using a sensible file format, you should never
delete the code
      to write older versions.
    - on another note, try to keep the order of records in
the file the same.
       This is important for diff, patch, CVS, etc.
    -  Include the version number of the file format in the
same 
    - always write the named attributes in a specific order
but don't require
      the same order when reading.
    - XML and similar formats are to some extent self
documenting.
    - a parser could be written in such a way that it
actually generates
       documentation files.

Rules for parsers:
   - use default values if value is omitted.
   - if you read attributes you don't understand, do not
abort the load.  
     Just issue a warning.
    - if at all possible, store unrecognized attributes with
each object
      and write them out when you save the file in the same
format.   
      That way, someone can edit a file created by a newer
version
      of the program (or a different fork) and preserve new
information
      even if it isn't recognized.       This can be done by
adding a pointer
      to a linked list (stoed as text, ATTRIBUTE=VALUE) to
each object.         
      Likewise, any unrecognized objects should be stored as
an unknown 
      object structure with a pointer to a copy of the
original text).   I have
     already done this on one of my utility programs
(gschem2xpcb); f 
     you add a new object type to an element file, the
program will pass it
      through unaltered.

Next time you break the format in a major way (if there is
such a time), switch to a sensible format.   One approach to
xml is illustrated as such:
<ELEMENT [...]>
  <PIN X=100 Y=200 THICKNESS=100 CLEARANCE=100 MASK=100
DRILLINGHOLE=50  NAME="" PINNUMBER="1"  flags="1" />
[...]
</ELEMENT>
(techincally, pure XML also wants quotes around numeric values).
Flags itself might be separated into individual attributes.
  The numeric version is distinctly not futureproof and not
human readable.

The extra space for the field names is non-consequential and
gives
considerable advantages.

One thing that would help with the current situation is to
add the ability to
compile a stand alone compile time version of the program that
just has the load and save code and a command line option
parser.
This could be used to bring old versions of files up to date
or possibly
convert newer files to older formats as needed by other
programs.
This could be used, for example, by:
   - board houses who want to accept xpcb format files
   - other cad programs that want to import xpcb files
   - utility programs could invoke it using a pipe to allow
older/newer 
     versions to be read.
   - can be used to batch convert layout files or symbols to
the newer
     version.
   - it allows you to throw out the existing parser in the
main program
     and move to a future proof file format and use the
conversion program
     in a pipe to read older formats instead of having to be
burdened with
     all the old formats.

It will be hard enough for people in the future to read one
unified format
of xpcb board layouts, let alone the current mishmash of
incompatible formats.

Future proof file formats are important.    Historians,
genealogists, repair technicians, librarians, people who
restore old equipment, and many others depend on reading old
files/documents much further into the future
than one might expect.   Even a document that seems as
inconsequential as a personal letter may be needed a century
later; much of the information in our history books actually
was gleaned from personal letters.   Old paper information
survived.    Badly designed electronic
formats will not survive even if the bits are preserved or
resurected by
future technologies that can read dead media.
Non-consumer equipment often has a lifetime of 20 or 30 
years.   People restore 100 year old radios.     In case of
apocolyptic events, the designs of today could be needed for
a very long time.
Indeed, older designs might be more useful than more recent
ones because they use semiconductors that don't fail after a
few years due
to diffusion.   Being able to convert old files can be
important for legal reasons.  Patents can last something
like 75 years.   The patent office can issue patents for
techniques that have been used for decades.
20 year old computers are still being used by collectors and
museums.
Old technology is sometimes shippped to third world
countries where it
is kept alive long after it is considered obsolete in
industrial countries.
The best speaker design today was developed in the 1945. 
That is 60 years old.   If Paul Klipsh had used a CAD system
to design those speakers, would we be able to read the files
today?   The company still hand builds klipschorns to order
even though there are not enough people with the money and
discriminating taste to permit mass production.
I have laboratory equipment that is 30 years old.    
Expensive medical imaging eqipment (MRI, Cat scan, etc.)
leads 4 lives just in the US:  paying humans, paying
animals, pro bono humans, pro bono animals.
How old will that equipment be by the time it is crated up
and shipped to the poorest country on earth?   The text can
be recovered from a 25 year old wordstar document in seconds
by stripping off the high bit and it would take a programmer
only a few hours to write a utility to recover the original
formating but a ten year old Microsoft Word document is
essentially unreadable today.   A clean HTML document that
doesn't rely on attrocities such as javascript, java, and
activeX will be readable 100 years from now.

Peter TB Brett (peter-b) on 2011-01-07

Changed in pcb:
importance:	Undecided → Medium

Peter Clifton (pcjc2) on 2011-01-07

Changed in pcb:
importance:	Medium → Undecided

Peter TB Brett (peter-b) on 2011-01-07

Changed in pcb:
importance:	Undecided → Medium

Traumflug (mah-jump-ing) on 2015-09-27

Changed in geda-project:
importance:	Undecided → Medium

Bert Timmerman (bert-timmerman) on 2015-09-27

Changed in pcb:
milestone:	none → next-bug-release

Bert Timmerman (bert-timmerman) on 2016-02-29

tags:

added: docs file-format

Bert Timmerman (bert-timmerman) on 2016-12-24

Changed in pcb:
milestone:	next-bug-release → future-bug-release

Revision history for this message

Bert Timmerman (bert-timmerman) wrote on 2018-02-05:

#3

An elaboration for square brackets in the file format lives in topic branch "LP699012".

To be continued.

Kind regards,

Bert Timmerman.

Changed in pcb:
assignee:	nobody → Bert Timmerman (bert-timmerman)
status:	New → In Progress

Revision history for this message

Bert Timmerman (bert-timmerman) wrote on 2018-02-06:

#4

More commits added to this branch, please review.

Kind regards,

Bert Timmerman.

Revision history for this message

Bert Timmerman (bert-timmerman) wrote on 2018-02-20:

#5

Added documentation for the GTK action Benchmark().

Revision history for this message

Bert Timmerman (bert-timmerman) wrote on 2018-02-21:

#6

Added documentation for the GTK action Busy().

Kind regards,

Bert Timmerman.

Revision history for this message

Bert Timmerman (bert-timmerman) wrote on 2018-02-21:

#7

Added documentation for the GTK action Export().

Kind regards,

Bert Timmerman.

Revision history for this message

Bert Timmerman (bert-timmerman) wrote on 2018-02-21:

#8

Added documentation for the GTK action Load().

Kind regards,

Bert Timmerman.

Revision history for this message

Bert Timmerman (bert-timmerman) wrote on 2018-02-23:

#9

Hi,

I added a start for the Toporouter chapter in the manual and pushed that to this topic branch.

Some behaviour of the toporouter still needs to be analysed to conclude documentation on this chapter.

Oh, and an example with graphics is needed ;-)

Kind regards,

Bert Timmerman.

BTW: I hope to merge this additions to master soonish.

Revision history for this message

Bert Timmerman (bert-timmerman) wrote on 2018-03-03:

#10

Hi,

Merged into master.

I consider the User Manual up to date (enough) and close the bug report.

The User Manual still can use some improvements and additions, which are beyond the scope of this report

If anyone has (another) issue with a part of the manual please open a new bug report.

Kind regards,

Bert Timmeman.

Changed in pcb:
status:	In Progress → Fix Committed

Bert Timmerman (bert-timmerman) on 2018-04-07

Changed in pcb:
status:	Fix Committed → Fix Released

gEDA project

File format documentation out-of-date

Bug Description

Other bug subscribers

Remote bug watches

Affects		Status	Importance	Assigned to	Milestone
	gEDA project	New	Medium	Unassigned
	pcb	Fix Released	Medium	Bert Timmerman	pcb pcb-4.1.1