gEDA-dev: SPICE rawwaveform file

[Anthony J Bybell]

On Mon, 3 Dec 2007, michel agoyan wrote:

> In fact I'm looking for a binary format because it is more compact than
> a text file.
> I simple have an huge amount of waveforms to deal with coming from a
> digital scope.
> I want to process them  (average correlation ...) and write the results
> using a format
> that is compact and that allows the use of waveform viewer such as gwave.

What exactly is "a huge amount" of waveforms?  As long as an individual
sim run isn't something like 10GB, it should be manageable.

If you'd like to chop the amount of data down, the following typically
works very well:

1) Separate the signals into different streams.  Include the absolute time
data as a separate stream.

2) Separate each individual floating point value change for a net as
time/sign/exp/mantissa values.  Note that for the time value in this
4-tuple, you want an index into an absolute time array, not the actual
time value.

3) Emit the deltas (compared to previous value) of each of the
time/sign/exp/mantissa to their respective (sub)streams as variable length
integers.

4) Filter through a compressor such as zlib.  Compression ratio will
improve if like streams (e.g., all exponents) are compressed back-to-back
but this will require some experimentation.

I do something similar to this with the vcd recoder in gtkwave and can
read 2GB of vcd into memory on my laptop.  Granted, binary data is
different from floating point, however, the grouping concept of bunching
like data with like data applies.  Note that compression will improve if
you can pre-process the value changes in (2) through a predictor--in that
case instead of deltas for (3) you would emit the error as compared to the
predicted value.  The sourcecode for FLAC probably could be modified or
used for ideas.

-t