|
|
Posted by The Natural Philosopher on 09/27/07 14:09
DM wrote:
> "Erwin Moller"
> <Since_humans_read_this_I_am_spammed_too_much@spamyourself.com> wrote in
> message news:46fb87ea$0$244$e4fe514c@news.xs4all.nl...
>> DM wrote:
>>> Hi All,
>>>
>>> I'm writing a program that connects to a data stream containing a number
>>> of
>>> different variable types. Ie 4 & 8 byte integers, ASCII characters and 8
>>> byte floats.
>>>
>>> I use fgetc to capture the data in characters then I reconstruct the
>>> variables from these bytes.
>>>
>>> For integers this is easy, just multiply the bytes by orders of 256 to
>>> get
>>> the right offset values. The thing I can't figure out is how to
>>> reconstruct
>>> a double precision floating point number from its 8 component bytes.
>>>
>>> Is there an easy way to do this in PHP? I couldn't find anything useful
>>> through
>>> Google...
>>>
>>> Any pointers would be greatly appreciated.
>>>
>>> -DM
>>>
>> Hi DM,
>>
>> This data stream, where is it coming from?
>> What program is producing it?
>> Also PHP?
>>
>> Regards,
>> Erwin Moller
>
> Hi Erwin, thanks for the reply.
>
> The data stream is coming from a custom Win32 Visual C++ real-time server
> program which pumps out up to 1,000,000 of these multi-type messages per
> day.
>
> I would like to grab and parse this data, then insert it into a postgreSQL
> database under Linux rather than looking for a Windows solution.
>
> I'm looking for a way to get this data into the postgres database so I can
> utillise a whole suite of PHP diagnostic & analysis program I wrote for
> another similar application.
>
> I had a look at the IEEE standard for double precision floats, but it looks
> like I would have to do around 10-15 calculations per message to manually
> restore the value. This would be impractical for the amount of through-put I
> am facing...
Not necessarily: especially if coded in C or assembler ;-)
It may be possible to 'patch' the postgres stuff to do this.
You could create a new function in there calld 'myfloat' or somesuch and
do 'set var=myfloat(inputstuff)' at the data base level.
Or add a new function to PHP. It has to be relatively trivial.
Or - best of all - write your data->SQL program in C.
If you can buffer database insertions enough, it will be blindingly quick.
From what I remember of floating point stuff, there is usually some
bytes for the mantissa, and some for the exponent. Its easy enough to do
shift left/right of the exponent to get it nto a multiplicand and a
floating point multiply by the exponent nets you the result.
IUf you know the range, its simply a matter of taking the mantissa and
adding lots of zeros to each end, and pitting the decimal point in the
right place, a defined by the exponent. That does fail on VERY large or
small numbers tho.
Tell us what the format is, and we can decide the optimal algorithm:
a million messages a day is only what? 12 a second?. Hardly a hard task
for a decent processor.
>
> -DM
>
>
Navigation:
[Reply to this message]
|