|
|
Posted by Centurion on 10/06/31 11:36
pc wrote:
> hi all,
>
> I have been blessed with the task of writing a web based database
> representing the state of our globally installed isam databases.
>
> there are basically four steps in setting this up:
>
> 1) schedule isam file reports to run at all remote sites. this
> generates a text file describing the current state of tables (files) in
> the database.
>
> 2) schedule an ftp job to pick up these text reports and dump them on a
> central server
>
> 3) parse the reports and write the information to a mysql database
>
> 4) front end web based gui. make the information available (reports,
> alerts, etc.)
>
> Our company wants to use PHP for the front end GUI. Not having had much
> experience with PHP it seems to me that the way to do the above would
> be to do steps 1, 2 and 3 with Shell / Perl scripting (Perl being
> designed for text file manipulation) and then to use PHP to provide the
> GUI.
>
> Anyone's thoughts and / or input on this matter would be much
> appreciated.
Steps #1 and #2 could be simplified (and conserve bandwidth if that's
important) by using rsync on the remote sites. Not sure what OS your
remote sites are using but I'm pretty sure rsync is available for Windows.
Generate an MD5 for the file and send that to database feeder too. That
way it can check to see if the file it's about to load is valid/complete.
Step #3 could be done in PHP, but if you're more comfortable with Perl, then
knock yourself out. Populating a database is one of the jobs that seems to
lend itself to Perl/Python. Additionally, you could daemonize the
process/script so it detects when steps #1/2 have changed or added files
that need pumping into the database.
Decide on whether you want a fundamental push or pull model. Personally, in
these situations, I go for a push model: let the database populating
functions sit and wait for the remote sites to feed it data. This absolves
the db-feed-process of dealing with things like incomplete file loads or
bad network connections etc and makes it all a lot more reliable. Make the
remote sites do some basic validation like checking the whole file was
uploaded (rsync return codes), sending an MD5 sum for the feeder to verify
against, that sort of thing.
Have fun!
James
--
A musician, an artist, an architect:
the man or woman who is not one of these is not a Christian.
-- William Blake
Navigation:
[Reply to this message]
|