Posted by "Alan Lord" on 10/11/05 21:47

Thanks for the reply.

I am indeed importing data, but from far away over the 'net, and it
changes frequently so needs updating regularly.

I like the idea of creating a file (I'll do it ram rather than a text
file I think) with the insert strings...

Thanks

Alan

> -----Original Message-----
> From: Brent Baisley [mailto:brent@landover.com]
> Sent: 11 October 2005 19:35
> To: Alan Lord
> Cc: php-general@lists.php.net
> Subject: Re: [PHP] Speed/Efficiency Question: Large array into DB
>
> Sounds to me like you are trying to import data. In which
> case it would be best to create a text file and import it
> rather than putting PHP in the middle.
>
> But if you are going us to PHP, I would not do individual INSERTs.
> The overhead will really slow things down with number of
> inserts you are doing. Use the multiple value insert syntax
> and do multiple bulk inserts or just one big one. Although
> doing one big one will end up being an all or nothing deal.
> With multiple bulk inserts, you'd be able to write to a log
> file between inserts so you can track it's progress.
>
> I would step through the array, creating the string for
> inserting (value1, value2, value3, value4),(value1, value2,
> value3, value4), (value1, value2, value3, value4),...
>
>
> On Oct 11, 2005, at 12:28 PM, Alan Lord wrote:
>
> > Hi All,
> >
> > Not being a professional programmer I am looking for some
> advice about
> > the best way to do the following:
> >
> > I have a large multidimensional array (up to 7Mb) It
> contains a list
> > of categories in a fairly normal Cat_ID and Parent_ID
> arrangement. The
> > array also contains a great deal of superfluous information which I
> > want to ignore.
> >
> > I wish to walk through the entire array, select 4 key/value
> pairs from
> > each sub-array and write those to my Database (Keys are: Cat_ID,
> > Parent_ID, CatName, CatLevel). I wish to ignore all other
> key/ values
> > in each array.
> >
> > I'm guessing that I should use array_walk_recursive() and apply my
> > test for valid keys then write this into my db... But, should I do
> > this for each iteration, or make a new array with just the
> information
> > I need then write that into my DB? And do I use a
> transaction/commit
> > block and try to do the whole thing in one db-write or
> again, INSERT
> > on each step through my array?
> >
> > Oh yes, the key names in the array correspond to field names in my
> > database.
> >
> > Sorry it's a bit long-winded...
> >
> > Thanks in advance for any suggestions.
> >
> > Alan
> >
> > --
> > PHP General Mailing List (http://www.php.net/) To
> unsubscribe, visit:
> > http://www.php.net/unsub.php
> >
> >
> >
>
> --
> Brent Baisley
> Systems Architect
> Landover Associates, Inc.
> Search & Advisory Services for Advanced Technology Environments
> p: 212.759.6400/800.759.0577
>
>
>

[Back to original message]