Dealing with bad CVS data and load data infile

    Date: 06/27/07 (MySQL Communtiy)    Keywords: no keywords

    I use the following query to rip the contents of a CSV file into a temp table:

    protected $loadfiledef = "
    load data concurrent local infile '%s'
    into table `%s`
    FIELDS
    TERMINATED by \",\"
    OPTIONALLY ENCLOSED BY '\"'
    LINES
    TERMINATED by \",\r\n\"
    IGNORE 1 LINES
    ;";

    It's been pretty effective over running a script to parse individual lines, except for weird events where a field ends with a \ like so.
    "JASMINE ESTATES\",80602,3,3,"RES"

    NOTE: Above is only 4 fields out of 180 fields for that row.

    That entire line ends up being mangled and causing corruption issues for all the following fields until the line is terminated.

    I suppose I could try using sed or something similar to look for \", and replace it with \\", but I think this is just a bad value and something not checked by the IDX source.

    Source: http://community.livejournal.com/mysql/115490.html

« two byte japanese || New Project »


antivirus | apache | asp | blogging | browser | bugtracking | cms | crm | css | database | ebay | ecommerce | google | hosting | html | java | jsp | linux | microsoft | mysql | offshore | offshoring | oscommerce | php | postgresql | programming | rss | security | seo | shopping | software | spam | spyware | sql | technology | templates | tracker | virus | web | xml | yahoo | home