Removing empty records from IFS stream (.csv) files

[WilliamTasker]

Hi Folks

Situation is: using CPYFRMIMPF to convert .csv files in the IFS to a PF.

No control over the data in the .csv (coming from outside). I don't have the authority to insist on cleaner data.

I'd like to write errors to an ERRRCDFILE during CPYFRMIMPF as a first sweep for problems (the second level help is most informative), but most of the .csvs have blank records at the end, so even if there are no other errors, the process will fail there. At present I'm monitoring CPF2817 to prevent that.

So, I would like to either remove the blank records in situ before converting the file, or if there's some other workaround I'm open to it.

[JonBoy]

One option you might consider is to move away from CPYFRMIMPF and instead use something like my Open Access IFS handler as described in this article. That would allow you - in the handler - to validate at a field level and even "fix" the data where appropriate in addition to being able to log any validation errors in a more meaningful manner than anything you can get from CPYFRMIMPF.

You can find the article here: http://ibmsystemsmag.com/ibmi/develo...nput_handlers/ and a followup one here: http://ibmsystemsmag.com/ibmi/develo...nput_handlers/

[WilliamTasker]

Thanks JonBoy

A rewrite is out of scope unfortunately, but I found the answer:

sed /^,/d filename

That's the easy bit. The annoying amount of work you have to do to implement it is due to the fact that there doesn't seem to be any way to make IBM i version of SED change the file in place. So you direct it to another file (even though you didn't want two files), and now it's no longer the right CCSID, so you fix that and now you find it has changed the CRLF endings to LF in the new file, so you fix that ... why make life so hard. Anyway it's done now.

I'll be glad to get back to proper OS/400 where you're trusted to change files when you need to and if you copy something it's an honest to goodness copy.