ibmi-brunch-learn

Announcement

Collapse
No announcement yet.

Decode Base64 from IFS

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Decode Base64 from IFS

    I've got a string of data parsed from an XML transmission.
    It's an image encoded in base64.
    We save that to the IFS and my program reads and decodes it into a new .PNG file on the IFS.
    My problem is that it's returning about 100 blank lines of data.

    I'm using ScottK's Base64_decode API.
    I think i've found a few things that are holding me up.

    #1 is my compile data. (image #1)
    #2 my InputData field isnt getting the entire string for some reason. (image #2)
    Any advice?
    See code Below------------------------------------------------------

    H DFTACTGRP(*NO) BNDDIR('FEDEXLIB/BASE64')
    H BNDDIR('QC2LE')
    /copy FEDEXLIB/QRPGLESRC,BASE64_H
    /copy qrpglesrc,httpapi_h
    /copy ifsio_h





    **************
    D BASE64DATA s 72A DIM(43) CTDATA PERRCD(1)
    DxAll s 10000a varying
    Dlong S 10 0 Inz(*Zero)
    *---------------------------------
    D ReportError PR
    **************
    D IFS_file2Var PR 65535a varying
    D pfile 256a Value
    D InputData s 32767a varying
    D OutputData s 32767a varying
    D DecData s 32764a
    D Decoded s 32764a
    D DecLen s 10I 0
    D Len s 10I 0
    D OutputFile s 200a
    D Outfile s 10I 0
    D rc s 10I 0
    D i s 10I 0
    D fd s 10I 0
    ************************************************** *******************

    //Declare the Files
    //this is the binary file we save to IFS first!

    InputData = IFS_file2Var('/fedexlab/new.bin');
    len = %len(InputData);
    OutputFile = '/fedexlab/fedex.png';

    //Prepare the OUT file

    fd = open(outputfile
    : O_WRONLY+O_CREAT+O_TRUNC+O_CCSID
    : S_IRGRP + S_IWGRP + S_IXGRP +
    S_IRUSR + S_IWUSR + S_IXUSR
    : 819);


    callp close(fd);
    fd = open(outputfile:O_WRONLY);

    // decode InputData
    declen = base64_decode(%addr(InputData : *DATA )
    : len
    : %addr(OutputData)
    : %len(OutputData));

    //Write it
    callp write(fd: %addr(OutputData: *DATA): declen);
    callp close(fd);

    *Inlr = *On;

    ******IFS ==> variable pgm----------------------------***************
    P IFS_file2Var b export
    D IFS_file2Var pi 65535a varying
    D pFile 256a value
    *************************
    D open pr 10i 0 extproc('open')
    D filename * value options(*string)
    D openflags 10i 0 value
    D mode 10u 0 value options(*nopass)
    D codepage 10u 0 value options(*nopass)
    D creatcnvid 10u 0 value options(*nopass)
    **************************
    D read pr 10i 0 extproc('read')
    D handle 10i 0 value
    D buffer * value
    D bytes 10u 0 value
    ***************
    D close pr 10i 0 extproc('close')
    D handle 10i 0 value
    D data s 65535a
    D bytesRead s 10i 0
    D handle s 10i 0
    D rc s 10i 0
    D O_RDONLY c 1
    D O_TEXTDATA c 16777216
    *----------------------------------------------

    handle = open(%trim(pFile): O_RDONLY+O_TEXTDATA);
    if handle < 0;
    // something is wrong
    return '';
    endif;

    bytesRead = read(handle: %addr(data): %size(data));
    if bytesRead <= 0;
    return '';
    endIf;
    rc = close(handle);
    return %trim(data);

    P e
    ****************** End of data ****************************************




  • #2
    this is where is stops on my eval of InputData. It's not getting all the data.

    Comment


    • #3
      The STRDBG tool will only show 1024 characters of a variable by default. The fact that its only showing 1024 does not mean that the program isn't reading more than that... That's just how the debugger works.. You can explicitly specify a length if you want to see the whole thing by typing something like this at the command line:
      Code:
      eval inputData:c 32767
      I'm not clear on why this is coded the way it is.
      1. Why do you include HTTPAPI_H when you don't use it?
      2. Why do you have a CTDATA array called BASE64DATA? You don't seem to use it.
      3. You have a lot of limitations due to character string sizes, 65535 in one routine, 32767 and 32764 elsewhere. Why do you do this? Are you sure you'll never need anything longer? These seem extremely small to me.
      4. Why do you open your output file, then close it, then open it again on the next line? Why not just write to it after opening it once?
      5. Why doesn't your program do any sort of error handling? A comment like "something is wrong" won't be very helpful when it fails... you need to know WHAT went wrong, not just "something"
      6. Why split the reading/writing into separate routines?
      7. What is 'xAll' meant to do?

      Comment


      • #4
        Also:

        8. Why do you discard the bytesRead variable after reading the file? This seems like useful information, but you're just ignoring it.
        9. You are doing %trim(data) towards the end. Are you sure that's safe to do? What if blanks are meant to be there?
        10. Why hard-code prototypes in the IFS_File2Var routine? You clearly have the IFSIO_H copybook, but you're hard-coding the prototypes instead of using it
        11. Your first screenshot shows the normal RPG warning (sev 0) messages you get when you ignore the return values on a prototype. Did you have a question about that? Why did you post that?
        12. Your second screenshot shows two completely different sets of base64 encoded data. (They aren't even close.) Why are they different? Were you working on two different files, and wanted to show screenshots of both?

        Comment


        • #5
          A lot of this is copied code from other programs.
          I'm still struggling to really grasp all of it.
          Someone working with me suggested the field lengths.
          xAll was from another program. It just ended up being there.
          The Base64Data array was there when I copied it. I think they threw their string into a compile time array?
          I just decided to use my file out of the IFS



          Comment


          • #6
            The screenshot with the split is actually the same data. I just paged down to the end of of (on the left)

            Comment


            • #7
              No one knows anything about web services/ IFS at this company.

              I was assigned to decode this XML string. into a PNG image.
              I'm trying to get a working proof of concept so that I can understand it.

              Comment


              • #8
                Ok, I'll try to help you... though, I'm not 100% clear on what format your input file is. Is it just raw base64 data? Is it broken into lines? are there header/footer records in it? Not sure about any of that, so I'll assume it's just one big raw base64 string with no additional formatting. (Since that's easiest for me to code)

                That being the case, here's a quick program I threw together to decode it:

                Code:
                **free
                
                ctl-opt dftactgrp(*no) option(*srcstmt:*noshowcpy)
                        bnddir('BASE64');
                
                /copy ifsio_h
                /copy base64_h
                
                dcl-s ifs_in  varchar(5000);
                dcl-s ifs_out varchar(5000);
                dcl-s err      varchar(500);
                
                dcl-pr QMHSNDPM extPgm;
                  Msgid          char(7) const;
                  Msgf           char(20) const;
                  msgdta         char(32767) const options(*varsize);
                  msgdtalen      int(10)  const;
                  msgtype        char(10) const;
                  callStackEntry char(10) const;
                  callStackCount int(10)  const;
                  MessageKey     char(4);
                  ErrorCode      char(32767) options(*varsize);
                end-pr;
                
                dcl-ds ApiErrEscape qualified;
                  bytesProvided  int(10) inz(0);
                  bytesAvailable int(10) inz(0);
                end-ds;
                
                dcl-s MsgKey char(4);
                
                ifs_in = 'input.b64';
                ifs_out = 'output.png';
                
                if b64_decode_stmf(ifs_in: ifs_out: err) = -1;
                   QMHSNDPM( 'CPF9897'
                           : 'QCPFMSG   QSYS'
                           : err
                           : %len(err)
                           : '*ESCAPE'
                           : '*PGMBDY'
                           : 1
                           : msgkey
                           : ApiErrEscape );
                endif;
                
                *inlr = *on;
                return;
                
                
                dcl-proc b64_decode_stmf export;
                
                  dcl-pi *n int(10);
                    stmf_in  varchar(5000) const options(*trim);
                    stmf_out varchar(5000) const options(*trim);
                    errorMsg varchar(500)  options(*nopass:*omit);
                  end-pi;
                
                  // Use these to get C-style 'errno' value
                  // and its corresponding error msg.
                
                  dcl-pr get_errno_ptr pointer extproc('__errno');
                  end-pr;
                  dcl-pr strerror pointer extproc(*dclcase);
                    errnum int(10) value;
                  end-pr;
                
                  dcl-s p_ifsErrno pointer inz(*null);
                  dcl-s ifsErrno int(10) based(p_ifsErrno);
                
                  dcl-s rdd int(10);
                  dcl-s wrd int(10);
                  dcl-s enclen int(10);
                  dcl-s declen int(10);
                  dcl-s errMsg varchar(500);
                  dcl-s rtnErrMsg ind inz(*off);
                
                  // NOTE: Sizes here are important!
                  //       'encbuf' size must be a multiple of 4 bytes
                  //       'decbuf' should be 3/4 size of encbuf.
                
                  dcl-s encbuf char(32768);
                  dcl-s decbuf char(24576);
                
                  if %parms >= %parmnum(errorMsg) and %addr(errorMsg) <> *null;
                    rtnErrMsg = *on;
                  endif;
                
                  // -------------------------------------------------
                  // Open input file.
                  // -------------------------------------------------
                
                  rdd = open( stmf_in
                            : %bitor(O_RDONLY : O_TEXTDATA : O_CCSID)
                            : 0
                            : 0 );
                  if rdd = -1;
                    if rtnErrMsg;
                      p_ifsErrno = get_errno_ptr();
                      errorMsg = 'input file: ' + %str(strerror(ifsErrno));
                    endif;
                    return -1;
                  endif;
                
                  // -------------------------------------------------
                  // Create output file
                  // (remove it if it exists)
                  // -------------------------------------------------
                
                  unlink( stmf_out );
                
                  wrd = open( stmf_out
                            : %bitor( O_WRONLY : O_CREAT : O_EXCL
                                    : O_CCSID : O_INHERITMODE )
                            : 0
                            : 819 );
                  if wrd = -1;
                    callp close(rdd);
                    if rtnErrMsg;
                      p_ifsErrno = get_errno_ptr();
                      errorMsg = 'output file: ' + %str(strerror(ifsErrno));
                    endif;
                    return -1;
                  endif;
                
                
                  // -------------------------------------------------
                  //   spin through file, decode in 32k chunks
                  // -------------------------------------------------
                
                  enclen = read( rdd: %addr(encbuf): %size(encbuf));
                  dow enclen > 0;
                     declen = base64_decode( %addr(encbuf)
                                           : enclen
                                           : %addr(decbuf)
                                           : %size(decbuf) );
                     callp write( wrd: %addr(decbuf): declen);
                     enclen = read( rdd: %addr(encbuf): %size(encbuf));
                  enddo;
                
                  // -------------------------------------------------
                  //   close files, we're done
                  // -------------------------------------------------
                  callp close(wrd);
                  callp close(rdd);
                
                  return 0;
                
                end-proc;
                I made the subprocedure exported, figuring you'd probably want to put it in a service program somewhere... but it should work fine declared inline in the program, too.

                This is only lightly tested, as I am very short on time this month.

                Comment


                • #9
                  First of all, thank you so much for helping me out. I got thrown into this project with NO prior knowledge of HTTP or even the IFS. It's been a really steep learning curve. So I really appreciate your time.

                  The format of the Base64 file is one of the most confusing aspects of this project to me.
                  It's a fedex shipping label that is sent though a web services XML transmission. (see image)

                  I know that the Base64_Decode only accepts EBCDIC, but I don't really understand how the text will be encoded once we parse it from the XML and save it to the IFS.
                  We aren't quite there in the project yet, so I have been copying the image string from sample data and trying to alter it in many ways, then FTP onto the IFS for decode.
                  ?I've saved it as a small HTML file with inside this...<img src="data:image/png;base64," /> and it shows the label
                  ? I've saved it as a .txt and a .bin and tried to decode it.
                  ?I've taken the string and converted it to EBCDIC and tried it as both a .txt and a .bin
                  I'm 100% sure the format/encoding will be different once we start receiving the XML transmissions.
                  The image I attached might be able to explain it better than me.
                  Thanks again!!!

                  Comment


                  • #10
                    Maybe you can adapt what I've posted?

                    There are many articles/examples out there on how to parse an XML file. All you need to do is read the XML and write the base64 data to a file with the base64 encoded data all as one string, as I mentioned above... then my code will work for you.

                    Comment


                    • #11
                      Yes I think I'm a lot closer to my goal now that you have helped me.
                      We don't have the free form declaration ability for the PR/PI's and what not.
                      I'm struggling a little but to get that correct but I know that's something I can do.
                      Thanks again man!

                      Comment


                      • #12
                        When you have the time can you tell me what this is supposed to look like in structured? the *N is confusing me..

                        dcl-pi *n int(10); stmf_in varchar(5000) const options(*trim); stmf_out varchar(5000) const options(*trim); errorMsg varchar(500) options(*nopass:*omit); end-pi; Also this... I'm not sure how to make it a pointer? dcl-pr strerror pointer extproc(*dclcase); errnum int(10) value; end-pr; I guessed something like this?

                        Comment


                        • #13
                          Code:
                               P b64_decode_stmf...
                               P                 B                   export
                          
                               D                 pi            10i 0
                               D    stmf_in                  5000a   varying const options(*trim)
                               D    stmf_out                 5000a   varying const options(*trim)
                               D    errorMsg                  500a   varying options(*nopass:*omit)
                          
                                 ... etc ...
                          
                               P                 E
                          Code:
                               D strerror        PR              *   extproc('strerror')
                               D    errnum                     10i 0 value

                          Comment


                          • #14
                            Thanks again man.
                            Everything is decoding perfectly.
                            I'm working on using the PRTSTMF routine I downloaded to get it to a Zebra printer, but thanks to you I'm well on my way.
                            You da' man!

                            Comment

                            Working...
                            X