ibmi-brunch-learn

Announcement

Collapse
No announcement yet.

YAJL - Are all special characters handled while creating/sending JSON payload?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • YAJL - Are all special characters handled while creating/sending JSON payload?

    Trying to create & post a JSON file to a URL using YAJL, but facing issue with certain special character.
    Can you please assist on resolving this?

    Details below:
    JSON is successfully created & received at destination. Data contains a special character ¬ (Logical negation symbol)
    When I opened ".json" file through notepad, the special character could be seen as is. However when I opened ".json" file directly through ACS IFS folder or with textpad ¬ is displayed as a different special character.

    I am in the process of verifying with receiver end as how they received this special character.
    Meanwhile would like to confirm if YAJL can take input of any special character (including ¬ but not limited to) and send the same way to destination?

    Unable to access Scott Klement's YAJL link due to timeout: www.scottklement.com/yajl

    Code snippets:
    Write to JSON -
    yajl_addChar('deliveryAddress': %trim(addr));

    Copy to Buffer -
    yajl_copyBuf( CCSID
    : %addr(myJSON:*data)
    : %len(myJSON)
    : length );

    Set CCSID, add headers & Post payload -
    http_setCCSIDs( 1208: 0 );

    http_xproc( HTTP_POINT_ADDL_HEADER
    : %paddr(add_headers) )

    rc = http_url_post( %trim(URL)
    : %addr( myJSON : *data )
    : %len( myJSON )
    : %Trim(LogFile)
    : HTTP_TIMEOUT
    : HTTP_USERAGENT
    : 'application/json; charset=UTF-8');

    Any inputs on this issue would be helpful!

  • #2
    yajl_addChar() will convert from your job CCSID to UTF-8.

    yajl_copyBuf(), the way you've coded it, will convert from UTF-8 to whatever is in your "CCSID" variable.

    http_url_post() will convert from your job CCSID to UTF-8 (because you've used http_setCCSIDs to tell it to do that.)

    Personally, I'm not a huge fan of converting to/from EBCDIC like this, since EBCDIC only supports a tiny percentage of the characters that UTF-8 supports. But, if the logical negation symbol exists in your job CCSID, this should work -- provided that you are encoding it properly to begin with. And provided that your 'CCSID' variable is your job CCSID, which you haven't told us.

    If you're still stuck, I would suggest that you determine where the process is failing. I wouldn't recommend basing your troubleshooting on what the character "looks like" on the screen, because that can too often lead to mistakes and confusion. Instead, I would look at its underling hex value. Is it the correct hex value for the CCSID that the data is supposed to be in at a given point in time? For example, if the CCSID is UTF-8, then it should have the proper hex value for the logical negation symbol in UTF-8. If it's in the job CCSID, then it should have the proper hex value for the logical negation symbol in your job's CCSID, etc. Looking at it that way, you should be able to determine precisely where it is being coded incorrectly.

    Comment


    • #3
      Thanks Scott for the response.

      Job CCSID is 65535. But variable CCSID was initialized with zero.
      For yajl_copyBuf(), I tried with CCSID as 1208, but that gave an Internal server error, so kept CCSID as zero and did not get that error anymore. Could you please suggest if I am missing anything.
      I actually do not want to convert from UTF-8 to EBCDIC, as receiver end also expects only UTF-8 encoding.

      Comment


      • #4
        0 is a special value that means "my job's CCSID". When the job's CCSID is set to 65535, it will fall back to the job's "default ccsid". You can find out what that is by typing DSPJOB, taking option 2 (Display job definition attributes) and paging down a few times. You'll see something like this:

        Code:
        Coded character set identifier  . . . . . . . . . :   65535
        Default coded character set identifier  . . . . . :   37
        (None of this is specific to YAJL or HTTPAPI, this is a feature of the IBM i operating system.)

        It's a bit confusing that you use a literal 0 in some places, but use a variable named CCSID that's set to 0 in others... that made me think they might be different, but of course, now that I know that CCSID=0, I know that they are the same. So that is correct behavior, and you should be getting the correct symbol provided that your job CCSID supports this character. If it is not, please look at the steps and check the hex values as I described above.

        Comment


        • Scott Klement
          Scott Klement commented
          Editing a comment
          In this example, my "default ccsid" is 37

      • #5
        Checked the Hex value and it matches the Default CCSID (37 - from job) value (5F) for the logical negation symbol. Thanks for your inputs Scott, it really helped to understand things better.

        Comment


        • #6
          i'm facing issues down the same path I believe...

          my rpgle pgm...

          d g_docNode s like(yajl_val)
          d g_payNode s like(yajl_val)
          d g_node s like(yajl_val)

          *---------------------------------------------------------------------
          * Support Objects
          *---------------------------------------------------------------------
          d jsonOBJ s 65535 varying
          d jsonOUTData s 65533

          *---------------------------------------------------------------------
          * PI
          *---------------------------------------------------------------------
          d WS000000 pi
          d jsonINP like(jsonOBJ)
          d jsonOUT like(jsonOBJ)


          EVAL jsoninp
          JSONINP =
          ....5...10...15...20...25...30...35...40...45...50 ...55...60
          1 '{"length":"121","string":"{"env":"dev","command": "GET'
          61 '_LOAN_BORROWER","payload":[{"loanNumber":0000002428}]}"'
          121 '} '
          181 ' '

          // load jSON object
          g_docNode = yajl_buf_load_tree(%addr(jsonINP) + 2
          :%len(%trim(jsonINP))
          :g_errMsg);

          return g_errMsg;

          g_node = YAJL_object_find(g_docNode: 'env');
          if yajl_is_false(g_node);
          g_errMsg = 'Error - env Not Found In Input';
          return g_errMsg;
          endif;

          g_environment = yajl_get_string(g_node);

          Again, everything runs successfully until the last statement (and this used to be working code on another box), though g_node shows up as G_NODE = SPP:*NULL and g_environment ends up with blanks... what am I overlooking?

          I am under the impression it has to do with the fact that I am testing this from the i and in am supplying the json input string via a column i get out of a table. CCSID 37

          This may not be the correct EBCDIC format it should be receiving.

          Can someone help me confirm that I must provide the EBCDIC string format to the program, and best way to do that under my scenario?

          Comment


          • #7
            sorry for repeated posts - my browser hung when i hit post reply and I cannot see how to remove subsequent repeat posts

            Comment


            • #8
              This appears to be related to the converation that Jay posted on midrange.com. I assume it is already solved?

              Comment


              • #9
                This appears to be related to the converation that Jay posted on midrange.com. I assume it is already solved?

                Comment

                Working...
                X