Tech Issues

Unusual Arbitrary Attribute

Warning !

This content is based on the the issue I faced while doing analyzing with Arbitrary attribute of HL7. It is common for cases where arbitrary attributes are similar to the case of HL7

When we are fetching data in WSO2 DAS through ESB from HAPI test-panel, we are also getting arbitrary attributes related to HL7 Messaging addition to existing attributes related to message flow. Basically these arbitrary attributes are for the content of message passed, it contains detailed classification of each and every element of HL7 messages that was transmitted.

For better use case of HL7 messaging we need to analysis these messages by indexing these arbitrary field. We can add these arbitrary field as normal or generic arbitrary field. But when we are analyzing we can’t directly use them.

It is because these arbitrary fields are consists of “.” which make following error

Caused by: org.apache.spark.sql.AnalysisException: cannot resolve '_MSH.MessageType' given input columns: 
[meta_server_name, correlation_activity_id, type, timestamp,__MSH.MessageType , content,service_name, meta_host, 
operation_name, message_direction, status];

To resolve this error we need to add “`” this while doing COUNT or SELECT  or INSERT query (not essentially require for CREATE schema) in spark.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s