Collectives™ on Stack Overflow

Find centralized, trusted content and collaborate around the technologies you use most.

Learn more about Collectives

Teams

Q&A for work

Connect and share knowledge within a single location that is structured and easy to search.

Learn more about Teams

We want to implement service request trace using http plugin of logstash in JSON Array format.

We are getting the following error when trying to parse the JSON array:

error:

:message=>"gsub mutation is only applicable for Strings, skipping", :field=>"message", :value=>nil, :level=>:debug, :file=>"logstash/filters/mutate.rb", :line=>"322", :method=>"gsub"}
:message=>"Exception in filterworker", "exception"=>#<LogStash::ConfigurationError: Only String and Array types are splittable. field:message is of type = NilClass>
My json array is :
    "data": [
            "appName": "DemoApp",
            "appVersion": "1.1",
            "deviceId": "1234567",
            "deviceName": "moto e",
            "deviceOSVersion": "5.1",
            "packageName": "com.DemoApp",
            "message": "testing null pointer exception",
            "errorLog": "null pointer exception"
            "appName": "DemoApp",
            "appVersion": "1.1",
            "deviceId": "1234567",
            "deviceName": "moto e",
            "deviceOSVersion": "5.1",
            "packageName": "com.DemoApp",
            "message": "testing illegal state exception",
            "errorLog": "illegal state exception"
my logstash config is :
    input {
            http {
            codec => "plain"
    filter{
            json {
                  source => "message"
            mutate { gsub => [ "message", "},", "shr" ] }
            split {
                  terminator => "shr"
                  field => "data"
output {
 stdout { codec => "json" }
    gelf{
        host => localhost
        facility => "%{type}"
        level =>["%{SeverityLevel}", "INFO"]
        codec => "json"
       file{
        path => "/chroot/result.log"
Any help would be appreciated.
                i think you would escape special chars in the regex: mutate { gsub => [ "message", "\}\,\r\n\r\n\{", "\}shr\{" ] }
– Kadir
                Feb 6, 2017 at 8:06
Logstash has a default metadata field named message. So your json message field is overlapping that. Consider changing json field name message to another.
The other option maybe using target setting and referencing the target field like:
json { source => "message" target => "data"}
mutate { gsub => [ "[data][message]", "\}\,\r\n\r\n\{", "\}shr\{" ] }
I hope this helps.
                @kair tried the above config as well... still getting the same error.   {:timestamp=>"2017-02-07T11:02:00.192000+0530", :message=>"gsub mutation is only applicable for Strings, skipping", :field=>"[data][message]", :value=>nil, :level=>:debug, :file=>"logstash/filters/mutate.rb", :line=>"322", :method=>"gsub"}
– Akshay Agarwal
                Feb 7, 2017 at 5:34
                ok I am now able to split my data.... have edited my config accordingly. I am now facing a new issue. My fields are being renamed as : data_appName instead of just being appName.  Any suggestion as to how to avoid this and get the field name that is needed.    I have tried out the following to achieve it as well :      ` mutate         {               rename => {"data_appName" => "appName"}         }`  But still got no positive output.
– Akshay Agarwal
                Feb 7, 2017 at 6:10
                As far as i know mutate { rename => {"[data][appName]" => "appName"} } should work. A better way is to rename all fields like above then remove the data field. mutate {             rename => [                     "[data][appName]", "appName",                     "[data][appVersion]", "appVersion", 					.. 					..             ]             remove_field => "data"     }
– Kadir
                Feb 7, 2017 at 7:22
        
Thanks for contributing an answer to Stack Overflow!
Please be sure to answer the question. Provide details and share your research!
But avoid …
Asking for help, clarification, or responding to other answers.
Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.