[Solved] Intermittent / burst logs in the Newrelic / ELK

 

Issue:- 

Although the application was writing the logs continuously and shipper shipping but the logs were missing for a particular period and burst of logs with spikes being observed in the Newrelic ELK.

Error:- 

Following graph shows the actual issue of intermittent or burst of the logs in ELK

Effect:-

Due to the non availability of the logs it was becoming difficult to troubleshoot the issue as the logs were getting delayed and sometimes might be missed out as well.

Resolution:-

Printing the error logs or logs required for troubleshooting helps to overcome this issue.

Explanation:-

More than 1million event logs are getting posted in an hour due to which the Disk would be becoming a bottleneck and burst of events are being pushed into the Newrelic ELK.

Lowering down and printing the error logs or logs required for troubleshooting should help to overcome this issue of intermittent logs in the Newrelic/ELK.