Send data from Logstash to Axiom

input{
  exec{
    command => "date"
    interval => "1"
output{
  opensearch{
    hosts => ["https://api.axiom.co:443/v1/datasets/$DATASET_NAME/elastic"]
    # api_key should be your API token
    user => "axiom"
    password => "$TOKEN"
​
Combining filters with conditionals on Logstash events
Logstash provides an extensive array of filters that allow you to enhance, manipulate, and transform your data. These filters can be used to perform tasks such as extracting, removing, and adding new fields and changing the content of fields.
Some valuable filters include the following.
​
Grok filter plugin
The Grok filter plugin allows you to parse the unstructured log data into something structured and queryable, and eventually send the structured logs to Axiom. It matches the unstructured data to patterns and maps the data to specified fields.
Here’s an example of how to use the Grok plugin:
input{
  exec{
    command => "axiom"
    interval => "1"
filter {
  grok {
    match => { "message" => "%{COMBINEDAPACHELOG}" }
  date {
    match => [ "timestamp" , "dd/MMM/yyyy:HH:mm:ss Z" ]
  mutate {
    add_field => { "foo" => "Hello Axiom, from Logstash" }
    remove_field => [ "axiom", "logging" ]
output{
  opensearch{
    hosts => ["https://api.axiom.co:443/v1/datasets/$DATASET_NAME/elastic"]
    # password should be your API token
    user => "axiom"
    password => "$TOKEN"
This configuration parses Apache log data by matching the pattern of COMBINEDAPACHELOG.
​





    
Mutate filter plugin
The Mutate filter plugin allows you to perform general transformations on fields. For example, rename, convert, strip, and modify fields in event data.
Here’s an example of using the Mutate plugin:
input{
  exec{
    command => "axiom"
    interval => "1"
filter {
  mutate {
    rename => { "hostname" => "host" }
    convert => { "response" => "integer" }
    uppercase => [ "method" ]
    remove_field => [ "request", "httpversion" ]
output{
  opensearch{
    hosts => ["https://api.axiom.co:443/v1/datasets/$DATASET_NAME/elastic"]
    # password should be your API token
    user => "axiom"
    password => "$TOKEN"
This configuration renames the field hostname to host, converts the response field value to an integer, changes the method field to uppercase, and removes the request and httpversion fields.
​
Drop filter plugin
The Drop filter plugin allows you to drop certain events based on specified conditions. This helps you to filter out unnecessary data.
Here’s an example of using the Drop plugin:
input {
  syslog {
    port => 5140
    type => syslog
filter {
  if [type] == "syslog" and [severity] == "debug" {
    drop { }
output{
  opensearch{
    hosts => ["https://api.axiom.co:443/v1/datasets/$DATASET_NAME/elastic"]
    # password should be your API token
    user => "axiom"
    password => "$TOKEN"
This configuration drops all events of type syslog with severity debug.
​
Clone filter plugin
The Clone filter plugin creates a copy of an event and stores it in a new event. The event continues along the pipeline until it ends or is dropped.
Here’s an example of using the Clone plugin:
input {
  syslog {
    port => 5140
    type => syslog
filter {
  clone {
    clones => ["cloned_event"]
output{
  opensearch{
    hosts => ["https://api.axiom.co:443/v1/datasets/$DATASET_NAME/elastic"]
    # password should be your API token
    user => "axiom"
    password => "$TOKEN"
This configuration creates a new event named cloned_event that is a clone of the original event.
​
GeoIP filter plugin
The GeoIP filter plugin adds information about the geographical location of IP addresses. This data includes the latitude, longitude, continent, country, and so on.
Here’s an example of using the GeoIP plugin:
input{
  exec{
    command => "axiom"
    interval => "6"
filter {
  geoip {
    source => "ip"
output{
  opensearch{
    hosts => ["https://api.axiom.co:443/v1/datasets/$DATASET_NAME/elastic"]
    # password should be your API token