How to create elastic beat from your code snippet
This article is based on the beats developer guide. Elastic supports only golang for beats at the time of writing. The full source code for this beat is available in cloudronics github repository
Elastic beats are shippers which sits on servers, containers etc. gathering data from the host and sending it to elasticsearch server. Read more about it here.
Table of Contents
- Versions
- Introduction
- Check for file availability on a server — snippet
- Convert the snippet to Elastic beat
- Conclusion and further reading
- References
Versions
Introduction
Lets say you have some code which analyses your system and you would like to monitor the same in a time series fashion i.e. the outcome over a limited/unlimited duration. Here I am going to give an example of check for file availability on a server. This snippet notifies when a file appears on a server. This case is useful for monitoring queues, buffers etc.
Check for file availability on a server — snippet
Here is a example snippet which finds and prints the occurrence of a fileabc
under /home/go
directory.
Convert the snippet to Elastic beat
Lets first name the beat: fileoccurancebeat.
- Export
env
variables
export USER=user1
export GOPATH=/home/${USER}/go
2. Create directories and clone the elastic beat repository
mkdir -p ${GOPATH}/src/github.com/elastic
git clone https://github.com/elastic/beats ${GOPATH}/src/github.com/elastic/beats
git checkout 7.2
3. Generate the beat
mkdir ${GOPATH}/src/github.com/${USER}
cd ${GOPATH}/src/github.com/${USER}
python $GOPATH/src/github.com/elastic/beats/script/generate.py
Beat Name [Examplebeat]: Fileoccurancebeat
Your Github Name [your-github-name]: cloudronics
Beat Path [github.com/cloudronics/fileoccurancebeat]:
Firstname Lastname: Cloudronics Oy
4. Set it up
cd ${GOPATH}/src/github.com/cloudronics/fileoccurancebeat
make setup
5. Now we have a boilerplate setup to start with. The following files needed to be modified:
- _meta/beat.yaml — The template file from which the configuration is generated
- config/config.go — Struct defining the above config
- beater/fileoccurancebeat.go — Actual implementation resides here
6. Include our snippet as a function fileOccurences
and add it to the event
// Checks for a given file under a given path recurssivelyfunc fileOccurences(rootPath string, fileName string) int64 { var count int64 = 0 err := filepath.Walk(rootPath, func(path string, info os.FileInfo, err error) error { if err != nil { logp.Error(err) return err } if info.Name() == fileName { count++ } return nil }) if err != nil { logp.Error(err) return 0 } return count}
7. An option to provide the root directory and the file name from config needs to be included. Add them to config.go
and beat.yaml
template and run the build with mage
(yes not make
):
mage build
8. Now try to launch fileoccurencebeat
locally (Ctl+c to stop):
./fileoccurencebeat -e -d "*"
Conclusion and further reading
Now we have a working beat, in the next article we can test the beat metrics, plot graphs etc. with an elastic stack running locally.
References
- Beats developer guide from Elastic — https://www.elastic.co/guide/en/beats/devguide/7.2/index.html
- What are elastic beats — https://www.elastic.co/products/beats
- Github repository — https://github.com/cloudronics/fileoccurancebeat