FLUME SOURCES
Avro Source
Listens on Avro port and receives events from external Avro client streams.
Property Name
|
Default
|
Description
|
type
|
-
|
The component type name, needs to be avro
|
bind
|
-
|
hostname or IP address to listen on
|
port
|
-
|
Port # to bind to
|
Exec Source
This source runs a given Unix command on start-up and expects that process to continuously produce data on standard out (stderr is simply discarded, unless logStdErr=true).
Property Name
|
Default
|
Description
|
type
|
-
|
The component type name, needs to be exec
|
command
|
-
|
The command to execute
|
restartThrottle
|
10000
|
Amount of tmie (in millis) to wait before attempting restart
|
restart
|
false
|
Whether the executed cmd should be restarted if it dies
|
logStdErr
|
false
|
Whether the commands stderr should be logged
|
Note: The ExecSource can not guarantee that if there is a failure to put the event into achannel, the client knows about it. In such cases, the data will be lost.
|
For example,
exec-agent.sources = tail
exec-agent.channels = memoryChannel-1
exec-agent.sinks = logger
exec-agent.sources.tail.type = exec
exec-agent.sources.tail.command = tail -f /var/log/secure
|
NetCat Source
A netcat-like source that listens on a given port and turns each line of text into an event. Acts
like nc -k -l [host] [port]. In other words, it opens a specified port and listens for data. The expectation is that the supplied data is newline separated text. Each line of text is turned into a Flume event and sent via the connected channel.
Property Name
|
Default
|
Description
|
type
|
-
|
The component type name, needs to be netcat
|
bind
|
-
|
Host name or IP address to bind to
|
port
|
-
|
Port # to bind to
|
max-line-length
|
512
|
Max line length per event body (in bytes)
|
Sequence Generator Source
A simple sequence generator that continuously generates events with a counter that starts from 0 and increments by 1. Useful mainly for testing.
Property Name
|
Default
|
Description
|
type
|
-
|
The component type name, needs to be seq
|
Syslog source
Reads syslog data and generate Flume events. The UDP source treats an entire message as a single event. The TCP source on creates a new event for a string of characters separated by carriage return (\n).
Syslog TCP
Name
|
Default
|
Description
|
type
|
-
|
The component type name, needs to be syslogtcp
|
host
|
-
|
Host name or IP address to bind to
|
port
|
-
|
Port # to bind to
|
For example, a syslog TCP source:
syslog-agent.sources = syslog
syslog-agent.channels = memoryChannel-1
syslog-agent.sinks = logger
syslog-agent.sources.syslog.type = syslogtcp
syslog-agent.sources.syslog.port = 5140
syslog-agent.sources.syslog.host = localhost
|
Syslog UDP
Property Name
|
Default
|
Description
|
type
|
-
|
The component type name, needs to be syslogudp
|
host
|
-
|
Host name or IP address to bind to
|
port
|
-
|
Port # to bind to
|
For example, a syslog UDP source:
