Plugin output

[lafeuil]

I would like to have your opinion on a new feature. Do you think you could agree to add a plugin system to handle output of passivedns ?

There is a new similar system in dnscap. In command line, you can pass plugins (.so) with -P argument. A plugin is a dynamic library. Each plugin can implement functions :

void <plugin_name>_usage();
void <plugin_name>_getopt(int *argc, char **argv[]);
void <plugin_name>_start();
void <plugin_name>_stop();

and must implement an output function :

void <plugin_name>_output(pdns_record *l, pdns_asset *p, ldns_rr *rr, ldns_rdf *lname, uint16_t rcode);

With this system, we can implement some outputs like a massive inject in a database with better performance than parsing log file.

Do you think you can integrate a patch with this feature ?

[thus]

If this is a feature you want, then you could write a patch, and send us a pull request :)

There is a million things I want to add to both passivedns and other projects, but time is a limiting factor.

[lafeuil]

I have created a first version of a plugin system to handle output of passivedns. You can see it in my repository : https://github.com/lafeuil/passivedns
You can see the change in this commit : lafeuil/passivedns@fc2ccde

The commit have an example with a JSON plugin output. It is a copy of the current output JSON implementation. You can build it with :

cd src/plugins/json
libtoolize && aclocal && autoconf && automake && ./configure && make

You can run passivedns with the -O parameter :

./src/passivedns -L - -l - -r dns-exemple.pcap -O
src/plugins/json/.libs/json.so -L - -l -

Tell me what do you thing about my patch.

I think the plugin output is a good idea when I see the new pull request #70 about Redis.

[maxtors]

Think this would be a good idea aswell, i felt that the code (when introducing Redis) became abit cluttered having to take into account the differet ifdef's and so on. A plugin system would make it easier to integrate new modules in a clean fashion perhaps?

But then again, i see that the proposed JSON output plugin logs to file, and the the Redis output would have to be implemented both here, and in the regular output format?

[thus]

@lafeuil: Nice! I haven't gotten around to take a proper look at the code yet, but I really like the concept :)

@maxtors: I don't see why the Redis output couldn't be implemented like this. Could you explain why?

[lafeuil]

@maxtors My implementation doesn't support more than one plugin. But If you want, I can change this and you can chain output plugins :

./src/passivedns -L - -l - -r dns-exemple.pcap -O src/plugins/txt/.libs/txt.so -L - -l - src/plugins/json/.libs/redis.so -h redis-host.com

The default output implementation can be implemented to a "txt" plugin and you can implement a "Redis" plugin.

Another solution is to provide two mode when we define a plugin in the command line :

• -o without default output
• -O with default output

If we want integrate this plugin feature, we must define how it work exactly. I think the json output must be implemented to a plugin.
But how the default output must be implemented ? In a plugin ? in the passivedns code ?

[maxtors]

My thought was that, the redis output (or say a SQL, Cassandra, Syslog, Kafka, what not) is more of a "where to output" not a "the format of the output", so the styling (formatters) like JSON or "default" should be seperated into perhaps "formatters" plugins?

Say if someone in the future wants to create a CEF formatter, or maybe a BSON output, but wants to use syslog on one server, but redis on the others.