The LogstreamerInput uses filepath.Walk() for locating files to compare to the file_match regexes.
This is normally fine in a mostly sane directory structure.
In the case of ridiculously large/complex filesystems with numerous levels containing millions of files, this can be somewhat inefficient.
The pattern matching is useful so not suggesting a replacement for regexing.
This change should still follow the same rules for log_directory and file_match.
I propose adding an additional config option to use globbing specifically for the filepath.Walk portion.
https://github.com/mozilla-services/heka/blob/dev/logstreamer/filehandling.go#L280
glob_pattern = "/this/*/path*/would/*take/*/*/*/forever/to/*/walk/*/in/a*/reasonable/*/*/*/timeframe/*"