Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
This release has three major changes over 0.3.3: * a durable mpsc is now used in place of std's mpsc * QOS is now guaranteed timestamp accurate * Kinesis Firehose sink is introduced In the situation of a very slow sink and a fast source cernan 0.3.3 and lower would allocate increasing amounts of memory. Obviously that is not desirable but cernan cannot drop telemetry on the floor. The solution is a durable mpsc which writes in-flight data to disk in a concurrency safe fashion. Extensive testing has been done to ensure safety of the durable mpsc. In cernan 0.3.3 and below QOS was inaccurate as points would be put into a timeless aggregation. A snapshot thread was added to simulate time but this was less than perfect. Now we store points with their timestamps in-aggregate and are able to do QOS elision with guaranteed precision. Finally, we introduce a Kinesis Firehose sink that will allow cernan to emit into Amazon's Firehose, forking from there to various AWS services, S3 and Elasticsearch of the most interest. At present this sink only ships LogLines but in the future when #39 lands we can choose to do more exciting things. Signed-off-by: Brian L. Troutwine <blt@postmates.com>
- Loading branch information