Graphite rendering scales data incorrectly

Hmmm digging into the code I see that the TimeSeries class has a "consolidate" property that defaults to average... sounds promising, but how do I use it and change it to "sum"? Can someone point me to the documentation?

What's happening is that since there are only so many pixels in the graph, when there are two many points Graphite calls a consolidate function on the series to reduce the number of points to fit the graph. By default this is average but you can change this to sum using the 'consolidate' function which changes the property on the series: http://readthedocs.org/docs/graphite/en/latest/functions.html#graphite.render.functions.cumulative

You might also check out this bug which is related to this behavior: https://bugs.launchpad.net/graphite/+bug/886411

Closing this out as "As designed"

I will be out of the office and unreachable on 6/7 and 6/8. I will return to the office on 6/11.

Please reach out to the ESM team for any concerns. Thanks.

--Bryant

On May 25, 2012, at 7:42 PM, Michael Leinartas <email address hidden> wrote:

> Closing this out as "As designed"
>
> ** Changed in: graphite
>       Status: New => Invalid
>
> --
> You received this bug notification because you are subscribed to
> Graphite.
> https://bugs.launchpad.net/bugs/1004764
>
> Title:
>  Graphite rendering scales data incorrectly
>
> Status in Graphite - Enterprise scalable realtime graphing:
>  Invalid
>
> Bug description:
>  I have a bunch of sparse data being fed in from StatsD.  Because it's
>  counter data, StatsD logs a "0" to carbon when no data is received in
>  a given time period.  Therefore the time series might look like:
>
>  0, 0, 0, 0, 100, 0, 0, 0, 0 ...
>
>  When I go to render this, the graph looks like a series of spikes.
>  The problem is that the height of the spikes can change as I vary the
>  time period.  At 0 - 6 hours the height of the spikes will be 100, but
>  at 0 - 5 hours the height will be 50.  Is this a discretization issue?
>  Is the rendering engine trying to interpolate between values for
>  certain time periods?
>
>  The reason I think this is solely a rending issue is that the data is
>  correct for all time periods when I fetch data using the JSON API.
>
>  Some thoughts (if it is indeed the rendering that is doing some interpolation):
>   - Perhaps it's an application/StatsD issue.  ie: we shouldn't be logging "0" values when there is no data. That way graphite won't try and interpolate
>
>  - Maybe there should be an aggregationMethod variable for rendering,
>  similar to storage-aggregation.conf, which could be set to "sum" or
>  "average"
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/graphite/+bug/1004764/+subscriptions

I will be out of the office and unreachable on 6/7 and 6/8. I will return to the office on 6/11.

Please reach out to the ESM team for any concerns. Thanks.

--Bryant

On May 25, 2012, at 7:16 PM, Michael Leinartas <email address hidden> wrote:

> What's happening is that since there are only so many pixels in the
> graph, when there are two many points Graphite calls a consolidate
> function on the series to reduce the number of points to fit the graph.
> By default this is average but you can change this to sum using the
> 'consolidate' function which changes the property on the series:
> http://readthedocs.org/docs/graphite/en/latest/functions.html#graphite.render.functions.cumulative
>
> You might also check out this bug which is related to this behavior:
> https://bugs.launchpad.net/graphite/+bug/886411
>
> --
> You received this bug notification because you are subscribed to
> Graphite.
> https://bugs.launchpad.net/bugs/1004764
>
> Title:
>  Graphite rendering scales data incorrectly
>
> Status in Graphite - Enterprise scalable realtime graphing:
>  New
>
> Bug description:
>  I have a bunch of sparse data being fed in from StatsD.  Because it's
>  counter data, StatsD logs a "0" to carbon when no data is received in
>  a given time period.  Therefore the time series might look like:
>
>  0, 0, 0, 0, 100, 0, 0, 0, 0 ...
>
>  When I go to render this, the graph looks like a series of spikes.
>  The problem is that the height of the spikes can change as I vary the
>  time period.  At 0 - 6 hours the height of the spikes will be 100, but
>  at 0 - 5 hours the height will be 50.  Is this a discretization issue?
>  Is the rendering engine trying to interpolate between values for
>  certain time periods?
>
>  The reason I think this is solely a rending issue is that the data is
>  correct for all time periods when I fetch data using the JSON API.
>
>  Some thoughts (if it is indeed the rendering that is doing some interpolation):
>   - Perhaps it's an application/StatsD issue.  ie: we shouldn't be logging "0" values when there is no data. That way graphite won't try and interpolate
>
>  - Maybe there should be an aggregationMethod variable for rendering,
>  similar to storage-aggregation.conf, which could be set to "sum" or
>  "average"
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/graphite/+bug/1004764/+subscriptions